Job: Lead Spark Developer
Start Date: Two weeks after acceptance
Team: In store pick and Item Data Management
Skills Required:
– Minimum 3 years of experience using Apache Spark, Kafka
– Minimum 3 years of experience with cloud database like SQL, and
Cosmos DB
– Minimum 3 years of experience with streaming and structured streaming
data
– Strong in Hadoop, HIVE, SQL, Shell scripting, PySpark and Python
– Experience with pre-processing data and development of complex
business rules in Python or Scala
– Experience with design and implementation of ETL/ELT framework for
complex warehouses/marts
– Experience with CSV, JSON, AVRO, Parquet and JDBC/ODBC connections
– Experience with atomic transactions, consistent updates using Delta
Lake, Hudi or Iceberg
– Knowledge of large data sets and experience with performance tuning
and troubleshooting
– Hands-on development, with a willingness to troubleshoot and solve
complex problems
– Strong problem- determination skills and resolution experience.
– Excellent teamwork and communication skills (written and verbal)
– Bachelor?s degree in Computer Science or related analytical field or
equivalent experience
Special Requirements:
– Hands-on experience in Databricks , preferred
– Understanding of common connections wasbs, adls, abfss, S3 , preferred
– Power BI and Tableau , preferred
– Understanding of Data Modeling concepts , preferred
– CI/CD exposure, preferred
Work Description:
– Defining and scripting private IP address spaces via Vnets, and subnets
– Defining and scripting Internet communications (Plan for inbound and
outbound connectivity)
– Defining and scripting communication between Azure resources via
service endpoints, private endpoints and DNS
– Ability to troubleshoot and identify issues that could lead to
performance degradation
– Collaborate with the feature development, architectures, and
operations/support team to share training and best practices
– Configure DevOps CI/CD pipeline for development
Reference : LEAD Hadoop/Spark/BIG DATA Developer jobs
Source: http://jobrealtime.com/jobs/technology/lead-hadoopsparkbig-data-developer_i14940
Jobs Real Time 