Senior Lead Data Engineer - R01557349
Senior Lead Data Engineer
Primary Skills
- Architect, construct, test and maintain ELT pipelines using Hadoop and Spark SQL
Specialization
- Hadoop and Spark SQL
Job requirements
- Key Responsibilities:
- Architect, construct , test and maintain ELT pipelines using Hadoop and Spark SQL
- - Manage and optimize the infrastructure ensuring reliability and performance
- -Design and implement data models and schemas for enterprise data warehouse and data marts
- -Write and maintain scripts in Python for various applications
- -Develop, optimize, troubleshoot complex SQL queries and database structures for relational and non-relational databases
- - Work closely with analysts and product managers to understand data requirements and deliver high-quality data
- -Provide technical guidance and mentorship to other team members fostering best practices in data engineering
- -Monitor system performance, identify bottlenecks, implement solutions to ensure high data availability
- -High proficiency in the Hadoop ecosystem (HDFS, MapReduce, Yarn) and Apache Spark, Scala for large scale data processing
- -Good knowledge in Python for data manipulation and scripting
- -Mastery of advanced SQL and deep experience working with relational databases and datawarehouses
- -Hands-on experience with workflow management tools like Apache Airflow
- -Familiarity with Unix shell scripting for automation and system management
Required Skills:
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
Recommended Jobs
Technology Lead 22821-1
Apply now: Technology Lead, Location is Hybrid (Burbank, CA). The start date is September 23, 2025, for this 11-month contract position with potential extension. Job Title: Technology Lead Loc…
Systems Engineer
Systems Engineer Overview We are seeking a motivated and well-qualified Systems Engineer to join a high-performing team supporting critical defense and technology programs. This position offers…
Acquisitions Assistant
Searchlight Pictures is a global specialty film and television company that develops, produces, finances and acquires motion pictures and series for both worldwide theatrical and streaming releases. I…
Manager Trainee (35942)
The Manager In Training program provides hands-on training in Sales, Operations, Customer Service, finance, fleet management, and people management. This position has a clearly defined career path …
San Diego - Cook & Prep-Cook
NOW HIRING: Cooks & Prep Cooks Do you thrive behind the line? Love the rush of event-based kitchens? Join The Party Staff and bring your culinary skills to a variety of exciting events across the …
Sr. Data Scientist, Revenue
Our Mission Launched in 2012, Tinder® revolutionized how people meet, growing from 1 match to one billion matches in just two years. This rapid growth demonstrates its ability to fulfill a fundame…
Hematology Oncology Nurse Practitioner
Palm Health is hiring an experienced Hematology Oncology Physician Assistant or Nurse Practitioner to join our dynamic health care team in sunny Palm Springs, CA! Outstanding Base salary! Â $10,000 …
Senior/Staff Site Reliability Engineer
Who we are Gatik, the leader in autonomous middle-mile logistics, is revolutionizing the B2B supply chain with its autonomous transportation-as-a-service (ATaaS) solution and prioritizing safe, …
Release Engineer
Veeva Systems is a mission-driven organization and pioneer in industry cloud, helping life sciences companies bring therapies to patients faster. As one of the fastest-growing SaaS companies in histo…
Sr. Product Manager, Content Safety
The Content Safety product and engineering team is responsible for ensuring that Pinterest is the safest corner of the internet. We’re responsible for evaluating all content on the platform to ensure…