Azure Data Engineer (San Francisco)

Direct message the job poster from GBIT (Global Bridge InfoTech Inc)

Sr. US IT Recruiter / Bench Sales at Global Bridge InfoTech Inc

Location : Remote ||SFO, CA preferred.

Contract : Long term

Job Summary :

We are seeking a highly skilled Azure Data Engineer to join our data engineering team. The ideal candidate will have strong experience in designing, building, and maintaining scalable data pipelines and infrastructure on cloud platforms like Azure and AWS. You will be responsible for developing robust ETL processes, working with big data technologies such as Hadoop and Spark, and contributing to system design for real-time analytics and fault-tolerant architectures.

Key Responsibilities :

• Design, build, and maintain ETL / ELT pipelines using modern orchestration tools like Luigi Azure Data Factory , or similar.
• Develop scalable data lakes and data ingestion frameworks leveraging Azure Data Lake Storage Databricks , or AWS S3
• Work closely with cross-functional teams to support real-time data processing and streaming analytics solutions.
• Implement and optimize big data solutions using technologies like Hadoop Apache Spark , and Delta Lake
• Design and implement secure, scalable cloud infrastructure using Azure (preferred) and AWS services for data storage and processing.
• Ensure robust access control and security policies across cloud resources using RBAC IAM network security groups , etc.
• Participate in system architecture discussions focused on fault tolerance high availability , and performance optimization
• Automate infrastructure and deployments using CI / CD practices and Infrastructure as Code (IaC) tools (e.g., Terraform, ARM templates).
• Monitor and troubleshoot pipeline and infrastructure issues to ensure data reliability and integrity.

Required Qualifications :

• Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or related field.
• 4+ years of experience in data engineering , with at least 2+ years on Azure and / or AWS
• Strong programming skills in Python Scala , or Java for building ETL workflows.
• Hands-on experience with Luigi Airflow , or other pipeline orchestration tools.
• Solid understanding of cloud access controls identity management , and security best practices
• Deep knowledge of big data technologies Hadoop, Spark, Hive, and related ecosystems.
• Experience in real-time analytics and streaming data processing using Kafka, Spark Streaming, or similar.
• Proven track record of contributing to the design of scalable, fault-tolerant architectures in cloud environments.

Preferred Skills :

• Experience with data modeling data governance , and metadata management
• Familiarity with containerization (Docker, Kubernetes) and DevOps practices.
• Exposure to machine learning pipelines or analytics platforms is a plus.

Seniority level

Seniority level

Mid-Senior level

Employment type

Employment type

Contract

Job function

Job function

Information Technology

Industries

IT Services and IT Consulting

Referrals increase your chances of interviewing at GBIT (Global Bridge InfoTech Inc) by 2x

Inferred from the description for this job

Medical insurance

401(k)

Vision insurance

Paid maternity leave

Get notified when a new job is posted.

Sign in to set job alerts for Data Engineer roles.

San Francisco, CA $160,000.00-$180,000.00 2 weeks ago

Foster City, CA $192,000.00-$260,000.00 1 week ago

San Francisco, CA $182,000.00-$232,000.00 1 day ago

San Francisco, CA $207,000.00-$279,000.00 6 days ago

San Francisco, CA $149,998.00-$250,000.00 5 days ago

San Francisco, CA $112,597.33-$152,810.66 5 days ago

San Mateo, CA $120,000.00-$160,000.00 2 weeks ago

San Francisco, CA $145,000.00-$165,000.00 2 weeks ago

San Francisco, CA $255,000.00-$405,000.00 2 days ago

San Francisco, CA $130,000.00-$238,000.00 2 weeks ago

San Francisco, CA $150,000.00-$200,000.00 2 weeks ago

San Francisco, CA $140,000.00-$180,000.00 3 days ago

San Francisco, CA $120,000.00-$150,000.00 2 months ago

San Francisco, CA $180,000.00-$220,000.00 2 weeks ago

San Francisco, CA $177,500.00-$210,000.00 2 months ago

San Francisco, CA $90,000.00-$150,000.00 1 month ago

San Francisco, CA $150,000.00-$260,000.00 5 months ago

San Francisco, CA $125,640.00-$174,500.00 5 days ago

We're unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.

J-18808-Ljbffr