Staff Software Engineer, Machine Learning

Staff Software Engineer, Machine Learning

Location: Burlingame, CA (On-site, 4 days a week)

Compensation: $145,000 -$250,000 (Base Salary) + Equity Options

We are a rapidly growing AI company applying Large Language Models (LLMs) to transform a critical, high-impact domain within the healthcare industry . We're looking for a highly skilled and experienced Staff Software Engineer, Machine Learning to drive the architecture and scaling of our LLM backend systems.

This role is ideal for an engineer who thrives at the intersection of backend architecture and applied AI, designing the APIs, data pipelines, and infrastructure that make LLMs reliable, secure, and cost-efficient in production. Join us and help push LLMs beyond demos into mission-critical healthcare workflows .

Job Description 🖥️

As a Staff Software Engineer, you’ll be a technical leader responsible for the end-to-end delivery of our core LLM backend projects, setting the engineering best practices, and mentoring peers in system design.

Key Responsibilities:

• Backend for LLMs: Architect and implement scalable, low-latency APIs and services that wrap, orchestrate, and optimize LLMs for complex, regulated use cases.
• Data & Retrieval Pipelines: Build ingestion, preprocessing, and Retrieval-Augmented Generation (RAG) pipelines to ground LLMs in specialized clinical and revenue-cycle data.
• LLMOps & Observability: Design systems for model monitoring, evaluation, cost tracking, and guardrails, ensuring reliability and responsible use in a production environment.
• Performance & Optimization: Engineer solutions for caching, batching, load balancing, and scaling LLM workloads across cloud and containerized environments.
• Security & Compliance: Implement HIPAA-ready infrastructure , data governance, and auditability for all LLM-powered applications.
• Cross-Functional Collaboration: Partner closely with product managers, ML engineers, and domain experts to translate business workflows into robust backend systems.
• Technical Leadership: Drive the technical direction, establish engineering excellence, and mentor team members in advanced system design.

• Experience: 5+ years of backend or full-stack software engineering experience, with 3+ years specifically working on ML/LLM-enabled applications .

• Coding Skills: Strong expertise in Python (and ideally one statically typed language such as Go, Java, or TypeScript).
• LLM Ecosystem: Experience with LLM integration frameworks (e.g., Hugging Face, LangChain, LlamaIndex, OpenAI APIs, Anthropic).
• Architecture: Deep knowledge of distributed systems , service-oriented architecture, and building APIs at scale.
• Cloud Native: Hands-on expertise with cloud-native technologies (AWS/GCP/Azure), Kubernetes, Docker, and infrastructure-as-code (e.g., Terraform).
• Practices: Familiarity with MLOps/LLMOps practices: CI/CD for models, robust evaluation harnesses, monitoring, and reproducibility.
• System Design: Excellent system design skills and the ability to align complex technical architecture with critical product goals.

Preferred Qualifications:

• Experience applying LLMs in healthcare or other highly regulated industries (e.g., familiarity with FHIR, HL7, HIPAA).
• Hands-on experience with advanced RAG pipelines , vector databases, and structured-output orchestration.
• Background in enterprise SaaS or mission-critical platforms where uptime, low-latency, and scale are paramount.

• High-Impact Work: The unique opportunity to apply cutting-edge Generative AI to one of the world's most important and complex industries—healthcare.
• Leadership Ownership: A key leadership role with significant ownership over core ML/LLM systems and direct influence on the company's technical direction.
• Financial Package: Competitive base salary, generous equity options, and comprehensive benefits.
• Health & Wellness: Full medical, dental, and vision coverage.
• Retirement: Access to a 401k plan.
• Time Off: Flexible and Unlimited PTO policy.
• Location & Collaboration: Work in our Burlingame, CA office four days a week, fostering deep collaboration with a dedicated, low-ego, and adaptive team.
• Visa Sponsorship: Visa sponsorship is available for this position.

Due to the high volume of applications we anticipate, we regret that we are unable to provide individual feedback to all candidates. If you do not hear back from us within 4 weeks of your application, please assume that you have not been successful on this occasion. We genuinely appreciate your interest and wish you the best in your job search.

Commitment to Equality and Accessibility:

At MLabs, we are committed to offer equal opportunities to all candidates. We ensure no discrimination, accessible job adverts, and providing information in accessible formats. Our goal is to foster a diverse, inclusive workplace with equal opportunities for all. If you need any reasonable adjustments during any part of the hiring process or you would like to see the job-advert in an accessible format please let us know at the earliest opportunity by emailing [email protected].

MLabs Ltd collects and processes the personal information you provide such as your contact details, work history, resume, and other relevant data for recruitment purposes only. This information is managed securely in accordance with MLabs Ltd’s Privacy Policy and Information Security Policy, and in compliance with applicable data protection laws. Your data may be shared only with clients and trusted partners where necessary for recruitment purposes. You may request the deletion of your data or withdraw your consent at any time by contacting [email protected].