Founding Backend Engineer - LLM Orchestration (San Francisco)

Join to apply for the Founding Backend Engineer - LLM Orchestration role at Nestmed

3 days ago Be among the first 25 applicants

Join to apply for the Founding Backend Engineer - LLM Orchestration role at Nestmed

Get AI-powered advice on this job and more exclusive features.

This range is provided by Nestmed. Your actual pay will be based on your skills and experience talk with your recruiter to learn more.

Base pay range

$300,000.00/yr - $360,000.00/yr

About Nestmed

Nestmed is redefining post-acute healthcare with AI-driven technology that helps clinicians work more efficiently and provide better patient care.

Within one year, we've processed over half a million patient visits, with tens of thousands of clinicians using our product daily. We're now working with 7 of the top 10 post-acute healthcare enterprises in the U.S., helping shape the future of home healthcare delivery.

Founded by Stanford and YC alumni with deep healthcare and AI expertise, our founding team combines years of clinical experience with cutting-edge technical backgrounds from companies like Amazon, Google, Meta, and leading healthcare organizations. Backed by top investors including SciFi VC (Max Levchin, PayPal co-founder) and Mischief Capital (Plaid founder), we're building the next generation of healthcare infrastructure.

About The Role

As the founding Backend Engineer on our LLM Orchestration team, you'll be deploying and managing LLMs at scale, learning how to orchestrate them in complex production scenarios that directly impact patient care. You'll rebuild and maintain our core AI inference engine that powers all of Nestmed's intelligent capabilities across several thousand clinical conversations daily.

Our system orchestrates over a dozen different AI models - both fine-tuned in-house models and third-party APIs - with low latency and high availability. You'll work on complex technical challenges like intelligent model routing based on clinical context, implementing sophisticated fallback strategies across multiple providers, optimizing inference costs through batching and caching, and ensuring clinical accuracy through comprehensive model evaluation pipelines.

This isn't about calling OpenAI APIs. You'll build sophisticated orchestration logic that selects optimal models for each clinical task, implements custom retry and circuit breaker patterns for provider failures, manages rate limits across multiple concurrent workflows, and maintains detailed performance metrics across the entire AI pipeline. You'll start as the solo engineer on this critical infrastructure and grow it into a robust team handling core AI engineering.

What You'll Do

• Build and optimize our core AI inference engine that routes requests across multiple LLM providers based on clinical context, cost optimization, and latency requirements
• Design robust model serving infrastructure with intelligent load balancing, failover mechanisms, and A/B testing frameworks for model evaluation in production
• Implement production-grade AI pipelines with comprehensive observability, distributed tracing, and real-time performance monitoring for healthcare-critical workloads
• Optimize inference costs and latency through intelligent request batching, response caching, model quantization, and dynamic provider selection algorithms
• Build custom model fine-tuning and deployment pipelines for healthcare-specific tasks using frameworks like Transformers, vLLM, and distributed training infrastructure
• Create sophisticated prompt engineering systems that dynamically optimize prompts based on clinical context and historical model performance data
• Design comprehensive evaluation frameworks that continuously monitor model accuracy, clinical safety, and regulatory compliance across all deployed models
• Build model versioning and deployment systems that support safe rollouts, instant rollbacks, and controlled experimentation in production healthcare environments

What You Bring

• 6+ years of backend engineering experience building high-performance distributed systems, with focus on latency-critical applications and reliability engineering
• Deep production experience with LLMs including multi-provider orchestration, custom model serving, and building reliable inference infrastructure at scale
• Strong expertise in ML infrastructure including model serving frameworks (TensorRT, vLLM, TorchServe), distributed training, and GPU optimization
• Experience with model evaluation and monitoring including A/B testing frameworks, performance monitoring, and building comprehensive observability for ML systems
• Proficiency in Python and ML frameworks with hands-on experience in model fine-tuning, prompt engineering, and deploying custom models to production
• Track record scaling ML systems with experience optimizing inference costs, managing multiple model providers, and building reliable AI infrastructure
• Understanding of healthcare or regulated industries where model accuracy, auditability, and compliance are mission-critical requirements
• San Francisco-based and excited about working closely with AI researchers to productionize cutting-edge models for healthcare applications

Why This Role Matters

You'll be building the AI infrastructure that processes millions of patient interactions, directly impacting care quality for thousands of patients daily. Every optimization you make reduces healthcare costs, improves clinical accuracy, and enables new AI capabilities that transform patient outcomes.

You'll start as the founding ML infrastructure engineer and build this into a world-class AI platform team. Join us in San Francisco to build the most sophisticated LLM orchestration system in healthcare alongside leading AI researchers and clinical experts.

If youre passionate about building high-impact products that solve real-world problems, wed love to hear from you. Apply today!

Compensation Range: $300K - $360K

Seniority level

• Seniority level

  Not Applicable

Employment type

• Employment type

  Full-time

Job function

• Job function

  Engineering and Information Technology

• Industries

  Hospitals and Health Care

Referrals increase your chances of interviewing at Nestmed by 2x

Get notified about new Back End Developer jobs in San Francisco, CA .

San Francisco, CA $160,000.00-$180,000.00 2 days ago

San Francisco, CA $130,000.00-$238,000.00 11 hours ago

San Francisco, CA $150,000.00-$250,000.00 3 weeks ago

Full-Stack Software Engineer (Jr/Mid level)

San Francisco, CA $120,000.00-$180,000.00 1 month ago

San Francisco, CA $99,500.00-$200,000.00 2 weeks ago

San Francisco, CA $150,000.00-$230,000.00 4 months ago

San Francisco, CA $180,000.00-$280,000.00 1 day ago

Software Development Engineer I - Frontend & Mobile

San Francisco, CA $99,500.00-$200,000.00 2 weeks ago

Software Engineer Intern, Frontend - Fall 2025

San Francisco, CA $56.25-$137,000.00 3 days ago

San Francisco, CA $160,000.00-$200,000.00 2 months ago

San Francisco, CA $150,000.00-$176,000.00 3 months ago

San Francisco, CA $120,000.00-$190,000.00 9 months ago

San Francisco, CA $130,000.00-$140,000.00 2 weeks ago

Software Engineer, AI Intern (Summer 2026)

San Francisco, CA $125,000.00-$175,000.00 2 months ago

Software Engineer, AI Intern (Winter 2026)

San Francisco, CA $130,000.00-$240,000.00 2 weeks ago

San Francisco, CA $163,200.00-$223,200.00 2 days ago

Software Engineer, Frontend (All Levels)

San Francisco, CA $150,000.00-$220,000.00 2 weeks ago

San Francisco, CA $99,500.00-$200,000.00 2 weeks ago

San Francisco, CA $140,000.00-$280,000.00 8 months ago

San Francisco, CA $155,000.00-$339,500.00 2 weeks ago

San Francisco, CA $190,000.00-$250,000.00 2 weeks ago

San Francisco, CA $165,000.00-$165,000.00 2 years ago

Were unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.

#J-18808-Ljbffr