Member of Technical Staff (Software Engineer, Data Platform)
ABOUT THE ROLE The Data Platform team owns the end-to-end data lifecycle at Perplexity, from ingestion through processing, storage, and serving, powering product features, analytics, experimentation, AI workloads, and the company’s data lake. The team defines the architecture for batch and streaming systems, the orchestration and observability stack, and a self-serve data platform, while thoughtfully combining platforms such as Databricks and Snowflake with open-source technologies including Spark, Kafka, Flink, Airflow, Dagster, dbt, Iceberg, Delta Lake, and ClickHouse. In this senior/staff role, you will shape architecture, set standards, and drive the long-term technical direction of Perplexity’s data ecosystem. KEY RESPONSIBILITIES - Design and operate large-scale batch and streaming data pipelines that directly power Perplexity product features, AI training and evaluation workflows, analytics, and experimentation. - Build event-driven and streaming systems (Kafka, Kinesis, PubSub, or similar) for real-time ingestion, transformation, and delivery, alongside batch frameworks for backfills, aggregations, and offline computation. - Lead the architecture of data orchestration using tools like Airflow or Dagster, owning scheduling, dependency management, retries, SLAs, and end-to-end observability for critical data flows. - Set and enforce guarantees for data correctness, freshness, lineage, and recoverability, designing systems that handle rapid scale growth, partial failures, and evolving schemas without disrupting AI workloads or product experiences. - Build self-serve data platforms that let engineers, data scientists, and analysts safely discover data, define contracts, and create and operate their own pipelines with minimal friction. - Improve developer experience through better abstractions, opinionated paved paths, and standards for data modeling, testing, validation, and deployment, treating the data platform as a product used by many teams. - Drive architectural decisions across storage, compute, orchestration, and data APIs, partnering closely with product engineering and data science to align the data ecosystem with Perplexity’s roadmap. - Mentor engineers, review designs, and raise the technical bar for data infrastructure through thoughtful feedback, documentation, and hands-on collaboration. QUALIFICATIONS - 5+ years (Senior) or 8+ years (Staff) of software engineering experience. - Strong experience building production data infrastructure systems. - Hands-on experience with batch and/or streaming data processing at scale. - Deep familiarity with data orchestration systems (Airflow, Dagster, or similar). - Proficiency in Python and at least one additional backend language (Go, TypeScript, etc.). - Strong systems thinking around reliability, latency, cost, and complexity tradeoffs. - Experience supporting ML/AI workflows, training pipelines, or evaluation systems. - Familiarity with data quality, lineage, observability, and governance tooling. - Prior ownership of internal platforms used by many teams. If you’re excited about this role, we encourage you to apply even if your experience doesn’t match every qualification listed above.
Recommended Jobs
Specialty Medical Director (Board Certified / Diplomate / Residency Trained) J772574
Unlock a premier opportunity in veterinary medicine with a role offering up to $360,000 annually. Enhance your career in a supportive environment where your expertise in specialty veterinary care will…
Youth Mentoring Sports Coach at Boys & Girls Club of the Foothills
Job Description Job Description Salary: About Coach Across America Founded in 2010, Coach Across America is a national nonprofit organization focused on the power of coaches to transform th…
Travel Nurse RN - Cardiac Catheterization Lab - $2,070 per week in National City, CA
Registered Nurse (RN) | Cardiac Catheterization Lab Location: National City, CA Agency: OneStaff Medical Pay: $2,070 per week Shift Information: Rotating - 4 days x 10 hours Co…
Travel Nurse RN - House Supervisor in Salinas, CA
Registered Nurse (RN) | House Supervisor Location: Salinas, CA Agency: Medical Edge Recruitment Pay: Competitive weekly pay (inquire for details) Shift Information: Days Contra…
Senior Analyst - BCG Vantage, B2B SaaS - San Francisco
Senior Analyst - BCG Vantage, Technology, Media, and Telecommunications - San Francisco, United States of America Locations : San Francisco | New York Who We Are Boston Consulting Group par…
Administrative Assistant
We a seeking an Administrative Assistant to join a non-profit organization in Pauma Valley, CA to support their office operations. Temp to Hire, $22-$23/hr DOE Part-Time: 25 hours per week S…
Condition Assessment Project Manager
The Opportunity Black & Veatch has an exciting opportunity for a Project Manager to join our asset engineering team in southern California. As part of our Infrastructure Advisory Solution team, th…
Survey Technician - Construction & Surveying
Job description: Are you a detail-oriented survey professional with a passion for precision and data? Join a leading engineering and construction support team as a Survey Technician , where your…
Salesforce Adoption Administrator (Fresno)
Part-Time Hybrid Salesforce Digital Adoption Administrator This Jobot Consulting Job is hosted by: Sunshine Pennington Are you a fit? Easy Apply now by clicking the Apply button and sending us …
Get paid for taking free surveys with Unlock Surveys
Engage in surveys and discussions to earn rewards! Join Unlock's dynamic community of survey panellists today and begin earning rewards for sharing your valuable opinions. Formerly known as Springbo…