Senior AI Data and Evaluation Engineer

What You Will Do:
We are looking for an experienced and highly skilled Senior AI Data and Validation Engineer. A successful candidate will be responsible for both dry and wet lab experiments for AI functionality acquiring datasets, developing labeling guidelines, and preparing dataset for AI model development. Also, candidate will be responsible for developing methods and processes for scientific evaluation of AI/ML models used in medical devices, ensuring compliance with all applicable FDA guidelines and regulations. This role is crucial for maintaining the safety, effectiveness, and quality of AI-enabled medical devices throughout their product lifecycle from initial training to final validation. The engineer will contribute to the design, execution, and analysis of validation studies, ensuring that the devices perform predictably and reliably for their intended use and across all relevant demographic groups., placing a strong emphasis on comprehensive data management practices, including data collection, processing, annotation, storage, and use. This includes ensuring data integrity, quality, and representativeness to mitigate potential biases and demonstrate the safety, effectiveness, and quality of AI-enabled medical devices throughout their lifecycle.

What you will do:

• Perform dataset collection, develop, synthesis, generation, preparing datasets for AI development, stress testing and validation.
• Execute web lab experiments that interact with human blood samples with good laboratory practice.
• Create detailed validation plans, protocols, and reports for AI-enabled medical device software functions encompassing both performance validation and model evaluation.
• Perform Data Operation such as management of data used in both the development and validation of AI models, ensuring the quality, diversity, and independence of datasets . This involves assessing data representativeness, addressing potential biases, and ensuring the appropriate separation of training and test datasets
• Conduct comprehensive risk assessment for AI models, considering potential and unintended biases, limitations, and cybersecurity vulnerabilities
• Conduct rigorous performance validation studies, analyzing performance across different subgroups of the intended user population .
• Perform analysis to address transparency and bias throughout the device lifecycle, from design to post-market surveillance . This includes evaluating whether a device benefits all relevant demographic groups similarly and incorporating transparency into device design
• Stay abreast of the latest FDA guidelines, regulations, and industry best practices related to AI model validation in medical devices.

Minimum Qualifications (Required):

• Bachelor's Degree in Computer Science, Machine Learning, Electrical Engineering, Mathematics, Statistics, Bioengineering or related field, 2+ years of work experience required.
• OR Master's Degree in the above fields

Preferred Qualifications (Strongly desired):

• 2+ years of experience in medical device R&D, preferably AI-enabled devices.
• Fluent in commercial data analysis tools or similar scripting and statistical language for data analysis, Python preferred.
• Experience in the development, evaluation and validation of AI models, specifically within the medical device or healthcare industry. This experience including one or more of the following R&D life cycle: Data Acquisition and preprocessing and labeling, model training and tuning, AI Model evaluation and performance, bias detection and mitigation, risk assessment, and QMS documentation
• Experience in Statistics and descriptive data analysis to help develop data-driven scientific rigor to AI model validation process, including experience with one or more of the following: confidence intervals development, power analysis, statistical tests, etc.
• Entry level knowledge with AI/ML frameworks such as PyTorch, OpenCV, TensorFlow, scikit-learn etc. for model training and development, testing and validation.
• Experience with commercial Data Operation tools
• General knowledge of the healthcare market and competitors.

• $116k - $193k salary plus bonus eligible + benefits. Individual pay is based on skills, experience, and other relevant factors.