Data Engineer III
- Data Transformation and Integration: Extracts data from identified databases. Creates data pipelines and transform data to a structure that is relevant to the problem by selecting appropriate techniques. Develops knowledge of current analytics trends.
- Data Source Identification: Supports the understanding of the priority order of requirements and service level agreements. Helps identify the most suitable source for data that is fit for purpose. Performs initial data quality checks on extracted data.
- Data Modelling: Analyses complex data elements, systems, data flows, dependencies, and relationships to contribute to conceptual, physical and logical data models. Develops the Logical Data Model and Physical Data Models including data warehouse and data mart designs. Defines relational tables, primary and foreign keys and stored procedures to create a data model structure. Evaluates existing data models and physical databases for variances and discrepancies. Develops efficient data flows. Analyses data-related system integration challenges and proposes appropriate solutions.
- Code Development and Testing: Writes code to develop the required solution and application features by determining the appropriate programming language and leveraging business, technical and data requirements. Creates test cases to review and validate the proposed solution design. Creates proofs of concept. Tests the code using the appropriate testing approach. Deploys software to production servers. Contributes code documentation, maintains playbooks, and provides timely progress updates.
- Cloud Optimized solutions: Optimize cloud infrastructure costs by leveraging cloud-native services efficiently. Implement cost-effective data storage solutions (e.g., tiered storage, lifecycle policies). Monitor and analyze cloud usage patterns to identify cost-saving opportunities.
- Problem Formulation: Translates business problems within one's discipline to data related or mathematical solutions. Identifies what methods (for example, analytics, big data analytics, automation) would provide a solution for the problem. Shares use cases and gives examples to demonstrate how the method would solve the business problem.
- Applied Business Acumen: Provides recommendations to business stakeholders to solve complex business issues. Develops business cases for projects with a projected return on investment or cost savings. Translates business requirements into projects, activities, and tasks and aligns to overall business strategy. Serves as an interpreter and conduit to connect business needs with tangible solutions and results. Recommends new processes and ways of working.
- Demonstrates up-to-date expertise and applies this to the development, execution, and improvement of action plans by providing expert advice and guidance to others. Supporting and aligning efforts to meet customer and business needs and building commitment for perspectives and rationales.
- Front-End & full stack knowledge: Collaborate closely with UI developers to integrate data pipelines into web applications. Ensure seamless data flow between the front-end and back-end systems. Optimize data retrieval and presentation for user interfaces. Understand and familiarized on the entire technology stack, including both front-end (UI) and back-end components. Next Gen Data Ingestion Techniques: Design systems that efficiently ingest data from various sources (databases, APIs, streaming platforms) using next gen tools based on Kubernetes etc
- 3-5 years of Data engineering experience
- You have consistently high standards, your passion for quality is inherent in everything
- Well versed with Hadoop, Hive, Spark using Scala, Kubernetes, Cloud, API and Data Lake concepts .
- You have a proven track record coding with at least one programming language (e.g., Java, Python)
- You are experienced in computing platforms (e.g., GCP, Azure)
- You are familiar and have basic knowledge in front-end technologies such as HTML, CSS, JavaScript, React, Angular etc and back-end technologies such as Python, Java, Node.js etc
- You have basic knowledge of RESTful APIs and microservices architecture.
- You have basic knowledge in Kubernetes to manage data pipelines, ensuring scalability, resilience, and efficient resource utilization.
- You evangelize an extremely high standard of code quality, system reliability, and performance
- You're skilled in data modelling & data migration protocols
- Experience with Kafka connect, Druid, Big Query and Looker is added advantage.
- Experience with the integration tools like Automic, Airflow is nice to have
- Stock
Recommended Jobs
Lot Attendant/Porter
At California Automotive Retail Group, we are committed to an environment where the customer is always treated with respect and dignity. Our associates are our most valuable resource and growth is enc…
Project Engineer Marine & Heavy Civil Construction
We are seeking a skilled and driven Project Engineer to join our Long Beach, CA team. This role supports marine and heavy civil construction projects , focusing on field coordination, production …
Director, Artist D2C Strategy - Santa Monica, 90404
Director, Artist D2C Strategy - Santa Monica, 90404, United States of America Famehouse, a division of UMG, is the preeminent leader in D2C solutions in music, defining & delivering the industry’s b…
Technical Program Manager -- New Product Introduction (NPI)
Job Overview Our Marketing department is seeking for a dedicated and passionate individual to join our team as a Digital Project Manager. As a Digital Project Manager, you are required to perfo…
AWS Security Engineer
Make a difference here. UltraViolet Cyber is a leading platform-enabled unified security operations company providing a comprehensive suite of security operations solutions. Founded and operated b…
Collections Specialist, Accounts Receivable
Description Position at Global Music Rights, LLC Accounts Receivable | Collections Specialist Global Music Rights is a music rights management company focused on licensing, surveying…
Housekeeper
Get hired for Nayereh's housekeeper Job in Gold River, CA. Drive may daughter to her activities. Find housekeeper care work in Gold River.
💻 Sales Representative - Flexible Hours - Work from Home
Location: Remote Position/Work from Home Job Type: Full-Time/Part-Time Compensation: 100% Commission About Us: GIA Legacy Planning is a dynamic, client-focused insurance agency that s…
Customer Support Manager
Are you customer-focused, personable, resourceful, and looking to start your career with a leading provider in customized advertising, sales, and brand awareness strategies? We are one of the top p…
Registered Veterinary Technician (RVT)
~ Vacaville Animal Hospital is under NEW LEADERSHIP, and we are looking for a motivated RVT to join our new team as we grow our business. Part-time to full-time, starting pay $22 - $26 per hour, depen…