Data Engineer
Data Engineer:
Our client is Delivering the most comprehensive identity insights, our client’s platform equips businesses with fully automated KYB (Know your Business) solutions for Risk, and Fraud management, setting new standards in business verification.
The solution is designed for FS institutions to emphasize their interest in a coordinated effort to mitigate B2B fraud, reduce the risk associated with working with small businesses, and create a centralized, privacy-compliant entity for data-sharing between financial institutions.
Looking for a data engineer to assist with data collection across a variety of fragmented data sources available from both government, public, and private databases.
Responsibilities:
- Designing the database schema
- Migrating production tables
- Working with large scale data in the terabyte range
- Maintaining operational efficiency of database
- Normalizing disparate schemas to a single unified scheme
- Abstracting reusable components
- Strength in minimizing the amount of new code for new pipelines and instead creating internal packages that allow a high level of reusability
Experience with:
- ETL (extract, transform, load)
- Database design
- Primary key, foreign key
- Indexing
- Partitioning
- Access patterns
- Migrations
- Data pipelining
- Core data concepts: ACID transactions, Idempotency, Orchestration
Technologies:
- Airflow
- Google Cloud Platform (GCP)
- GCP Dataflow (aka Apache Beam)
- PostgreSQL
- Python
- Pydantic (a python library)
- Distributed systems
Recommended Jobs
Customer Service Representative
Job Overview We are looking for an efficient and skilled Customer Service Representative for our call center. As a Customer Service Representative, your job is to provide excellent customer suppor…
Sales Ticket Writer / Sales Associate
Job Details: ~$500 to $1,000 Sign on Bonus. ~ Hourly plus commission and incentives. ~ Medical, Dental, Vision, Critical Illness & Accident Insurance Plans. ~401k with Employer Matching. ~ …
Clinical Psychologist
We are hiring a Clinical Psychologist to join our team of talented medical professionals to conduct a one-time, non-treatment Veteran Disability Exam in Fresno, CA . ~ Employment Status: Part-T…
Explore Adventure: RN Role in Stunning Fresno!
Registered Nurse - Labor & Delivery - Travel - (LD RN) Embark on an adventurous journey as a Labor and Delivery Registered Nurse in stunning Fresno! Join a dedicated healthcare team in a state-of-the…
Full-Stack Software Engineer, Intelligent Support Engineering
About the Team The Intelligent Support Engineering team is scaling OpenAI with OpenAI. We apply our latest models, technologies, and products (some external, some still in the lab) to real-world p…
House Supervisor - Nursing Administration
Overview Paradise Valley Hospital is the South Bay’s oldest hospital, the second oldest hospital in all of San Diego County, and the largest employer in National City. For more than 100 years, Pa…
Recruiter | HR Specialist - Human Resources
Channel Factory is in search of a Recruiter|HR Generalist to join our growing team. This hybrid role will be instrumental in driving both recruitment efforts and supporting core HR functions, with a …
Sales Manager
Description The Sales Manager is responsible for overseeing and directing all aspects of the automotive sales department. This role involves leading a team of sales representatives to achieve ma…
DISHWASHER (FULL TIME)
We are hiring immediately for full time DISHWASHER positions. Location : Catalina Island Marine Institute - 1 Toyon Bay Road, Avalon, CA 90704. Note: online applications accepted only . …
Clerical Support Technician
Job Description and Duties The Contractors State License Board is recruiting for an Office Technician (Typing) within the Licensing and Examination Division in the License Management and Processin…