Job Description
Design, develop, and maintain efficient and reliable data pipelines using modern ELT tools like DBT, Fivetran, Matillion, Talend, or Ab Initio.
- Implement data transformations and cleansing processes to ensure data quality and accuracy.
- Build and optimize data models for cloud data warehouses like Databricks, Snowflake, or Redshift.
- Write complex SQL queries for data extraction, transformation, and loading.
- Develop and maintain shell or Python scripts for automation and data processing.
- Monitor and troubleshoot data pipelines to ensure optimal performance and stability.
- Collaborate with data scientists and analysts to understand their data needs and provide support.
- Stay up-to-date with the latest data engineering trends and technologies.
- Contribute to the development of data engineering best practices and standards.
- Participate in code reviews and ensure code quality.
- Work closely with the infrastructure team to deploy and manage data pipelines.