Budget:
5 to 8 yrs - 1.5 to 2.2

9 to 12 yrs - 2.0 to 2.7

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using modern orchestration frameworks (Apache Airflow, Apache Spark, Kafka)

• Build data ingestion solutions from diverse sources including ERP systems (SAP, Oracle), CRM platforms (Salesforce, Workday), and cloud data warehouses

• Implement data quality frameworks, validation rules, and monitoring to ensure high-quality data across all applications

• Optimize data processing workflows for performance, reliability, and cost-efficiency in cloud-native environments

• Collaborate with BI Engineers and Data Modelers to ensure seamless data flow from source systems to analytical dashboards

• Write clean, reusable, and well-tested code following software engineering best practices (code reviews, unit testing, CI/CD)

________________________________________

Required Qualifications

• 5-12 years of proven experience in building production-grade data pipelines and ETL/ELT solutions

• Strong proficiency in Python for data engineering (pandas, PySpark, SQLAlchemy, data processing libraries)

• Advanced SQL skills including complex joins, window functions, query optimization, and performance tuning

• Hands-on experience with cloud platforms (AWS, Azure, or GCP) and cloud-native data services

• Familiarity with orchestration frameworks such as Apache Airflow, Apache Spark, Kafka, or similar tools

• Experience with modern data warehouses (Snowflake, Databricks, BigQuery, Redshift) including data modeling and optimization

• Knowledge of ERP/CRM systems integration (SAP, Oracle, Salesforce, Workday) - extracting data via APIs, connectors, or database access

• Strong understanding of data quality principles, testing frameworks, and validation methodologies

• Experience in leveraging or critically thinking about how to integrate AI into work processes, decision-making, or problem-solving.

________________________________________

Nice-to-Have Qualifications

• Experience with AI-powered productivity tools (Cursor, Windsurf, Claude, GitHub Copilot) for accelerated development

• Hands-on Java experience and understanding of enterprise application design patterns

• Experience working with ServiceNow platform or similar enterprise workflow platforms

• Knowledge of real-time streaming architectures (Kafka Streams, Flink) for industrial/IoT analytics

• Experience with containerization (Docker, Kubernetes) and modern CI/CD practices

• Familiarity with AI/ML pipeline integration, feature engineering, and model serving infrastructure

Sharing screening questions and their acceptable answer. Once resume screening is completed, please ask the below mentioned mandatory questions and if candidates are able to provide acceptable answers for all the questions, then we can take an interview for those candidates. 

1. How many years of hands-on experience do you have in data engineering and ETL/ELT pipeline development?

5+ years

2. How many years of hands-on experience do you have with Apache Airflow or any similar orchestration including DAG development and job orchestration?

2+ years

3. Are you proficient with SQL and PySpark/Python based distributed data processing frameworks?

Yes; hands-on experience withboth

4. How many years of experience do you have working with AWS/Azure/GCP services and their components

3+ years

5. Are you proficient in SQL and Python scripting?

Yes; both required

6. Have you worked with SAP data, particularly in finance modules (GL, AP/AR, Fixed Assets, CO) and experience?

Yes; 1+ years of SAP finance data experience

7. Do you have experience with SAP data extraction tools or SAP S/4HANA migration projects?

Yes; name specific tools (SLT, BODS, SAP Data Services)

8. Have you designed and implemented end to end pipelines ETL/ELT pipelines for large-scale data migration or transformation projects?

Yes

9. Are you familiar with BI/reporting tools (Power BI, Tableau, or SAP Analytics Cloud)?

Yes; hands-on experience with at least one tool

10. Have you worked with Snowflake, Redshift, Databricks, Big Query or other data warehouse platforms with data modeling experience?

Yes; experience with at least one data warehouse platform (preferred qualification)

experience of integrating data from data sources such as CRM,ERP, ITSM, or other platforms. Data Integration