The role
Job description
Job Description: SDET – Data Engineering & SAP BW
Company: CX Data Labs
Department: Engineering / Quality Engineering
Role: Software Development Engineer in Test (SDET)
Role Overview
CX Data Labs is looking for a hands-on Software Development Engineer in Test (SDET) with strong expertise in test automation, data quality validation, and enterprise data platforms. The role will focus on designing and implementing automated testing frameworks to validate the accuracy, integrity, performance, and reliability of data pipelines and integrations across SAP BW, Databricks, and downstream reporting systems.
The ideal candidate combines strong software development and automation skills with experience in data-centric testing, ETL/ELT validation, and SAP data warehousing. You will work closely with development, data engineering, and architecture teams to identify defects early, automate repetitive validation processes, and improve overall data quality.
Key Responsibilities1. Test Automation & Framework Development
Design, develop, and maintain reusable automated testing frameworks using Python and SQL.
Build automated test suites for data pipelines, ETL/ELT workflows, APIs, and data integrations.
Develop unit, integration, regression, and end-to-end tests for data processing workflows.
Create reusable test utilities, test data generators, validation scripts, and automation components.
Identify opportunities to replace manual testing with scalable, repeatable automated tests.
2. Data Quality & ETL Testing
Validate data extraction, transformation, loading, and migration processes across source and target systems.
Develop automated checks for data accuracy, completeness, consistency, uniqueness, referential integrity, and reconciliation.
Test complex data transformations using SQL, PySpark, and Apache Spark.
Perform source-to-target validation, record-count comparisons, aggregation checks, and business-rule validation.
Investigate data discrepancies, conduct root-cause analysis, and collaborate with engineering teams to resolve defects.
Validate incremental loads, historical data processing, and data refresh workflows.
3. SAP BW & Databricks Testing
Design and execute automated tests for data flows between SAP BW/SAP BW/4HANA, Databricks, and downstream reporting platforms.
Validate SAP BW data extraction, transformation logic, data models, and reporting outputs.
Test Databricks notebooks, Spark transformations, data processing jobs, and Delta Lake workflows.
Verify data consistency and reconciliation across SAP and modern cloud data platforms.
Support data migration testing and regression testing following changes to data models, pipelines, or integration logic.
4. Continuous Testing & Quality Engineering
Integrate automated tests into CI/CD pipelines to enable continuous testing and early defect detection.
Implement automated regression suites and quality gates for data engineering releases.
Monitor test execution, analyze failures, and maintain reliable automated test suites.
Define test strategies, test scenarios, test cases, and traceability for complex data workflows.
Contribute to quality metrics, defect reporting, test coverage, and continuous improvement initiatives.
5. Collaboration & Technical Ownership
Work closely with developers, data engineers, SAP specialists, and solution architects to understand system requirements and data flows.
Participate in design and code reviews to identify testability gaps and potential quality risks.
Translate functional requirements and data specifications into comprehensive test scenarios.
Document test frameworks, validation logic, defects, and test results.
Take ownership of test automation deliverables from planning and implementation through execution and maintenance.
Required Qualifications
Strong professional experience in SDET, software test automation, quality engineering, or data testing.
Proficiency in Python programming and advanced SQL for developing automated tests and validating complex datasets.
Hands-on experience with test automation frameworks and reusable testing utilities.
Experience testing ETL/ELT pipelines, data integrations, data migrations, and data transformation workflows.
Hands-on experience with SAP BW or SAP BW/4HANA, including data extraction, transformation, and validation.
Practical experience with Databricks, PySpark, or Apache Spark for testing data processing workloads.
Understanding of unit, integration, regression, and end-to-end testing methodologies.
Experience with Git, CI/CD pipelines, defect tracking, and test execution reporting.
Strong analytical and debugging skills, with the ability to identify data inconsistencies and trace defects to their root cause.
Good to Have
Experience with SAP HANA, SAP BODS, SAP Datasphere, or related SAP data integration technologies.
Familiarity with Azure Data Factory or other cloud-based data integration platforms.
Experience validating Delta Lake tables, incremental data loads, and large-scale distributed data processing.
Exposure to API testing, performance testing, and test data management.
Experience validating Power BI dashboards, reports, and underlying datasets.
Familiarity with data quality frameworks and tools such as Great Expectations or comparable solutions.
Experience working in Agile/Scrum teams and collaborating across QA and engineering functions.
Success Metrics
Automation coverage for critical data workflows and regression scenarios.
Accuracy and reliability of source-to-target data validation.
Reduction in manual testing effort and escaped data defects.
Defect detection effectiveness and turnaround time for root-cause analysis.
Stability, maintainability, and execution efficiency of automated test suites.
Timely delivery of quality engineering milestones.
Ideal Candidate
An automation-focused SDET who writes production-quality Python and SQL, understands data engineering architectures, and can independently design robust testing solutions for complex enterprise data workflows. The candidate should be comfortable validating SAP BW and Databricks integrations, investigating data discrepancies, and embedding automated quality checks throughout the software delivery lifecycle.