• Work Location: Remote (India)

• Candidates should be flexible to work from an EXL office whenever business requires. While the role is primarily remote, it is not a permanent work-from-home position.
  

Key Responsibilities

Data Pipeline Testing:

·         Validate end-to-end data ingestion, transformation, and consumption pipeline across bronze, silver, and gold layers.

·         Perform source-to-target data validation for structured.

·         Validate business rules and data transformations.

·         Ensure data consistency, completeness, uniqueness and referential integrity.

Automation Testing:

·         Design and develop automated test frameworks for data validation and regression testing.

·         Automate ETL and data quality testing using python, SQL, PySpark, or testing frameworks.

·         Build reusable validation scripts for schema validation, reconciliation, and data drift detection.

·         Integrate automated test suites into CI/CD pipelines.

Databricks and Cloud Validation:

·         Validate Databricks notebooks, workflows, Delta Tables and job orchestration.

·         Test Delta Lake functionalities include ACID compliance, schema evolution and time travel.

·         Validate performance and scalability of Spark-based transformations.

Required Skills:

·         SQL

·         Python

·         PySpark

·         Databricks

·         Batch Pipelines

·         Incremental Loads

·         CDC pipelines

·         Knowledge of Medallion Architecture concepts

Preferred Qualifications:

·         4-6 Years of experience in QA/Data Testing.

·         2+ years of experience in Databricks/Data Lakehouse projects.

·         Experience working in Agile/Scrum delivery models.

·         Certification in Databricks, Cloud or Testing tool is an advantage.