Required Technical Skills (Mandatory)

● Certification: Active Azure Databricks Certification (e.g., Databricks Certified Data Engineer

Associate/Professional).

● PySpark: Expert-level, hands-on experience in developing, tuning, and optimizing large-scale data

processing applications using PySpark (Python for Apache Spark).

● SQL: Mastery of Advanced SQL (including window functions, complex joins, stored procedures, and

query performance tuning) across various database systems (e.g., Snowflake, Redshift, PostgreSQL).

● Programming: Strong proficiency in Python for scripting, automation, and general data manipulation

libraries (e.g., Pandas).

● Big Data Architecture: Deep understanding of Big Data concepts, distributed systems architecture, data

lakes, and modern data warehousing principles.

● ETL/ELT: Proven experience in designing and implementing enterprise-grade ETL/ELT pipelines.

Preferred Qualifications (Good to Have)

● Hands-on experience with wider Azure ecosystem components (Azure Data Factory, Azure Synapse, Key

Vault).

● Familiarity with workflow orchestration tools like Apache Airflow.

● Experience with real-time/streaming data processing (e.g., Spark Structured Streaming, Kafka, or Event

Hubs).

● Advanced knowledge of Data Governance, Data Cataloging, and Data Security best practices.

Candidate Profile

● Educational Background: Bachelor’s or Master’s degree in Computer Science, Engineering, or a

related quantitative field.

● Mindset: Proactive, self-motivated, strategic thinker with a strong sense of ownership and urgency.

● Communication: Excellent verbal and written communication skills to articulate complex technical

concepts to non-technical stakeholders and executive leadership.

● Availability: Must be an Immediate Joiner or have a short notice period (7 days maximum).