DE&A - Core - Advanced Visualization & Cloud Reporting - Azure Analysis Services

• *Job Description \- Data Engineer (Azure \| Databricks) \+ AI Engineering Exposure**

• *Role Title:** Data Engineer (Azure Databricks)

• *Engagement Type:** Contract (Vendor Hiring)

• *Experience Range:** 3–8 years

• *Location:** India

• *Team:** IDO CIS \- BI \& Analytics

• *Role Overview**

We are looking for a contract **Data Engineer** to support enterprise BI \& Analytics deliveries within IDO. The role will design, build, and operate scalable data pipelines and curated datasets on **Microsoft Azure** , with strong hands\-on delivery in **Databricks (Spark/Delta)** . Exposure to **AI engineering / GenAI enablement** is a strong plus for future roadmap initiatives.

• *Key Responsibilities**

• Build and maintain data pipelines on Azure (ingestion, transformation, validation, and publishing) using **Databricks** and Azure services.
• Develop curated, analytics\-ready datasets for BI/reporting and downstream consumers.
• Good understanding of FACT and dimensional modelling.
• Implement **incremental load** patterns, backfills, and handling of late\-arriving data to minimize full reloads.
• Ensure strong **data quality** , reconciliation checks, and pipeline monitoring/alerting.
• Optimize performance and cost (Spark tuning, partitioning strategies, file formats).
• Maintain documentation (source\-to\-target mappings, runbooks) and support releases/production issues in an Agile model.

• *Must\-Have Skills (Azure Data Engineering \+ Databricks)**

• 3–8 years of hands\-on **Data Engineering** experience.
• Strong experience with **Azure** data ecosystem and **Databricks** :

+ Spark (PySpark), **Delta Lake** (MERGE/upserts, schema evolution), Lakehouse patterns

• Strong **SQL** and **Python** (production\-grade coding).
• Orchestration experience with **Azure Data Factory** and/or **Databricks Workflows** .
• Practical experience implementing **incremental processing** and robust operational pipelines (retry, idempotency, monitoring).

• *Good\-to\-Have Skills (AI Engineering / Future\-Forward)**

• Exposure to **ML/AI enablement** (feature datasets, ML lifecycle awareness, reproducible training data).
• Familiarity with **GenAI/RAG** data preparation (document ingestion, chunking, metadata enrichment, embeddings).
• Understanding of **vector search/vector databases** concepts (hands\-on is a plus).
• Awareness of governance and secure data handling (PII, access control).