ID: 59391
5 \- 7 Years
1 Opening
Pune
### **Role description**
We are seeking a skilled Azure Data Engineer with 5\+ years of experience in designing, developing, and maintaining modern data pipelines and data integration solutions using Azure services. The ideal candidate should have strong expertise in Azure Data Factory (ADF), Azure Databricks, Azure Synapse, and Azure Data Lake Storage (ADLS). You will work closely with business analysts, architects, and data scientists to deliver reliable and scalable data solutions that power analytics and business intelligence platforms.
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
Data Ingestion \& Integration
Design, build, and maintain data pipelines using Azure Data Factory for batch and incremental data ingestion.
Connect to various data sources (SQL Server, REST APIs, CSV, JSON, SAP, etc.) and integrate into the Azure ecosystem.
Develop metadata\-driven and parameterized pipelines to improve reusability.
Implement data validation, error handling, and logging frameworks in ADF.
Data Transformation \& Processing
Use Azure Databricks (PySpark) for data cleansing, transformation, and enrichment.
Optimize Spark jobs for performance and cost efficiency.
Implement ETL/ELT workflows with Delta Lake and Medallion (Bronze, Silver, Gold) architecture.
Data Storage \& Modeling
Work with Azure Data Lake Storage Gen2 for raw and curated data zones.
Develop data models in Azure Synapse Analytics / SQL Server for reporting and analytics.
Implement partitioning, indexing, and performance tuning strategies.
Deployment \& DevOps
Implement CI/CD pipelines using Azure DevOps or GitHub Actions for data workflows.
Collaborate with architects to automate deployments and version control using Git.
Security \& Governance
Manage data access using Azure RBAC, Managed Identities, and Key Vault.
Ensure data security, compliance, and privacy as per organizational standards.
Collaboration
Work with data analysts, BI developers, and business users to define data requirements.
Participate in code reviews and adhere to best practices in data engineering.
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
Category Skills
Azure Services: Azure Data Factory, Azure Databricks, Azure Synapse, ADLS Gen2, Azure SQL Database
Programming: Python (PySpark), SQL, Spark SQL
Data Modeling: Star/Snowflake schema, Dimensional modeling
Source Systems: SQL Server, Oracle, SAP, Flat Files (CSV, JSON, XML), REST APIs
Version Control \& CI/CD: Git, Azure DevOps
Scheduling \& Monitoring: ADF triggers, Databricks jobs, Log Analytics
Security: Managed Identity, Key Vault, Access Control
Preferred: Power BI basics, exposure to DataBricks Delta Live Tables or Synapse Pipelines
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
Strong analytical and problem\-solving skills.
Good communication and collaboration abilities.
Ability to work in agile/scrum environments.
Self\-driven and proactive in identifying process improvements.
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
Bachelor s degree in Computer Science, Information Technology, or a related field.
Azure Data Engineer Associate (DP\-203\) certification preferred.
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
Design and implement end\-to\-end data ingestion from on\-prem SQL Server to Azure Data Lake using ADF.
Build Databricks notebooks for data cleansing and transformations using PySpark.
Implement Delta Lake tables and load curated data into Synapse for reporting.
Collaborate with BI teams to publish Power BI dashboards on top of Synapse datasets.
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
Experience with Real\-time data processing (Event Hub / Stream Analytics).
Knowledge of Infrastructure as Code (IaC) using Terraform or ARM templates.
Familiarity with data quality and data catalog tools (Purview).
### **Skills**
data engineering,azure data factory,spark,sql,python,
### **About UST**
UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world’s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future\-proof philosophy, UST embeds innovation and agility into their clients’ organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact—touching billions of lives in the process.
Senior Staff Developer - AI SOC Automation
Arctic Wolf Networks · Remote
Sen. Mobile App Tester
Testvox · Mumbai
GenAI / AI-ML Engineer
Premier IT Solutions · Ghaziabad