Noida, Uttar Pradesh, India;Gurugram, Haryana, India;Bengaluru, Karnataka, India;Indore, Madhya Pradesh, India;Pune, Maharashtra, India;Hyderabad, Telangana, India
Qualification
:* Have 10\+ years of experience in design, architecture, implementation, and optimization of data engineering solutions over large volume (TB, PB scale) of data.
- Have expertise in design and implement end\-to\-end data architectures using Azure Databricks, including data ingestion pipelines, transformation logic, and data warehousing strategies to handle large\-scale batch and real\-time data processing.
- Proven expertise in Azure services including Azure Databricks, Azure Data Factory (ADF), ADLS, Azure Synapse, Azure SQL Database, and Azure Functions. Experience building scalable data lakes and pipelines.
- Strong hands\-on experience processing large volumes of data, Proficiency in PySpark, Python, Spark SQL, and automating workflows.
- Have good exposure of Implement robust data governance using Unity Catelog and security measures
- Have proficiency in requirements analysis, solution design, development, testing, deployment, and ongoing support, including cloud migration projects for large\-scale data platforms.
Skills Required
Azure,Databricks
Role
- Have 10\+ years of experience in design, architecture, implementation, and optimization of data engineering solutions over large volume (TB, PB scale) of data.
- Proven expertise in Azure services including Azure Databricks, Azure Data Factory (ADF), ADLS, Azure Synapse, Azure SQL Database, and Azure Functions. Experience building scalable data lakes and pipelines.
- Proficiency in PySpark, Python, Spark SQL, and automating workflows.
- Hands on exposure in configure and manage Azure Databricks clusters, workspaces, and Unity Catalog for optimal performance, cost management, and scalability in processing terabyte\-scale datasets.
- Have good exposure of writing optimized SQL
- Have good communication and problem solving skills
- Able to create POC to achive the solutions and participate in proposals and RFPs
- Have good understanding of GenAI technologies and able to implement solutions using GenAI
Experience
10 to 12 years
Job Reference Number
13531