- *Role description** **Job Role**: Senior Data Engineer
- *Job Location**: Any UST Location
- *Notice Period**: Immediate Joiners Only
We are seeking a highly skilled **Senior Data Engineer** with 6\+ years of experience in designing, developing, and optimizing large\-scale data platforms and pipelines. The ideal candidate will have strong expertise in **Databricks, Apache Spark, ETL/ELT development, cloud data engineering, and data modelling**. This role involves building scalable data solutions, implementing modern Lakehouse architectures, and collaborating with cross\-functional teams to support data\-driven business initiatives.
- Design, develop, and maintain scalable **ETL/ELT pipelines** using **Databricks** and distributed data processing frameworks.
- Build and optimize high\-performance **big data solutions** using **Apache Spark** with **Python, Scala, or SQL**.
- Develop and manage **Data Lake** and **Lakehouse architectures** on cloud platforms such as AWS, Azure, or GCP.
- Implement batch and real\-time data ingestion pipelines from diverse structured, semi\-structured, and unstructured data sources.
- Optimize Databricks workloads through performance tuning, cluster configuration, workload management, and cost optimization.
- Create and maintain reusable Databricks notebooks, workflows, jobs, and automation frameworks.
- Leverage **Delta Lake** features including ACID transactions, schema evolution, partitioning, and data versioning.
- Design efficient data models to support analytics, reporting, and machine learning workloads.
- Collaborate with Data Scientists, Data Analysts, Product Teams, and Business Stakeholders to deliver scalable data solutions.
- Implement data quality checks, monitoring frameworks, and automated validation processes.
- Ensure compliance with enterprise data governance, security, privacy, and regulatory standards.
- *Required Skills \& Qualifications**
- 6\+ years of experience in **Data Engineering** and Big Data technologies.
- Strong hands\-on experience with **Databricks** and **Apache Spark**.
- Proficiency in **Python, Scala, and SQL** for data processing and transformation.
- Experience designing and implementing **ETL/ELT pipelines** in enterprise environments.
- Strong knowledge of **Data Modeling**, Data Warehousing, and Lakehouse architectures.
- Experience with **Delta Lake**, data partitioning, schema management, and performance optimization.
- Hands\-on experience with cloud platforms:
- + Amazon Web Services (AWS)
+ Microsoft Azure
+ Google Cloud Platform (GCP)
- Knowledge of distributed computing concepts and large\-scale data processing.
- Experience implementing CI/CD pipelines and deployment automation.
- Strong troubleshooting, debugging, and performance tuning skills.
- Excellent communication and stakeholder management abilities.
- *Preferred Qualifications**
- Experience with streaming technologies such as **Spark Structured Streaming**, Kafka, or Event Hubs.
- Knowledge of Data Governance, Data Quality, and Data Security frameworks.
- Experience with Infrastructure as Code (Terraform, CloudFormation, ARM Templates).
- Databricks, AWS, Azure, or GCP certifications are preferred.
- Exposure to MLOps and advanced analytics platforms is a plus.
Data Engineering, Databricks, Data Processing, Data Modeling
UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world’s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future\-proof philosophy, UST embeds innovation and agility into their clients’ organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact—touching billions of lives in the process.