### **General Information**
Hyderabad, Telangana
Working Schedule
Full\-Time
Work Arrangement
Hybrid
Relocation Assistance Available
No
Posted Date
01\-Jun\-2026
Job ID
18134
### **Description and Requirements**
We are seeking a skilled **Databricks Data Engineer** to design, build, and optimize scalable data pipelines and data platforms using the Databricks Lakehouse architecture. The role involves working closely with data scientists, analysts, and business stakeholders to enable data\-driven decision\-making through robust, high\-quality data solutions.
A Databricks Data Engineer primarily focuses on building and maintaining pipelines that transform raw data into usable insights using platforms like Apache Spark and Databricks. **Key Responsibilities****1\. Data Pipeline Development*** Design, develop, and maintain scalable batch and streaming data pipelines on Databricks
- Implement end\-to\-end ingestion, transformation, and modeling (Bronze, Silver, Gold layers)
- Build ETL/ELT workflows for structured and unstructured data
- *2\. Data Processing \& Transformation*** Process large\-scale data using **Apache Spark (PySpark/Scala)**
- Perform data cleansing, transformation, and enrichment
- Enable efficient data modeling for analytics and reporting
- *3\. Performance Optimization*** Optimize Spark jobs for performance (partitioning, joins, caching)
- Troubleshoot and resolve pipeline performance bottlenecks
- Tune clusters, workloads, and resource utilization
- *4\. Data Quality \& Governance*** Ensure data quality, consistency, and reliability across pipelines
- Implement validation checks, monitoring, and alerting mechanisms
- Apply governance practices such as schema enforcement and versioning
- *5\. Platform \& Workflow Management*** Develop and manage Databricks notebooks, jobs, and workflows
- Orchestrate pipelines using tools like Databricks Workflows / Airflow
- Monitor pipeline execution and ensure SLAs are met
- *6\. Collaboration \& Stakeholder Engagement*** Work closely with data scientists, analysts, and architects to gather requirements
- Translate business requirements into scalable data solutions
- Support downstream reporting, analytics, and ML use cases
- *Required Skills \& Competencies****Technical Skills*** Strong experience with **Databricks \& Apache Spark**
- Proficiency in **Python / PySpark / Scala / SQL**
- Experience with **ETL/ELT and Data Warehousing concepts**
- Familiarity with **Delta Lake, Lakehouse architecture**
- Experience with **Cloud platforms** (Azure / AWS / GCP)
- Knowledge of **big data technologies** (Kafka, Hadoop, etc.)
- *Engineering Practices*** Experience with **CI/CD, Git, and SDLC practices**
- Exposure to **pipeline orchestration tools** (Airflow, ADF, etc.)
- Understanding of **data governance and security**
- *Soft Skills*** Strong problem\-solving and analytical capabilities
- Effective communication and stakeholder management
- Ability to work in cross\-functional teams
- *Qualifications*** Bachelor’s / Master’s degree in Computer Science, Engineering, or related field
- Typically **3\+ years of experience** in data engineering or big data environments
- Certifications in Databricks or cloud platforms are a plus
- *Nice\-to\-Have*** Experience with **ML pipelines or feature engineering**
- Exposure to **real\-time streaming frameworks**
- Knowledge of **Power BI / analytics tools** (for enterprise setups)
- *Tools \& Technologies*** Databricks (Lakehouse Platform)
- Apache Spark (PySpark / Scala)
- SQL
- Delta Lake
- Azure Data Factory / Airflow
- Cloud Storage (ADLS, S3, GCS)
Recognized on Fortune magazine's list of the "World's Most Admired Companies" and Fortune World’s 25 Best Workplaces™, MetLife, through its subsidiaries and affiliates, is one of the world’s leading financial services companies; providing insurance, annuities, employee benefits and asset management to individual and institutional customers. With operations in more than 40 markets, we hold leading positions in the United States, Latin America, Asia, Europe, and the Middle East.
Our purpose is simple \- to help our colleagues, customers, communities, and the world at large create a more confident future. United by purpose and guided by our core values \- Win Together, Do the Right Thing, Deliver Impact Over Activity, and Think Ahead \- we’re inspired to transform the next century in financial services. At MetLife, it’s \#AllTogetherPossible. Join us!