Sr Cloud Data Engineer

Job Description: We are seeking a highly skilled Senior Data Engineer to lead the design, development, and optimization of scalable data solutions that power enterprise analytics and business decision\-making. The ideal candidate will bring deep expertise in Azure and Databricks, strong hands\-on experience in building robust ETL/ELT pipelines, and a solid understanding of data quality, governance, and performance tuning. This role will be instrumental in enabling reliable, high\-quality data flows across complex systems, with a strong focus on supporting insurance domain use cases, regulatory needs, and data\-driven business transformation.

• Responsibilities: Lead the design, development, and optimization of scalable, high\-performance data pipelines , ensuring reliable data ingestion and transformation across distributed systems, while proactively identifying and resolving performance bottlenecks and system inefficiencies.
• **Define and enforce enterprise\-grade data quality frameworks** , implementing advanced validation rules, anomaly detection, and reconciliation processes to ensure data accuracy, completeness, and integrity across large\-scale datasets.
• **Own end\-to\-end monitoring and reliability of data platforms** , establishing observability practices (alerts, SLAs, dashboards) and driving root\-cause analysis to minimize downtime and ensure seamless continuity of critical data workflows.
• **Govern and continuously improve data pipelines through periodic audits and reviews** , ensuring adherence to internal standards, data governance policies, and regulatory compliance, while driving standardization and best practices across teams.
• **Collaborate with cross\-functional stakeholders (architects, analysts, product owners)** to align data solutions with business objectives, and actively contribute to backlog prioritization, sprint planning, and technical problem\-solving in Agile environments.
• **Mentor junior engineers and drive technical excellence** , promoting reusable frameworks, coding standards, and best practices to improve overall team productivity and solution quality.

• Qualifications: 6–10 years of hands\-on experience in data engineering, with strong expertise in Azure cloud and Databricks\-based data platform development.
• Bachelor’s degree in computer science, Information Technology, Engineering, or a related discipline.
• Strong programming skills in PySpark, Python, and SQL, with proven experience in building scalable and maintainable data pipelines.
• Solid experience in data modeling, ETL/ELT pipeline design, Delta Lake, and enterprise data warehousing concepts.
• Hands\-on expertise with Azure Data Factory, Azure Databricks, Azure Data Lake Storage, and related Azure data services.
• Experience with Databricks capabilities such as Delta Live Tables, Auto Loader, Unity Catalog, and workflow orchestration.
• Strong understanding of performance tuning, optimization of Spark workloads, and pipeline reliability in large\-scale data environments.
• Working knowledge of data governance, data quality, and secure data handling practices within enterprise platforms.
• Preferred exposure to insurance domain data models, business processes, and regulatory reporting requirements.
• Excellent analytical, communication, and problem\-solving skills, with the ability to collaborate effectively across cross\-functional teams and deliver under tight deadlines.