Enabling Areas - National Office - Manager - Data Engineering

• *Enabling Areas \- National Office \- Manager \- Data Engineering**

• **Job requisition ID** : 116330

• **Location**: Bengaluru

• **Entity**: Deloitte Shared Services India LLP

• *The team**

The Office of the COO, Deloitte India is powered by a dynamic team known for its agility and results. Despite its size, the team manages critical projects, consistently delivering high\-impact outcomes.

Collaboration is the key to the team’s success, with members working closely together while taking ownership of their responsibilities. They operate in a fast\-paced environment, engaging directly with senior leadership (CXOs’) to implement key initiatives and drive strategic outcomes.

• *Your work profile**

• *Data Lake Architecture \& Engineering**
• Design, develop, and maintain scalable data lake solutions using Microsoft Azure technologies.
• Build and manage data ingestion, transformation, and processing pipelines to integrate data from multiple enterprise applications and source systems.
• Develop and maintain structured data layers within the data lake, ensuring data is organized, accessible, and optimized for downstream consumption.
• Work extensively with Azure Data Lake Storage (ADLS Gen2\), Azure Data Factory (ADF), Azure SQL, and other relevant Azure data services.
• Implement efficient data storage, partitioning, transformation, and processing strategies to improve performance and scalability.
• Support the development and maintenance of enterprise data models catering to financial, operational, and management reporting requirements.
• Evaluate existing data architecture and recommend improvements to enhance performance, reliability, and maintainability.

• *SQL Development \& Data Transformation**
• Develop, maintain, and optimize complex SQL queries, stored procedures, views, functions, and data transformation logic.
• Design and implement robust ETL/ELT processes for large\-volume enterprise datasets.
• Perform SQL query tuning, performance optimization, and database troubleshooting to ensure efficient data processing.
• Develop reusable data transformation components and standardized data engineering frameworks.
• Translate complex business requirements into technical data structures, transformation rules, and data models.
• Ensure accurate handling of historical data, incremental loads, data reconciliation, and changes in source data structures.
• Troubleshoot complex data issues involving inconsistent source data, transformation logic, and downstream reporting discrepancies.

• *Data Integration \& Pipeline Management**
• Design and implement automated data pipelines to support integration between enterprise systems, data lake environments, and downstream applications.
• Work with structured and semi\-structured data from databases, APIs, flat files, and enterprise applications.
• Develop and manage scheduled and event\-driven data ingestion processes.
• Implement appropriate error handling, logging, monitoring, and alerting mechanisms across data pipelines.
• Ensure timely availability of data for business\-critical reporting and analytics platforms.
• Investigate and resolve data pipeline failures, processing delays, and integration issues.
• Coordinate with source\-system teams to manage changes in data structures, interfaces, and integration requirements.

• *Data Quality, Validation \& Governance**
• Establish and implement data quality frameworks covering completeness, accuracy, consistency, and integrity of information.
• Develop automated validation and reconciliation mechanisms between source systems, data lake layers, and downstream reporting applications.
• Identify root causes of data discrepancies and implement sustainable technical solutions.
• Maintain appropriate documentation of data pipelines, transformation logic, source\-to\-target mappings, and data lineage.
• Implement data access controls and security practices in alignment with organizational information security policies.
• Support data governance initiatives, including standardization of data definitions and master data across systems.
• Ensure engineering solutions comply with applicable data privacy, security, and retention requirements.

• *Enterprise Analytics \& Reporting Enablement**
• Collaborate with business and technology teams to understand enterprise reporting and analytics requirements.
• Develop and maintain data structures that support financial planning, business performance reporting, and operational dashboards.
• Support integration of data lake information with downstream applications such as Power BI, Anaplan, and other enterprise analytics platforms.
• Work with large datasets covering financial and operational metrics such as revenue, billing, collections, workforce, utilization, costs, and profitability.
• Optimize datasets and data transformation processes to improve reporting performance and data refresh timelines.
• Support the implementation of new reporting dimensions, business hierarchies, and changes in organizational structures.
• Provide technical support for data reconciliation and issue resolution during implementation, testing, and production operations.

• *Technical Delivery \& Team Management**
• Independently lead the technical design, development, testing, and implementation of data engineering solutions.
• Manage multiple development initiatives and ensure delivery within agreed timelines and quality standards.
• Review SQL code, pipeline designs, data models, and technical solutions developed by team members.
• Provide technical guidance and mentoring to data engineers and other members of the development team.
• Establish and enforce coding standards, documentation practices, version control, and deployment processes.
• Collaborate with IT infrastructure, application development, data architecture, and business teams to resolve technical dependencies.
• Proactively identify technical risks, performance bottlenecks, and areas requiring architectural improvements.
• Ensure appropriate testing, release management, and production support processes are followed.

• *Continuous Improvement \& Automation**
• Identify opportunities to automate manual data processing, reconciliation, and operational activities.
• Develop reusable frameworks and utilities to improve engineering productivity and reduce development effort.
• Implement improvements to reduce data processing time, infrastructure costs, and recurring operational issues.
• Evaluate emerging Azure technologies and data engineering capabilities for potential adoption.
• Drive continuous improvement in data pipeline reliability, scalability, observability, and maintainability.

• *Key skills required**

• **8–12 years of overall experience** in data engineering, database development, data integration, or related technology roles.
• Minimum 4–5 years of hands\-on experience working with Microsoft Azure data engineering technologies.
• Strong experience in SQL development, query optimization, and enterprise database management.
• Proven experience designing and implementing end\-to\-end ETL/ELT pipelines and data lake solutions.
• Experience working with large\-scale enterprise datasets and integrating information from multiple source systems.
• Prior experience managing technical deliverables and providing guidance to small engineering teams.
• Experience in financial services, professional services, or enterprise finance and operations reporting would be advantageous.
• Previous experience in a Big 4 organization or a large multinational enterprise would be an added advantage.

• *Mandatory Technical Skills**

• *Microsoft Azure Data Engineering**
• Azure Data Lake Storage Gen2 (ADLS Gen2\)
• Azure Data Factory (ADF)
• Azure SQL Database / SQL Server
• Azure data integration and pipeline orchestration
• Data lake architecture and data processing frameworks
• Pipeline monitoring, troubleshooting, and performance optimization

• *SQL \& Database Development**
• Advanced SQL and T\-SQL
• Complex queries, joins, CTEs, and window functions
• Stored procedures, functions, and views
• Query performance tuning and execution plan analysis
• Database design, indexing, and optimization
• Data validation and reconciliation

• *Data Engineering \& Integration**
• ETL/ELT architecture and development
• Data ingestion and transformation
• Data warehousing and dimensional modelling
• Incremental loading and change data capture concepts
• Data quality management
• Source\-to\-target mapping and data lineage
• Integration with APIs, databases, and flat files
• Handling large datasets and complex transformation logic

• *Good\-to\-Have Technical Skills**
• Azure Databricks
• Python and PySpark
• Apache Spark
• Microsoft Fabric
• Azure Synapse Analytics
• Delta Lake and Medallion Architecture
• Power BI datasets and semantic models
• Azure DevOps, Git, and CI/CD pipelines
• REST APIs and application integration
• Experience integrating data with SAP, ERP, or enterprise finance systems
• Knowledge of cloud security, access management, and infrastructure cost optimization