Data & Analytics Data Pipelines Lead

Key Responsibilities

Data Pipeline Leadership

• Lead the design, development, deployment, and support of enterprise data pipelines and integration solutions.
• Establish standards, patterns, and best practices for data ingestion, transformation, orchestration, and delivery.
• Oversee data movement across the full analytics lifecycle:

o Source systems and external data providers

o Landing/staging databases

o Enterprise data warehouses

o Data lake environments

o Analytics platforms

o Reporting databases and data marts

• Ensure scalable, secure, and high\-performing data integration architectures.
• Drive automation and operational efficiency within data pipeline environments.

Data Integration Engineering

• Manage batch, near\-real\-time, and streaming data ingestion processes.
• Coordinate data onboarding and integration activities for new source systems and external data suppliers.
• Define and maintain ETL, ELT, and data replication standards.
• Support cloud and on\-premises data integration platforms.
• Collaborate with enterprise architects to align data movement solutions with strategic architecture standards.

Data Quality Governance

• Establish and maintain enterprise data quality controls and monitoring processes.
• Define data validation, reconciliation, exception handling, and alerting frameworks.
• Monitor pipeline performance and proactively identify data integrity issues.
• Partner with data governance teams to ensure compliance with organizational standards.
• Implement and maintain data lineage documentation across data movement processes.
• Support audit, compliance, and regulatory reporting requirements related to data traceability.

Data Lineage Metadata Management

• Document end\-to\-end data flows across all pipeline stages.
• Maintain lineage mapping between source systems, transformation processes, and downstream reporting assets.
• Ensure metadata accuracy and availability for analytical consumers.
• Support impact assessments related to upstream and downstream changes.
• Drive adoption of data catalog and metadata management capabilities.

Production Support Operations

• Lead operational support for production data integration and analytics pipelines.
• Manage ServiceNow incidents, service requests, problem records, and change activities related to data pipeline operations.
• Coordinate incident triage, root cause analysis, issue resolution, and stakeholder communications.
• Establish and monitor service level agreements (SLAs) and operational metrics.
• Ensure rapid resolution of critical data availability and quality issues.
• Coordinate production releases and change management activities.

Monitoring Reliability

• Implement pipeline monitoring, observability, and alerting solutions.
• Track pipeline health, throughput, latency, failure rates, and data quality metrics.
• Develop operational dashboards and reporting for platform performance.
• Lead efforts to improve platform reliability, resiliency, and recoverability.
• Support disaster recovery and business continuity processes for critical data assets.

Stakeholder Team Leadership

• Serve as the primary point of contact for data pipeline operations and support.
• Partner with business intelligence, analytics, reporting, application, and infrastructure teams.
• Mentor data engineers and analysts on integration standards and best practices.
• Facilitate prioritization of enhancements, technical debt reduction, and operational improvements.
• Communicate risks, issues, and performance metrics to leadership.

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Required Qualifications

• Bachelor's degree in Computer Science, Information Systems, Engineering, Data Analytics, or related field.
• 7\+ years of experience in data engineering, data integration, ETL/ELT development, or data platform operations.
• 3\+ years of experience leading enterprise\-scale data pipeline and integration initiatives.
• Experience supporting production data environments and operational processes.
• Experience managing incident, problem, and change management processes within ServiceNow or similar ITSM platforms.
• Strong understanding of data warehousing, dimensional modeling, and data lake architectures.
• Experience implementing data quality and data governance practices.
• Strong analytical, troubleshooting, and problem\-solving skills.

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Preferred Qualifications

• Experience with cloud data platforms such as Oracle, Azure, AWS, or Google Cloud.
• Experience with the Teradata platform and Teradata Data Mover (TDM)
• Experience with modern data engineering platforms including:

o Azure Data Factory

o Databricks

o Synapse Analytics

o Snowflake

o Informatica

o Talend

o SSIS

o Kafka

o Fivetran

o dbt

• Experience with metadata management and data lineage tools.
• Knowledge of DevOps, CI/CD, infrastructure automation, and data observability platforms.
• Familiarity with Agile delivery methodologies and product operating models.

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Key Competencies

Technical Competencies

• Data Engineering
• ETL/ELT Architecture
• Data Warehousing
• Data Lake Architectures
• Data Quality Management
• Metadata Management
• Data Lineage
• Production Support Operations
• Monitoring and Observability
• ServiceNow Administration and Workflow Processes

Leadership Competencies

• Operational Excellence
• Stakeholder Management
• Team Leadership
• Incident Management
• Strategic Planning
• Continuous Improvement
• Risk Management
• Communication and Influence