Data Engineer

• *Title: Senior Technical Specialist \- PBI Databricks**Location: Remote

Duration: 12\+ Months

Education: Any Degree

Exp Level: 8\+ Years

• *Job Responsibilities:*** Design and build conformed dimensions and transactional fact tables for enterprise reporting.
• Develop and maintain data pipelines on A **zure Databricks using PySpark, Spark SQL, and Delta Lake.**
• Implement best practices in dimensional modeling, including:
• **Star schemas**
• **Conformed dimensions**
• Surrogate and deterministic hash keys (e.g., SHA\-256\)
• Transactional fact design
• Translate existing SQL Server and stored procedure logic into efficient Spark\-based transformations.
• Implement incremental data processing and change data capture (CDC) pipelines.
• Ensure data reconciliation with source systems to meet accuracy and audit standards.
• Collaborate closely with a senior Databricks lead to deliver core data assets within defined timelines.

• *Required Skills:*** 8\+ years of experience in data engineering, with strong focus on **Azure Databricks** .
• Proficiency in:

• **PySpark**
• **Spark SQL**
• **Delta Lake**
• Deep expertise in dimensional modelling and data warehousing concepts.
• Experience with:

• **Building scalable production pipelines**
• **Incremental data loads and CDC patterns**
• **Performance optimization in distributed systems**
• Strong understanding of relational databases and ability to interpret complex SQL/stored procedures

• *Good to have****Experience with:*** Lakeflow / Spark Declarative Pipelines or dbt
• Unity Catalog
• Exposure to financial/General Ledger data models
• Familiarity with JD Edwards systems, including:

• **F0902 (balances)**
• **F0911 (journal details)**