Data Engineer - Snowflakes/AWS

• *Snowflake Developer / Data Engineer** designs, builds, optimizes, and maintains large\-scale cloud data platforms and ETL/ELT pipelines using Snowflake.

• *Core Responsibilities**

• **Data Pipeline \& Architecture:** Design, build, and deploy scalable data ingestion and transformation workflows.
• **SQL \& Performance Optimization:** Write, tune, and troubleshoot complex SQL queries, stored procedures, views, and Snowflake virtual warehouses for speed and cost efficiency.
• **Data Modeling:** Design dimensional models, star/snowflake schemas, fact/dimension tables, and Slowly Changing Dimensions (SCD).
• **DevOps \& CI/CD:** Implement version control (Git/GitHub), CI/CD pipelines, state\-based deployments, and dbt (data build tool) integrations.
• **Security \& Governance:** Ensure role\-based access control (RBAC), data masking, encryption, and compliance best practices are applied.
• Python: Backend service development, REST API integration, retry and error\-handling mechanisms, data schema modelling and validation with Pydantic

SQL and Snowflake: MERGE\- and stream\-based CDC pipelines, Tasks, stored procedures, Secure Views, Snowpipe and Snowpipe Streaming, Secure Data Sharing, RBAC, role and grant management, and warehouse performance tuning

Terraform: Infrastructure as code for Snowflake databases, roles, grants, warehouses, and repeatable multi\-environment deployments

• AWS: Lambda\-based event\-driven ingestion, Amazon S3 data lake storage, AWS Glue Data Catalog, and IAM\-based access controls

Palantir Foundry: Workshop\-based GUI development and backend workflow orchestration for data\-request, approval, and provisioning processes

Version Control and CI Engineering: Git\-based development workflows, pre\-commit hooks, Ruff\-based linting and formatting, pull\-request templates, review gates, and structured AI\-assisted development workflows

Data Modelling and Interchange Formats: JSON\-based configuration and request schemas, and synchronisation of hierarchical and relational data models across systems

• Platform and Domain Expertise

Enterprise data provisioning: End\-to\-end ownership of data\-request portals using Palantir Foundry Workshop, Foundry workflow orchestration, and Python backend services targeting Snowflake and AWS

Data virtualisation and analytics integration: Output\-port configuration, ownership and privilege logic, and hierarchical data\-model synchronisation between source systems and downstream platforms

Industrial and OT data integration: Event\-driven ingestion of Aveva PI process\-historian data using Lambda, Snowpipe, and Snowflake, with downstream delivery through Secure Data Shares

Multi\-stage approval workflows: Role\-based approval lifecycles involving requesters, data stewards, data owners, engineering teams, and business verifiers

Data access governance: Request and approval workflows, role\-based access control, and self\-service provisioning for business data owners

Regulated\-environment delivery: Dev\-to\-QA\-to\-production promotion workflows supporting validated and regulated, GxP\-equivalent system requirements

• *Required Skills \& Qualifications**

• **Experience:** Typically **3 to 5\+ years** of hands\-on experience in data engineering or cloud data warehousing.
• **Core Technologies:**
• + **Snowflake Features:** Virtual warehouses, clustering, micro\-partitions, Time Travel, Streams \& Tasks, Snowpipe, Dynamic Tables, and Snowflake Cortex (where applicable).

+ **Programming \& Scripting:** Expert\-level SQL and strong **Python** (Pandas, NumPy).

+ **Transformation Tools:** **dbt** (models, snapshots, tests, macros, Slim CI).

+ **Cloud \& Orchestration:** Exposure to AWS, Azure, or GCP, plus tools like Apache Airflow.

• **Certifications:** Preferred credentials often include **SnowPro Core** or advanced Snowflake certifications.