Programming and Data Processing: Strong proficiency in python (date engineering, API’s, error handling, logging).
Strong proficiency in SQL. Hands\-on with Pyspark for distributed data processing.
Expertise in Azure Databricks (workspace management, clusters, jobs, notebooks).
Working knowledge of Azure Data Lake, Azure Data Factory.
Familiarity with AWS Redshift and Snowflake as cloud data warehouse. Designing, building and managing scalable, reliable, and performant ETL/ELT pipelines using PySp ark and Databricks.
Working with Delta Lake and optimizing delta lake.
Working knowledge in Parquet, Avro or similar file formats. Git\-based CI/CD workflows in Databricks.