Job Description: • Databricks, PySpark and Python project development work experience is a must • Develop, design, tune and maintain PySpark scripts using Databricks Notebook. • Expertise in Bigdata Eco systems like HDFS and Spark • Experience in GitHuB repository • Data Warehouse/Data Marts/Data Modelling/Analytics experience is a must • Able to convert the SQL stored procedures to Python code in Pyspark frame work using Dataframes. • Implementing data ingestion pipelines from multiple data sources using Azure Data Factory, Azure Databricks and other ETL tools. • Developing Big Data and non\-Big Data cloud\-based enterprise solutions in PySpark and SparkSQL and related frameworks/libraries. • Developing scalable and re\-usable, self\-service frameworks for data ingestion and processing. • Integrating end to end data pipelines to take data from data source to target data repositories ensuring the quality and consistency of data. • Processing performance analysis and optimization. • Collaborate with business users, support team members, and other developers throughout the organization to help everyone understand issues that affect the data warehouse • Good experience on customer interaction is required. • Possesses good interpersonal and communication skills.
Developer – Azure Full Stack / Custom Business Applications - Husky (Guindy, Chennai)
Husky Technologies · Chennai, Tamil Nadu, India