Manager & Deputy Manager – Pyspark Developer

Position Location Qualification Age Limit

• *Manager \& Deputy Manager – Pyspark Developer** Mumbai Bachelor's or Master's degree in Computer Science, Data Science, or a related field. Below 35

• *Experience \& Role:**

1\. Having experience 4\-5 years 2\. Having experience 2\-3 years \- Strong expertise in Big Data ecosystem like Spark, Hive, Sqoop, HDFS, Map Reduce, Oozie, Yarn, HBase, Nifi.

• Design, develop, and maintain PySpark data processing pipelines to process large volumes of structured and unstructured data.

• Collaborate with data engineers and data scientists to understand data requirements and design efficient data models and transformations.

• Optimize and tune PySpark jobs for performance, scalability, and reliability.

• Implement data quality checks, error handling, and monitoring mechanisms to ensure data accuracy and pipeline robustness.

• Develop and maintain documentation for PySpark code, data pipelines, and data workflows.

• Developed production ready Spark applications using Spark RDD apis, Data frames, Datasets, Spark SQL and Spark Streaming.

• Strong experience of HIVE Bucketing and Partitioning.

• Well versed in writing complex hive queries using analytical functions.

• Knowledge in writing custom UDF's in Hive to support custom business requirements.

• *Job Description :**

• We are looking for a skilled PySpark Developer having 4\-5 or 2\-3 years of experience to join our team.

• As a PySpark Developer, you will be responsible for developing and

• maintaining data processing pipelines using PySpark, Apache Spark's Python API.

• You will work closely with data engineers, data scientists, and other stakeholders to design and implement scalable and efficient data processing solutions.

• *Current CTC and Expected CTC to be mentioned clearly.**

• *Please email your resume, mentioning the position applied for in the subject column at :****careers@cdslindia.com**