Lead I - Data Engineering

• *Role description**

• *Job Title: PySpark Developer (AWS)**
• *Experience:** 5–8 Years

• *Location:** Hyderabad

• *Employment Type:** Full time

### **Job Summary**

We are looking for an experienced PySpark Developer with strong expertise in AWS to design, develop, and optimize scalable data processing solutions. The ideal candidate should have hands\-on experience in big data technologies, distributed computing, and cloud\-based data platforms.

### **Key Responsibilities**

• Design, develop, and maintain scalable data pipelines using **PySpark**
• Work with large datasets and ensure efficient data processing and transformation
• Develop and optimize ETL workflows using **AWS services**
• Collaborate with data engineers, analysts, and stakeholders to meet business requirements
• Implement data quality checks and ensure data integrity
• Monitor and troubleshoot production data pipelines
• Optimize performance of Spark jobs and AWS resources
• Maintain documentation and follow best practices in data engineering

### **Required Skills \& Qualifications**

• 5–8 years of experience in data engineering or related roles
• Strong hands\-on experience with **PySpark and Apache Spark**
• Experience working with **AWS services** such as:

+ S3, Glue, EMR, Lambda, Redshift

• Proficiency in **Python** and SQL
• Experience in building ETL/ELT pipelines
• Familiarity with **data warehousing concepts**
• Good understanding of distributed computing
• Experience with version control systems (Git)

### **Preferred Skills**

• Experience with **Airflow** or other orchestration tools
• Knowledge of **Delta Lake / Iceberg / Hudi**
• Familiarity with **CI/CD pipelines**
• Exposure to streaming frameworks (Kafka, Spark Streaming)
• Understanding of data governance and security best practices

### **Soft Skills**

• Strong analytical and problem\-solving abilities
• Good communication and collaboration skills
• Ability to work in an agile environment

### **Nice to Have**

• AWS certifications (e.g., AWS Data Engineer, Solutions Architect)
• Experience in domain\-specific projects (Finance, Banking, etc.)

• *Skills**

pyspark,spark,spark mllib,amazon web services,

• *About UST**

UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world’s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future\-proof philosophy, UST embeds innovation and agility into their clients’ organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact—touching billions of lives in the process.