Data Engineer-Customer Analytics AIML

Required Experience, minimum: 4 years

Required Experience, maximum: 6 years

• Skills: Python, pyspark, AWS, Git, CI/CD Pipeline, Terraform

Minimum Qualification: B Tech

• M Tech

• *What awaits you/ Job Profile**

Analyze, Organize data and Build data systems with multiple data sources

• · Working closely with Stakeholders and Cross functional teams for the Data analysis.

• *What should you bring along**

• Build data systems and pipelines
• Standardize the data assets which maximize data reusability
• Evaluate business needs and objectives
• Implementing data pipeline to ingest and cleanse raw data from various source systems including SaaS, Cloud based and On\-prep databases
• Conduct complex data analysis and report on results
• Combine raw information from different sources
• Explore ways to enhance data quality and reliability
• Implementing functional requirements including incremental loads, data snapshots and identity resolution
• Optimizing data pipelines to improve performance and cost, while ensuring a high quality of data within the data lake:
• Monitoring services and jobs for cost and performance, ensuring continual operations of data pipelines, and fixing of defects.
• Constantly looking for opportunities to optimize data pipelines to improve performance.

• *Must have technical skill**

• · Must have coding skills in Spark/Pyspark, Python and SQL

· Must have Knowledge of AWS tools, Glue, Athena ,step functions and S3

• *Good to have technical skills**

• Good knowledge of CI/CD (Github / Github Actions, Terraform)
• Knowledge on Agile methodologies.