● **Architect and Implement Scalable Data Solutions:** Design, develop, and deploy comprehensive data solutions encompassing ETL pipelines, data warehousing, data architecture, data cataloguing, and data delivery mechanisms. Ensure these solutions are scalable, reliable, and maintainable to support evolving business needs.
● **Lead Data Platform Strategy:** Develop and execute the strategic vision for the data platform, ensuring alignment with organizational objectives. Oversee the architecture and management of data systems to support efficient data storage, processing, and utilization.
● **Collaborate Cross\-Functionally:** Work closely with product managers, engineering teams, data scientists, and other stakeholders to design and implement effective data integration and hand\-off mechanisms. Facilitate seamless data flow and accessibility across various systems and teams.
● **Optimize Performance and Scalability:** Identify and address performance bottlenecks within the data platform. Implement strategies to enhance system performance, scalability, and reliability, ensuring the infrastructure can handle large\-scale data processing efficiently.
● **Oversee Data Governance and Quality:** Implement and maintain data governance frameworks to ensure data quality, accuracy, consistency, and security. Establish policies and procedures for data management, ensuring compliance with regulatory requirements and industry best practices.
● **Mentor and Lead Engineering Teams:** Provide technical leadership and mentorship to data engineers and other team members. Foster a culture of innovation, ownership, and continuous learning within the team, guiding them through complex technical challenges and project deliveries.
● **Conduct Code and Architecture Reviews:** Lead code and architecture reviews to maintain high standards of code quality and system design. Provide constructive feedback and ensure adherence to best practices and organizational standards.
● Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
● Over 5 years of software development experience, with at least 2 years in a technical leadership role overseeing data engineering or data platform teams.
● Strong proficiency in big data technologies, including Apache Spark/PySpark, Apache Airflow and Apache Kafka.
● Experience with programming languages such as Scala and Python for developing robust data pipelines.
● In\-depth knowledge of both relational and non\-relational databases, with the ability to design and optimize complex queries.
● Hands\-on experience with data visualization tools and techniques to present data insights effectively.
● Proficiency in developing and maintaining CI/CD pipelines using tools like Jenkins and GitHub Actions.
● Strong experience in writing and maintaining unit, integration, and end\-to\-end tests to ensure data pipeline reliability and accuracy.
● Excellent problem\-solving abilities with a keen attention to detail, capable of diagnosing and resolving complex technical issues.
● Strong communication skills, with the ability to convey complex technical concepts to diverse audiences.
● Proven ability to collaborate cross\-functionally with product managers, data scientists, and other engineering teams to deliver cohesive data solutions.
● Experience with Infrastructure as Code (IaC) tools such as Pulumi and Terraform for managing cloud resources.
● Prior experience working with Databricks and AWS cloud services.
● Experience with Agile development methodologies and leading Agile teams.
● Experience in working with Distributed Graph Databases.
● Familiarity with data governance and security practices to ensure compliance and data integrity.
Pay: ₹6,000,000\.00 \- ₹6,500,000\.00 per year
Work Location: Hybrid remote in Bangalore City, Bengaluru, Karnataka
Senior Java Software Engineer
Agile Technology Solutions - Your Technology Partner · Pune District, Maharashtra, India
Sr. Software Engineer(T-SQL,MySQL,SSIS,SSRS,Claude,GITlab)
Vertafore · Remote
Product Manager-Manager TD&E
EY · KL, IN