Big Data Engineer

• *We’re Hiring: Big Data Engineer**

Are you passionate about building scalable big data solutions and high\-performance data pipelines? Join our team and work with cutting\-edge technologies to develop enterprise\-grade data platforms.

• *Experience Required**

• 4\+ years of software engineering experience developing Big Data pipelines.
• 3\+ years of hands\-on experience programming in Scala and PySpark.
• 3\+ years of experience building solutions on AWS, including AWS Glue.
• 3\+ years of experience working with Big Data file formats, including Parquet.
• 1\+ years of experience with Data Validation, Data Profiling, or building Rule\-Based Data Engines.
• 1\+ years of experience leveraging AI coding assistants or code\-generation tools (e.g., GitHub Copilot, Codex) in daily software development tasks.

• *Key Responsibilities**

• Design, build, and optimize a highly scalable, rule\-based data validation engine.
• Develop robust, high\-performance Big Data pipelines using Scala, PySpark, and AWS Glue.
• Write clean, well\-tested code utilizing optimized formats like Parquet for efficient storage and retrieval.
• Perform detailed data validation and profiling to ensure data quality and consistency across data systems.
• Use enterprise\-approved AI tools to streamline software development workflows and automate repetitive tasks.
• Collaborate with cross\-functional teams to translate business requirements into scalable technical solutions.
• Evaluate emerging AI and Big Data technologies to drive innovation.
• Participate in code reviews, testing, debugging, and performance optimization.

• *Required Skills**

• Scala
• PySpark
• AWS
• AWS Glue
• Parquet
• Big Data Pipelines
• Data Validation
• Data Profiling
• AI Coding Assistants (GitHub Copilot/Codex)