AI Engineering

AI/ML Development: Design, develop, and deploy machine learning models, LLMs, and AI workflows within the Databricks environment.

Data Pipeline Engineering: Build scalable, efficient Delta Lake pipelines using PySpark and Python to ingest, clean, and preprocess large\-scale datasets for AI applications.

MLOps Implementation: Set up and manage the ML lifecycle using MLflow for model tracking, versioning, deployment, and monitoring.

Optimization \& Performance: Optimize PySpark jobs, Databricks clusters, and Python code for maximum performance and cost efficiency.

Collaboration \& Integration: Partner with Data Scientists, Data Engineers, and DevOps teams to integrate AI models into core business applications and APIs.

Governance \& Security: Implement Unity Catalog for data governance, access control, and lineage tracking across AI assets.