Elasticsearch Lead CDP GCP Agile
Experience
814 years overall with 5 years handson Elasticsearch architecture operations and 2 years leading delivery teams
We are looking for an Elasticsearch Lead to design build and run search and analytics capabilities for a Consumer Data Platform CDP on Google Cloud Platform GCP This role will lead the Elasticsearch engineering and operational strategycovering cluster architecture indexing strategy ingestion pipelines performance tuning observability security and reliabilitywhile working closely with product owners data engineering platformSRE and application teams in an Agile ScrumKanban delivery model
The ideal candidate has strong experience implementing Elasticsearch for highvolume customerevent data enabling near realtime search segmentation identity resolution support and analytics use cases typical of CDP ecosystems
Key Responsibilities
1 Architecture Solution Design
Own endtoend Elasticsearch architecture for CDP use cases search analytics segmentation customer 360 support behavioralevent search
Define index strategy mapping shardreplica strategy ILM policies rollover patterns data tiering and lifecycle design
Create reference architectures for Elasticsearch on GCP compute storage networking security DR
Lead design reviews and ensure alignment with enterprise standards security privacy scalability resiliency
2 Delivery Leadership Agile
Lead Elasticsearch workstream in Agile squads planning refinement estimation sprint execution demos
Translate product requirements into technical stories acceptance criteria and measurable outcomes latency throughput relevancy
Mentor engineers and review codeconfigurations pipelines mappings queries templates dashboards
Coordinate crossteam dependencies Data Engineering PlatformSRE API teams IAMSecurity Governance
3 Implementation Engineering
Implement ingestion patterns into Elasticsearch using one or more
oKafka Connect Elasticsearch
oDataflowBeam
oLogstashBeats
oPubSub pipelines
obatch ingestion via GCS Dataflow
Build search experiences and APIs leveraging
oquery DSL aggregations filters scoring synonyms analyzers multilanguage search if needed
Design and implement reindexing strategies zerodowntime migrations and schema evolution patterns
4 Performance Scale Reliability
Drive performance engineering query latency indexing throughput JVMheap tuning caching shard allocation refresh intervals
Implement high availability and scalability patterns multizone clusters autoscaling approach hotwarmcold tiers
Own operational readiness runbooks SLIsSLOs capacity planning cost optimization load testing
5 Security Governance Compliance
Implement RBAC indexlevel security encryption in transitat rest and secrets management
Ensure compliance with customer data governance retention deletion auditability and access controls
Support privacy and regulatory requirements relevant to CDP data eg consent PII handling righttoerasure workflows
6 Observability Operations
Setup monitoring and ing for cluster health node stats indexingsearch rates thread pools GC disk watermarks
Integrate with observability tools ElasticKibana PrometheusGrafana Cloud Monitoring etc
Partner with SREPlatform to define incident response DR drills upgrade strategy patching cadence
Core Elasticsearch Expertise
Strong knowledge of
ocluster architecture and distributed systems fundamentals
oindexmapping design analyzers tokenizers relevance tuning
oaggregations query optimization search templates
oILM rollover snapshots shard allocation strategies
Experience with largescale clusters TB data high QPS high ingestion rates
Expertise in troubleshooting redyellow clusters unassigned shards circuit breakers GC pressure slow logs
Cloud Platform GCP
Handson experience deployingrunning Elasticsearch on GCP using
oGCE GKE managed instance groups autoscaling
oVPC design firewalling private connectivity
oCloud Storage for snapshots and backup
oCloud MonitoringLogging integration
Understanding of infrastructure automation
oTerraform IaC
oHelmKubernetes manifests if on GKE
oCICD pipelines for configuration and deployment
Data Integration
Strong understanding of CDPstyle data
oconsumer profiles identities eventsclickstream attributes segments
Integration patterns with
oKafka PubSub DataflowBeam Spark BigQuery
Experience modeling eventtime and profile data for search analytics
Agile Leadership
Proven ability to lead teams in Agile delivery sprint ceremonies story slicing estimation incremental delivery
Strong communication and stakeholder management ability to drive decisions
LTM is an AI\-centric global technology services company and the Business Creativity partner to the world’s largest and most disruptive enterprises. We bring human insights and intelligent systems together to help clients create greater value at the intersection of technology and domain expertise. Our capabilities span integrated operations, transformation, and business AI — enabling new ways of working, new productivity paradigms, and new roads to value. Together with over 87,000 employees across 40 countries and our global network of partners, LTM — a Larsen \& Toubro company — owns business outcomes for our clients, helping them not just outperform the market, but to Outcreate it. Please also note that neither LTM nor any of its authorized recruitment agencies/partners charge any candidate registration fee or any other fees from talent (candidates) towards appearing for an interview or securing employment/internship. Candidates shall be solely responsible for verifying the credentials of any agency/consultant that claims to be working with LTM for recruitment. Please note that anyone who relies on the representations made by fraudulent employment agencies does so at their own risk, and LTM disclaims any liability in case of loss or damage suffered as a consequence of the same. Recruitment Fraud Alert \- https://www.ltimindtree.com/recruitment\-fraud\-alert/
IN-Senior Associate_ Full Stack Developer__ Data & Analytics _ Advisory _Bangalore
PwC · Bengaluru
Principal Engineer - Java with Gen AI
Wells Fargo · Bengaluru
Python Full Stack Developer Intern
AEVEVO TECHNOLOGY PRIVATE LIMITED · Chennai