Officer / Assistance Manager - Big Data / Hadoop HIVE / Spark Developer

Position Location Qualification Age Limit

• *Officer / Assistance Manager** Mumbai B.E. / MCA / B.Tech / M.sc (I.T.) 25\-30

• *Job Description :**

• Around 2\-3 years of experience in Software Development with strong focus on Big Data, Hadoop and Java.

• Strong expertise in Big Data ecosystem like Spark, Hive, Sqoop, HDFS, Map Reduce, Oozie, Yarn, HBase, Nifi.

• Strong expertise in building scalable applications using various programming languages (Java, Scala and Python)

• In depth Knowledge on Architecture of Distributed Systems and Parallel Computing.

• Experience implementing end to end data pipelines for serving reporting and data science capabilities

• Experience in fine tuning applications written in Spark and Hive and to improve the overall performance of the pipelines

• Developed production ready Spark applications using Spark RDD apis, Data frames, Datasets, Spark SQL and Spark Streaming

• Strong experience working with various configurations of Spark like broadcast thresholds, increasing shuffle partitions, caching, repartitioning etc., to improve the performance of the jobs

• Strong experience of HIVE Bucketing and Partitioning

• Well versed in writing complex hive queries using analytical functions

• Knowledge in writing custom UDF's in Hive to support custom business requirements

• Solid experience in using the various file formats like CSV, TSV, Parquet, ORC, JSON and AVRO

• Experience in using the compression techniques like Gzip, Snappy with in Hadoop

• Good to have knowledge of NoSQL databases and worked with HBase

• Experience automating end\-to\-end data pipelines with strong resilience and recoverability

• Involved in production monitoring using workflow monitor and experience in development and support environments

• Good level of experience in Core Java, J2EE technologies, JDBC, Servlets and JSP

• Good knowledge in Oracle PL/SQL and shell scripting

• *Current CTC and Expected CTC to be mentioned clearly.**

• *Please email your resume, mentioning the position applied for in the subject column at :****careers@cdslindia.com**