Accolite

Data Engineer Lead

Job Description

  • Implement complex big data solutions with a focus on collecting, parsing, and manipulating large datasets using data engineering tools and platforms such as Spark (Scala), Hadoop, and distributed processing platforms.
  • Develop and maintain big data pipelines for ingestion, transformation, and consumption in production settings, ensuring data quality assessments in a Big Data environment.
  • Collaborate with various teams within the Big Data Platform, including C360, Data Lake, Dev Ops, QA, and infrastructure to design and maintain efficient data solutions.
  • Create and maintain big data reference architectures and uphold production systems like Kafka, Hadoop, Cassandra, and Elasticsearch.
  • Stay updated with the latest developments in the field of big data technologies and contribute to the tuning of Hadoop solutions to enhance performance and end-user experience.
  • Manage, monitor, and automate tasks related to Hadoop clusters and platform infrastructure, ensuring optimal performance and reliability of the systems.
  • Perform performance tuning of Hadoop clusters and Hive queries while providing technical expertise in performance improvement and fine-tuning of ETL Hadoop implementations.
  • Take ownership of the end-to-end Hadoop lifecycle within the organization, from installation and configuration to maintenance and optimization of Hadoop clusters and related technologies.

Requirements:

  • Bachelor’s degree in Computer Science, Information Technology, or related fields.
  • Strong expertise and hands-on experience in big data technologies such as Apache Spark, Hadoop, Kafka, etc.
  • Proficiency in scripting languages like Shell, Perl, and Python for data processing and automation tasks.
  • Experience in setting up and managing Hadoop clusters, including configuration management and security implementation.
  • Strong analytical and problem-solving skills with a good understanding of data modelling and distributed SQL query engines.
  • Prior exposure to non-relational and relational databases (SQL, MySQL, NoSQL, MongoDB) and big data stacks is preferred.
  • Ability to work collaboratively in a fast-paced environment, demonstrating excellent communication and interpersonal skills.
  • Familiarity with cloud technologies, AWS services, and performance improvement techniques in ETL Hadoop implementations would be advantageous.


Jobs at Bangalore

KPMG

Technology Enablement SAP-BI/BW-SAC…

KI Professional

Bangalore, Karnataka

View Details

Last Date: Oct. 29, 2025

Amazon

Senior Delivery Consultant - AI/ML …

Freshers/Experienced

Bangalore, Karnataka

View Details

Last Date: Nov. 4, 2025

KPMG

Executive - TPRM-Advisory Services

KI Professional

Bangalore, Karnataka

View Details

Last Date: Oct. 29, 2025

Virtusa

React js

Full Time

Bangalore, Karnataka

View Details

Last Date: Dec. 31, 2025

KPMG

Risk Advisory Finance Advisory Exec…

KI Professional

Bangalore, Karnataka

View Details

Last Date: Oct. 29, 2025

Amazon

SDET, Alexa Audio

Freshers/Experienced

Bangalore, Karnataka

View Details

Last Date: Oct. 28, 2025

Amazon

Business Intelligence Engineer (BIE…

Freshers/Experienced

Bangalore, Karnataka

View Details

Last Date: Oct. 28, 2025

Amazon

Research Scientist, Last Mile Scien…

Freshers/Experienced

Bangalore, Karnataka

View Details

Last Date: Nov. 4, 2025

KPMG

Senior - SAP-CPI/BTP

KI Professional

Bangalore, Karnataka

View Details

Last Date: Oct. 29, 2025

Amazon

Account Manager, VSP

Freshers/Experienced

Bangalore, Karnataka

View Details

Last Date: Oct. 28, 2025

KPMG

Senior - SAP-CPI

KI Professional

Bangalore, Karnataka

View Details

Last Date: Oct. 29, 2025

Amazon

Sr. Delivery Consultant, Customer E…

Freshers/Experienced

Bangalore, Karnataka

View Details

Last Date: Oct. 28, 2025




More Jobs at Accolite

Accolite

Software Developer- UI Angular & .…

8 - 12 Years Exp.

Bangalore, Karnataka

View Details

Last Date: Nov. 8, 2025

Accolite

SAC Analyst

5 - 8 Years Exp.

Bangalore, Karnataka

View Details

Last Date: Nov. 8, 2025

Accolite

Broadridge Regulatory Reporting SME…

6 - 15 Years Exp.

Bangalore, Karnataka

View Details

Last Date: Nov. 8, 2025

Accolite

Murex Collateral Consultant

2 - 6 Years Exp.

Bangalore, Karnataka

View Details

Last Date: Nov. 8, 2025

Accolite

Data Engineer Lead

7 - 12 Years Exp.

Bangalore, Karnataka

View Details

Last Date: Nov. 8, 2025

Accolite

Murex Integration/Reporting Consult…

5 - 15 Years Exp.

Gurugram, Haryana

View Details

Last Date: Nov. 1, 2025

Accolite

Murex Integration Specialist

5 - 15 Years Exp.

Bangalore, Karnataka

View Details

Last Date: Nov. 8, 2025

Accolite

Sr. iOS Developer

3 - 7 Years Exp.

Bangalore, Karnataka

View Details

Last Date: Nov. 8, 2025

Accolite

Senior Data Engineer

4 - 8 Years Exp.

Gurugram, Haryana

View Details

Last Date: Nov. 1, 2025

Accolite

React lead

10 - 15 Years Exp.

Gurugram, Haryana

View Details

Last Date: Nov. 1, 2025

Accolite

Senior Software Developer – C# and …

8 - 15 Years Exp.

Bangalore, Karnataka

View Details

Last Date: Nov. 8, 2025

Accolite

Director, IT Operations Engineering

12 - 16 Years Exp.

Bangalore, Karnataka

View Details

Last Date: Nov. 8, 2025




Actively Recruiting Companies at Bangalore, Karnataka