Accolite
Data Engineer Lead
Job Description
- Implement complex big data solutions with a focus on collecting, parsing, and manipulating large datasets using data engineering tools and platforms such as Spark (Scala), Hadoop, and distributed processing platforms.
- Develop and maintain big data pipelines for ingestion, transformation, and consumption in production settings, ensuring data quality assessments in a Big Data environment.
- Collaborate with various teams within the Big Data Platform, including C360, Data Lake, Dev Ops, QA, and infrastructure to design and maintain efficient data solutions.
- Create and maintain big data reference architectures and uphold production systems like Kafka, Hadoop, Cassandra, and Elasticsearch.
- Stay updated with the latest developments in the field of big data technologies and contribute to the tuning of Hadoop solutions to enhance performance and end-user experience.
- Manage, monitor, and automate tasks related to Hadoop clusters and platform infrastructure, ensuring optimal performance and reliability of the systems.
- Perform performance tuning of Hadoop clusters and Hive queries while providing technical expertise in performance improvement and fine-tuning of ETL Hadoop implementations.
- Take ownership of the end-to-end Hadoop lifecycle within the organization, from installation and configuration to maintenance and optimization of Hadoop clusters and related technologies.
Requirements:
- Bachelor’s degree in Computer Science, Information Technology, or related fields.
- Strong expertise and hands-on experience in big data technologies such as Apache Spark, Hadoop, Kafka, etc.
- Proficiency in scripting languages like Shell, Perl, and Python for data processing and automation tasks.
- Experience in setting up and managing Hadoop clusters, including configuration management and security implementation.
- Strong analytical and problem-solving skills with a good understanding of data modelling and distributed SQL query engines.
- Prior exposure to non-relational and relational databases (SQL, MySQL, NoSQL, MongoDB) and big data stacks is preferred.
- Ability to work collaboratively in a fast-paced environment, demonstrating excellent communication and interpersonal skills.
- Familiarity with cloud technologies, AWS services, and performance improvement techniques in ETL Hadoop implementations would be advantageous.
Jobs at Bangalore
KPMG
Technology Enablement SAP-BI/BW-SAC…
KI Professional
Bangalore, Karnataka
Last Date: Oct. 29, 2025
Amazon
Senior Delivery Consultant - AI/ML …
Freshers/Experienced
Bangalore, Karnataka
Last Date: Nov. 4, 2025
KPMG
Executive - TPRM-Advisory Services
KI Professional
Bangalore, Karnataka
Last Date: Oct. 29, 2025
KPMG
Risk Advisory Finance Advisory Exec…
KI Professional
Bangalore, Karnataka
Last Date: Oct. 29, 2025
Amazon
SDET, Alexa Audio
Freshers/Experienced
Bangalore, Karnataka
Last Date: Oct. 28, 2025
Amazon
Business Intelligence Engineer (BIE…
Freshers/Experienced
Bangalore, Karnataka
Last Date: Oct. 28, 2025
Amazon
Research Scientist, Last Mile Scien…
Freshers/Experienced
Bangalore, Karnataka
Last Date: Nov. 4, 2025
KPMG
Senior - SAP-CPI/BTP
KI Professional
Bangalore, Karnataka
Last Date: Oct. 29, 2025
Amazon
Account Manager, VSP
Freshers/Experienced
Bangalore, Karnataka
Last Date: Oct. 28, 2025
Amazon
Sr. Delivery Consultant, Customer E…
Freshers/Experienced
Bangalore, Karnataka
Last Date: Oct. 28, 2025
More Jobs at Accolite
Accolite
Software Developer- UI Angular & .…
8 - 12 Years Exp.
Bangalore, Karnataka
Last Date: Nov. 8, 2025
Accolite
Broadridge Regulatory Reporting SME…
6 - 15 Years Exp.
Bangalore, Karnataka
Last Date: Nov. 8, 2025
Accolite
Murex Collateral Consultant
2 - 6 Years Exp.
Bangalore, Karnataka
Last Date: Nov. 8, 2025
Accolite
Data Engineer Lead
7 - 12 Years Exp.
Bangalore, Karnataka
Last Date: Nov. 8, 2025
Accolite
Murex Integration/Reporting Consult…
5 - 15 Years Exp.
Gurugram, Haryana
Last Date: Nov. 1, 2025
Accolite
Murex Integration Specialist
5 - 15 Years Exp.
Bangalore, Karnataka
Last Date: Nov. 8, 2025
Accolite
Sr. iOS Developer
3 - 7 Years Exp.
Bangalore, Karnataka
Last Date: Nov. 8, 2025
Accolite
Senior Data Engineer
4 - 8 Years Exp.
Gurugram, Haryana
Last Date: Nov. 1, 2025
Accolite
Senior Software Developer – C# and …
8 - 15 Years Exp.
Bangalore, Karnataka
Last Date: Nov. 8, 2025
Accolite
Director, IT Operations Engineering
12 - 16 Years Exp.
Bangalore, Karnataka
Last Date: Nov. 8, 2025
Actively Recruiting Companies at Bangalore, Karnataka