Accolite
Data Engineer Lead
Job Description
- Implement complex big data solutions with a focus on collecting, parsing, and manipulating large datasets using data engineering tools and platforms such as Spark (Scala), Hadoop, and distributed processing platforms.
- Develop and maintain big data pipelines for ingestion, transformation, and consumption in production settings, ensuring data quality assessments in a Big Data environment.
- Collaborate with various teams within the Big Data Platform, including C360, Data Lake, Dev Ops, QA, and infrastructure to design and maintain efficient data solutions.
- Create and maintain big data reference architectures and uphold production systems like Kafka, Hadoop, Cassandra, and Elasticsearch.
- Stay updated with the latest developments in the field of big data technologies and contribute to the tuning of Hadoop solutions to enhance performance and end-user experience.
- Manage, monitor, and automate tasks related to Hadoop clusters and platform infrastructure, ensuring optimal performance and reliability of the systems.
- Perform performance tuning of Hadoop clusters and Hive queries while providing technical expertise in performance improvement and fine-tuning of ETL Hadoop implementations.
- Take ownership of the end-to-end Hadoop lifecycle within the organization, from installation and configuration to maintenance and optimization of Hadoop clusters and related technologies.
Requirements:
- Bachelor’s degree in Computer Science, Information Technology, or related fields.
- Strong expertise and hands-on experience in big data technologies such as Apache Spark, Hadoop, Kafka, etc.
- Proficiency in scripting languages like Shell, Perl, and Python for data processing and automation tasks.
- Experience in setting up and managing Hadoop clusters, including configuration management and security implementation.
- Strong analytical and problem-solving skills with a good understanding of data modelling and distributed SQL query engines.
- Prior exposure to non-relational and relational databases (SQL, MySQL, NoSQL, MongoDB) and big data stacks is preferred.
- Ability to work collaboratively in a fast-paced environment, demonstrating excellent communication and interpersonal skills.
- Familiarity with cloud technologies, AWS services, and performance improvement techniques in ETL Hadoop implementations would be advantageous.
Jobs at Hyderabad
Virtusa
Platform Support Engineer
Full Time
Hyderabad, Telangana
Last Date: Jan. 31, 2026
Amazon
Quality Specialist - Japanese
Freshers/Experienced
Hyderabad, Telangana
Last Date: Jan. 27, 2026
Oracle
Oracle CX Marketing Expert Eloqua a…
Professional
Hyderabad, Telangana
Last Date: May 3, 2026
Amazon
Software Development Engineer II, G…
Freshers/Experienced
Hyderabad, Telangana
Last Date: Jan. 27, 2026
Oracle
Consulting Project Principal Consul…
Professional
Hyderabad, Telangana
Last Date: May 11, 2026
Amazon
Manager II, ML Data Operations, ABD…
Freshers/Experienced
Hyderabad, Telangana
Last Date: Jan. 27, 2026
Accolite
AI Product Manager
10 - 15 Years Exp.
Hyderabad, Telangana
Last Date: Jan. 24, 2026
Accolite
FS Developer (React Next AEM)
4 - 6 Years Exp.
Hyderabad, Telangana
Last Date: Jan. 24, 2026
Amazon
Risk Manager, Selling Partner Ident…
Freshers/Experienced
Hyderabad, Telangana
Last Date: Jan. 27, 2026
Amazon
Transportation Specialist, ATS Line…
Freshers/Experienced
Hyderabad, Telangana
Last Date: Jan. 27, 2026
Accolite
TIBCO APIGEE Java
5 - 9 Years Exp.
Hyderabad, Telangana
Last Date: Jan. 24, 2026
Amazon
Systems Admin, Global Data Analytic…
Freshers/Experienced
Hyderabad, Telangana
Last Date: Jan. 27, 2026
More Jobs at Accolite
Accolite
Java + AWS Developer
10 - 15 Years Exp.
Hyderabad, Telangana
Last Date: Jan. 24, 2026
Accolite
Senior Full Stack Developer – Angul…
7 - 12 Years Exp.
Hyderabad, Telangana
Last Date: Jan. 24, 2026
Accolite
Senior Backend Developer- AEM, Java…
4 - 6 Years Exp.
Hyderabad, Telangana
Last Date: Jan. 24, 2026
Accolite
Junior Cloud Engineer
2 - 3 Years Exp.
Hyderabad, Telangana
Last Date: Jan. 24, 2026
Accolite
Delivery Director
15 - 25 Years Exp.
Hyderabad, Telangana
Last Date: Jan. 24, 2026
Accolite
AWS Data Engineer
5 - 10 Years Exp.
Hyderabad, Telangana
Last Date: Jan. 24, 2026
Accolite
QA Automation Engineer
5 - 7 Years Exp.
Hyderabad, Telangana
Last Date: Jan. 24, 2026
Accolite
Systems Analyst 2
3 - 5 Years Exp.
Hyderabad, Telangana
Last Date: Jan. 24, 2026
Accolite
ServiceNow Senior Developer
5 - 8 Years Exp.
Hyderabad, Telangana
Last Date: Jan. 24, 2026
Accolite
TIBCO EMS APIGEE
5 - 9 Years Exp.
Hyderabad, Telangana
Last Date: Jan. 24, 2026