Accolite
Data Engineer Lead
Job Description
- Implement complex big data solutions with a focus on collecting, parsing, and manipulating large datasets using data engineering tools and platforms such as Spark (Scala), Hadoop, and distributed processing platforms.
- Develop and maintain big data pipelines for ingestion, transformation, and consumption in production settings, ensuring data quality assessments in a Big Data environment.
- Collaborate with various teams within the Big Data Platform, including C360, Data Lake, Dev Ops, QA, and infrastructure to design and maintain efficient data solutions.
- Create and maintain big data reference architectures and uphold production systems like Kafka, Hadoop, Cassandra, and Elasticsearch.
- Stay updated with the latest developments in the field of big data technologies and contribute to the tuning of Hadoop solutions to enhance performance and end-user experience.
- Manage, monitor, and automate tasks related to Hadoop clusters and platform infrastructure, ensuring optimal performance and reliability of the systems.
- Perform performance tuning of Hadoop clusters and Hive queries while providing technical expertise in performance improvement and fine-tuning of ETL Hadoop implementations.
- Take ownership of the end-to-end Hadoop lifecycle within the organization, from installation and configuration to maintenance and optimization of Hadoop clusters and related technologies.
Requirements:
- Bachelor’s degree in Computer Science, Information Technology, or related fields.
- Strong expertise and hands-on experience in big data technologies such as Apache Spark, Hadoop, Kafka, etc.
- Proficiency in scripting languages like Shell, Perl, and Python for data processing and automation tasks.
- Experience in setting up and managing Hadoop clusters, including configuration management and security implementation.
- Strong analytical and problem-solving skills with a good understanding of data modelling and distributed SQL query engines.
- Prior exposure to non-relational and relational databases (SQL, MySQL, NoSQL, MongoDB) and big data stacks is preferred.
- Ability to work collaboratively in a fast-paced environment, demonstrating excellent communication and interpersonal skills.
- Familiarity with cloud technologies, AWS services, and performance improvement techniques in ETL Hadoop implementations would be advantageous.
Jobs at Chennai
Amazon
Business Analyst, Amazon Selection …
Freshers/Experienced
Chennai, Tamil Nadu
Last Date: Sept. 16, 2025
Amazon
Software Development Manager III, A…
Freshers/Experienced
Chennai, Tamil Nadu
Last Date: Sept. 16, 2025
Amazon
Software Development Engineer, Digi…
Freshers/Experienced
Chennai, Tamil Nadu
Last Date: Sept. 16, 2025
Oracle
Senior Systems Analyst - Support
Professional
Chennai, Tamil Nadu
Last Date: Sept. 16, 2025
More Jobs at Accolite
Accolite
Sr. iOS Developer
5 - 12 Years Exp.
Chennai, Tamil Nadu
Last Date: Sept. 27, 2025
Accolite
SAP BASIS and SAC datasphere SAP Te…
5 - 8 Years Exp.
Chennai, Tamil Nadu
Last Date: Sept. 27, 2025
Accolite
Murex Market Risk Consultant
5 - 10 Years Exp.
Chennai, Tamil Nadu
Last Date: Sept. 27, 2025
Accolite
Lead ML Integration Engineer
6 - 9 Years Exp.
Chennai, Tamil Nadu
Last Date: Sept. 27, 2025
Accolite
Copilot Power Apps Consultant
10 - 15 Years Exp.
Chennai, Tamil Nadu
Last Date: Sept. 27, 2025
Accolite
Google Cloud Developer
5 - 7 Years Exp.
Chennai, Tamil Nadu
Last Date: Sept. 27, 2025
Accolite
Murex Accounting
4 - 15 Years Exp.
Chennai, Tamil Nadu
Last Date: Sept. 27, 2025
Accolite
Data Engineer Lead
7 - 12 Years Exp.
Chennai, Tamil Nadu
Last Date: Sept. 27, 2025
Actively Recruiting Companies at Chennai, Tamil Nadu