IBM

Data Engineer

Posted On

Oct. 24, 2024

Location

Hyderabad, Telangana

Experience Level

Professional

Job Type

Permanent

Apply By

Nov. 22, 2024

Job Description

Introduction
At IBM, work is more than a job - it's a calling: To build. To design. To code. To consult. To think along with clients and sell. To make markets. To invent. To collaborate. Not just to do something better, but to attempt things you've never thought possible. Are you ready to lead in this new era of technology and solve some of the world's most challenging problems? If so, lets talk.

Your Role and Responsibilities
As a Data Engineer to join our dynamic, global team. This role requires strong expertise in Spark, databases, and cloud technologies to design, implement, and optimize scalable data solutions that support business requirements. The ideal candidate will have experience with data pipelines, database management, and testing methodologies in a distributed, fast-paced environment.

Responsibilities:

Design, develop, and optimize scalable data pipelines and architectures using Apache Spark and databases.
Work with cloud platforms (IBM Cloud) to build robust data solutions and ensure seamless integration.
Collaborate closely with cross-functional teams to gather and understand business requirements and translate them into efficient data models and workflows.
Implement and maintain data structures, ensuring the integrity, availability, and scalability of the data.
Develop and implement testing strategies for data validation, ensuring high data quality and system performance.
Support the team in data analysis and insights to enable data-driven decision-making.
Contribute to documentation and best practices to facilitate knowledge sharing within the global team.
Participate in regular syncs with the worldwide team, ensuring clear communication and alignment across time zones.

High level job responsibilities:

Develop elegant, flexible, maintainable, and scalable solutions to complex problems by delivering data and analytics.
Demonstrate experience leading development efforts and coordinating work for junior developers.
Define and consolidate development and modelling practices within a team.
Leverage strong collaborative skills to define requirements, develop plans, and deliver iteratively to support our organization’s mission.
Leverage strong analytical skills to understand business processes and propose effective data solutions.
Communicate and manage relationships with business and development teams - provide guidance, mentorship, and direction, as required.
Translate business needs into technical requirements.
Learn new tools, technologies, and processes for continuous improvement.
As part of a self-directed team, take ownership of activities and deliver them on-time and with quality.

Required Technical and Professional Expertise

Above all, we value curiosity, teamwork, and a desire to learn. We are confident that if you possess the right attitude, work ethic, and skill set that you could succeed in the role, even if you do not meet every one of the requirements below.

5+ years of experience in Data Engineering with Big Data.
Comfortable multi-tasking and working as part of a global team, as well as providing technical leadership and taking ownership.
Adaptive to ambiguity and willing to change in a fast-paced environment.
Advanced proficiency in Python, Scala, and PySpark.
Advanced experience with SQL for complex queries and data manipulation.
Expertise in developing and maintaining data pipelines using Apache Spark and Scala.
Experience making continuous documentation improvements and maintaining clear and concise technical documentation.
Strong skills in data modeling, including designing complex, dimensional data models.
Strong understanding of Coud platforms, particularly IBM Cloud and Cloud Object Storage.
Strong understanding and application of clean code principles and best development practices.
Proficiency in data validation, testing, and ensuring data quality.
Proficiency in creating and managing workflows with Apache Airflow and Argo Workflows.
Experience in test-driven development (TDD) and continuous improvement of code quality.
Familiarity with CI/CD tools like Tekton/Jenkis plus experience in setting up and maintaining CI/CD pipelines.
Experience in data engineering or business intelligence roles contributing to a shared codebase.
Experience with data structures, algorithms, software design and writing software in Python, Scala, Java, or similar.
Experience with relational databases including writing and optimizing SQL queries and designing schema.
Experience using continuous integration and deployment systems (e.g., Cloud Build, GitLab, Jenkins).
Experience with OpenShift / Kubernetes.

Preferred Technical and Professional Expertise

Experience in one or more of the following: data protection, disaster recovery, storage, Relational Databases (Postgres, MYSQL), No-SQL Databases (Cloudant, MongoDB), messaging queues (Kafka) and K-V stores (Redis) and storage technologies (block/file/object/CSI).
Experience writing unit tests (e.g., PyTest, Selenium, JUnit).
Basic understanding of machine learning

About Business Unit

The IBM Finance organization is responsible for driving enterprise performance and transformation. We are the financial stewards of IBM, delivering IBM’s financial strategy, developing new business models, and mitigating enterprise risk. Do you have a passion for creating business value? Join our team in accounting, financial planning, pricing, business controls, tax, treasury, business development (acquisitions & divestitures), and global financing.

Your Life @ IBM

In a world where technology never stands still, we understand that, dedication to our clients success, innovation that matters, and trust and personal responsibility in all our relationships, lives in what we do as IBMers as we strive to be the catalyst that makes the world work better.

Being an IBMer means you’ll be able to learn and develop yourself and your career, you’ll be encouraged to be courageous and experiment everyday, all whilst having continuous trust and support in an environment where everyone can thrive whatever their personal or professional background.

Our IBMers are growth minded, always staying curious, open to feedback and learning new information and skills to constantly transform themselves and our company. They are trusted to provide on-going feedback to help other IBMers grow, as well as collaborate with colleagues keeping in mind a team focused approach to include different perspectives to drive exceptional outcomes for our customers. The courage our IBMers have to make critical decisions everyday is essential to IBM becoming the catalyst for progress, always embracing challenges with resources they have to hand, a can-do attitude and always striving for an outcome focused approach within everything that they do.

Are you ready to be an IBMer?

About IBM

IBM's greatest invention is the IBMer. We believe that through the application of intelligence, reason and science, we can improve business, society and the human condition, bringing the power of an open hybrid cloud and AI strategy to life for our clients and partners around the world.

Restlessly reinventing since 1911, we are not only one of the largest corporate organizations in the world, we’re also one of the biggest technology and consulting employers, with many of the Fortune 50 companies relying on the IBM Cloud to run their business.

At IBM, we pride ourselves on being an early adopter of artificial intelligence, quantum computing and blockchain. Now it’s time for you to join us on our journey to being a responsible technology innovator and a force for good in the world.

Location Statement

When applying to jobs of your interest, we recommend that you do so for those that match your experience and expertise. Our recruiters advise that you apply to not more than 3 roles in a year for the best candidate experience.

For additional information about location requirements, please discuss with the recruiter following submission of your application.

Being You @ IBM

IBM is committed to creating a diverse environment and is proud to be an equal-opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, caste, genetics, pregnancy, disability, neurodivergence, age, veteran status, or other characteristics. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.

Jobs at Hyderabad

Virtusa

IOS Developer

Full Time

Hyderabad, Telangana

Data Engineer Apply Now

Job Description

Jobs at Hyderabad

More Jobs at IBM

Actively Recruiting Companies at Hyderabad, Telangana

Data Engineer