Virtusa
Senior Data Engineer
Job Description
Develop and Maintain ETL Pipelines: Design, develop, and implement scalable ETL workflows using PySpark, Python, and AWS Glue, Databricks.
Data Transformation and Integration: Extract, transform, and load data from various sources to AWS S3 and Redshift.
Performance Optimization: Identify and resolve performance bottlenecks in ETL processes, ensuring optimal performance across large datasets.
Debugging & Reverse Engineering: Should be able to debug PySpark programs/Job and reverse Engineer the code.
Automation and Monitoring: Implement automation scripts using AWS Lambda, Step Functions to schedule and monitor data pipelines.
Data Quality: Ensure data integrity and quality across all stages of the ETL pipeline.
Collaboration: Work closely with data architects, analysts, and stakeholders to understand requirements and provide clear communication throughout the project lifecycle.
Documentation: Create and maintain technical documentation, including data mapping, workflow designs, and ETL processes.
Knowledge of CI/CD pipelines and best practices in deployment automation.
Jobs at Bangalore
Infosys
Project Manager - Selenium
8 - 10 Years Exp.
Bangalore, Karnataka
Last Date: March 31, 2025
Amazon
Software Development Manager, Alexa…
Freshers/Experienced
Bangalore, Karnataka
Last Date: Jan. 28, 2025
Capgemini
Network Routing & Switching | 4 to …
Experienced Professionals
Bangalore, Karnataka
Last Date: Feb. 5, 2025
Amazon
Lab Support engineer, Ring Software…
Freshers/Experienced
Bangalore, Karnataka
Last Date: Jan. 28, 2025
Amazon
Business Analyst, OTS-Service Desk
Freshers/Experienced
Bangalore, Karnataka
Last Date: Jan. 28, 2025
Capgemini
Senior Software Engineer | SAP BRIM…
Experienced Professionals
Bangalore, Karnataka
Last Date: Feb. 5, 2025
Capgemini
Test Automation
Experienced Professionals
Bangalore, Karnataka
Last Date: Feb. 5, 2025
Capgemini
Finance transformation strategy | S…
Experienced Professionals
Bangalore, Karnataka
Last Date: Feb. 5, 2025
More Jobs at Virtusa
Virtusa
salesforce financial services T3
Full Time
Bangalore, Karnataka
Last Date: Sept. 30, 2025
Virtusa
Java Microservices Developer
Full Time
Chennai, Tamil Nadu
Last Date: Feb. 28, 2025
Actively Recruiting Companies at Bangalore, Karnataka