Virtusa
Big data engineer
Job Description
- Position: Big data engineer
- Primary skills: Big data, aws
- Location: HYD
- Create Trigger based automation framework for Data Migration
Identify roles/access needed for data migration from federated bucket to managed bucket and Build APIs for the same
Integrate CDMS framework with Lake and Data bridge API
Data migration from S3 Managed to Hadoop On prem
Jobs for Daily and Bulk loads
Test support for AVRO to test lake features
Test support for Compression types like LZO, .ENC to test lake features
ABINITIO integration: Build feature to create operation trigger for ABI pipeline
Movement to new datacenter -SQL server migration
Carlstadt to Ashburn (DR switchover)
Develop and maintain data platforms using Python.
Work with AWS and Big Data, design and implement data pipelines, and ensure data quality and integrity.
Collaborate with cross functional teams to understand data requirements and design solutions that meet business needs .
Implement and manage agents for monitoring, logging, and automation within AWS environments.
Handling migration from PySpark to AWS.
(Secondary) Resource must have hands on development experience with various Ab Initio components such as Rollup Scan join Partition by key Partition by Round Robin Gather Merge Interleave Lookup etc.
Must have experience with SQL database programming SQL performance tuning relational model analysis.
Good knowledge in developing UNIX scripts Oracle SQLPLSQL.
Leverage internal tools and SDKs, utilize AWS services such as S3, Athena, and Glue, and integrate with our internal Archival Service Platform for efficient data purging.
Lead the integration efforts with the internal Archival Service Platform for seamless data purging and lifecycle management.
Collaborate with the data engineering team to continuously improve data integration pipelines, ensuring adaptability to evolving business needs.
Jobs at Hyderabad
Software Engineer III, Google Cloud
Freshers/Experienced
Hyderabad, Telangana
Last Date: Jan. 2, 2025
Virtusa
HR - Associate Consultant
Full Time
Hyderabad, Telangana
Last Date: Jan. 1, 2025
Amazon
UX Designer, AWS Fintech
Freshers/Experienced
Hyderabad, Telangana
Last Date: Jan. 7, 2025
Virtusa
Senior Java FSD Developer
Full Time
Hyderabad, Telangana
Last Date: Dec. 31, 2024
Infosys
Salesforce Consultant
5 - 7 Years Exp.
Hyderabad, Telangana
Last Date: Dec. 31, 2024
Virtusa
.Net Core FSD Developer
Full Time
Hyderabad, Telangana
Last Date: Nov. 30, 2025
Oracle
Technical Lead Architect
Professional
Hyderabad, Telangana
Last Date: March 22, 2025
Investigator, Extended Workforce In…
Freshers/Experienced
Hyderabad, Telangana
Last Date: Jan. 2, 2025
More Jobs at Virtusa
Virtusa
Sr Product Engineer Way4 (Cards)
Full Time
Chennai, Tamil Nadu
Last Date: Dec. 31, 2024
Virtusa
Senior React JS Developer
Full Time
Chennai, Tamil Nadu
Last Date: Dec. 31, 2024
Actively Recruiting Companies at Hyderabad, Telangana