Virtusa

Big data engineer

Job Description

  • Position: Big data engineer
  • Primary skills: Big data, aws
  • Location: HYD
  • Create Trigger based automation framework for Data Migration
    Identify roles/access needed for data migration from federated bucket to managed bucket and Build APIs for the same
    Integrate CDMS framework with Lake and Data bridge API
    Data migration from S3 Managed to Hadoop On prem
    Jobs for Daily and Bulk loads
    Test support for AVRO to test lake features
    Test support for Compression types like LZO, .ENC to test lake features
    ABINITIO integration: Build feature to create operation trigger for ABI pipeline
    Movement to new datacenter -SQL server migration
    Carlstadt to Ashburn (DR switchover)

    Develop and maintain data platforms using Python.
    Work with AWS and Big Data, design and implement data pipelines, and ensure data quality and integrity.
    Collaborate with cross functional teams to understand data requirements and design solutions that meet business needs .
    Implement and manage agents for monitoring, logging, and automation within AWS environments.
    Handling migration from PySpark to AWS.
    (Secondary) Resource must have hands on development experience with various Ab Initio components such as Rollup Scan join Partition by key Partition by Round Robin Gather Merge Interleave Lookup etc.
    Must have experience with SQL database programming SQL performance tuning relational model analysis.
    Good knowledge in developing UNIX scripts Oracle SQLPLSQL.
    Leverage internal tools and SDKs, utilize AWS services such as S3, Athena, and Glue, and integrate with our internal Archival Service Platform for efficient data purging.
    Lead the integration efforts with the internal Archival Service Platform for seamless data purging and lifecycle management.
    Collaborate with the data engineering team to continuously improve data integration pipelines, ensuring adaptability to evolving business needs.


Jobs at Hyderabad

Google

Software Engineer III, Google Cloud

Freshers/Experienced

Hyderabad, Telangana

View Details

Last Date: Jan. 2, 2025

Virtusa

HR - Associate Consultant

Full Time

Hyderabad, Telangana

View Details

Last Date: Jan. 1, 2025

Amazon

UX Designer, AWS Fintech

Freshers/Experienced

Hyderabad, Telangana

View Details

Last Date: Jan. 7, 2025

Virtusa

Senior Java FSD Developer

Full Time

Hyderabad, Telangana

View Details

Last Date: Dec. 31, 2024

Virtusa

UNIX/LINUX Admin

Full Time

Hyderabad, Telangana

View Details

Last Date: Dec. 31, 2024

Virtusa

Abinitio - Support

Full Time

Hyderabad, Telangana

View Details

Last Date: Dec. 31, 2024

Infosys

Salesforce Consultant

5 - 7 Years Exp.

Hyderabad, Telangana

View Details

Last Date: Dec. 31, 2024

Virtusa

.Net Core FSD Developer

Full Time

Hyderabad, Telangana

View Details

Last Date: Nov. 30, 2025

Oracle

Technical Lead Architect

Professional

Hyderabad, Telangana

View Details

Last Date: March 22, 2025

Virtusa

Angular Developer

Full Time

Hyderabad, Telangana

View Details

Last Date: Dec. 31, 2024

Google

Investigator, Extended Workforce In…

Freshers/Experienced

Hyderabad, Telangana

View Details

Last Date: Jan. 2, 2025

Virtusa

MLC Developer

Full Time

Hyderabad, Telangana

View Details

Last Date: Dec. 31, 2025




More Jobs at Virtusa

Virtusa

TIBCO Developer

Full Time

Bangalore, Karnataka

View Details

Last Date: March 31, 2025

Virtusa

MDM

Full Time

Pune, Maharashtra

View Details

Last Date: Dec. 31, 2024

Virtusa

Senior Java Developer

Full Time

Hyderabad, Telangana

View Details

Last Date: March 31, 2025

Virtusa

Senior Angular Developer

Full Time

Pune, Maharashtra

View Details

Last Date: Dec. 31, 2024

Virtusa

Sr Product Engineer Way4 (Cards)

Full Time

Chennai, Tamil Nadu

View Details

Last Date: Dec. 31, 2024

Virtusa

ETL Testing

Full Time

Chennai, Tamil Nadu

View Details

Last Date: March 31, 2025

Virtusa

Azure Infra Architect

Full Time

Bangalore, Karnataka

View Details

Last Date: Dec. 31, 2024

Virtusa

Angular Developer

Full Time

Bangalore, Karnataka

View Details

Last Date: Dec. 31, 2024

Virtusa

Senior React JS Developer

Full Time

Chennai, Tamil Nadu

View Details

Last Date: Dec. 31, 2024

Virtusa

AWS DE

Full Time

Hyderabad, Telangana

View Details

Last Date: June 30, 2025

Virtusa

PMT Scrum Master

Full Time

Chennai, Tamil Nadu

View Details

Last Date: March 31, 2025

Virtusa

Unqork

Full Time

Hyderabad, Telangana

View Details

Last Date: April 7, 2025