Virtusa
Snowflake Pyspark
Job Description
The Data Engineer will play a crucial role in designing, building, and maintaining scalable data pipelines and systems to support data analytics and business intelligence initiatives. This individual will work closely with data scientists, analysts, and other stakeholders to ensure data is accessible, accurate, and actionable.
Key Responsibilities
Data Pipeline Development Design, develop, and implement robust and scalable ETL Extract, Transform, Load pipelines to integrate data from various sources into data warehouses or data lakes.
Data Modeling Develop and maintain data models that support business requirements and enhance data accessibility and quality.
Database Management Manage and optimize relational and non-relational databases, ensuring high performance, reliability, and security.
Data Quality Monitor data quality and integrity, implementing data validation and cleansing processes to ensure accuracy and consistency.
Collaboration Work with data scientists, analysts, and business stakeholders to understand data needs and provide solutions to support data-driven decision-making.
Performance Tuning Optimize data processing and query performance through indexing, partitioning, and other techniques.
Documentation Create and maintain comprehensive documentation for data pipelines, data models, and other technical processes.
Troubleshooting Identify and resolve data-related issues, ensuring minimal disruption to data workflows and business operations.
Compliance Ensure that data handling practices comply with relevant regulations and industry standards.
Innovation Stay current with emerging technologies and industry trends, recommending and implementing new tools and practices to enhance data engineering capabilities.
Qualifications
Education Bachelor degree in Computer Science, Engineering, Mathematics, or a related field. Advanced degrees or certifications are a plus.
Experience Minimum of years of experience as a Data Engineer or in a similar role, with a proven track record of managing and optimizing data pipelines and databases.
Technical Skills
Proficiency in SQL and experience with relational databases e.g., PostgreSQL, MySQL .
Experience with big data technologies e.g., Hadoop, Spark and data warehousing solutions e.g., Amazon Redshift, Snowflake .
Familiarity with ETL tools and frameworks e.g., Apache NiFi, Talend .
Knowledge of programming languages such as Python, Java, or Scala.
Experience with cloud platforms e.g., AWS, Azure, Google Cloud Platform and their data services.
Analytical Skills Strong problem-solving abilities with a keen eye for detail and a commitment to delivering high-quality results.
Communication Excellent verbal and written communication skills, with the ability to effectively collaborate with cross-functional teams and present technical information to non-technical stakeholders.
Project Management Ability to manage multiple tasks and projects simultaneously, with strong organizational and time-management skills.
Preferred Qualifications
Experience with data visualization tools e.g., Tableau, Power BI .
Knowledge of data governance and data privacy practices.
Familiarity with DevOps practices and tools e.g., Docker, Kubernetes .
Jobs at Chennai
Virtusa
Java Full Stack Developer
Full Time
Chennai, Tamil Nadu
Last Date: Dec. 31, 2024
Infosys
Angular Senior Developer
3 - 5 Years Exp.
Chennai, Tamil Nadu
Last Date: Sept. 30, 2024
Amazon
Sr. Product Manager - Tech, Catalog…
Freshers/Experienced
Chennai, Tamil Nadu
Last Date: Oct. 1, 2024
More Jobs at Virtusa
Actively Recruiting Companies at Chennai, Tamil Nadu