Accolite

Data Engineer – Databricks / PySpark / Spark

Posted On

Dec. 2, 2025

Location

Hyderabad, Telangana

Experience Level

5 - 7 Years Exp.

Job Type

Full Time

Apply By

Jan. 24, 2026

Job Description

Job Summary:

We are seeking an experienced Data Engineer with strong hands-on expertise in Databricks, PySpark, and Apache Spark to design and develop scalable data pipelines for analytics and operational workloads. The ideal candidate will work across data ingestion, transformation, optimization, and orchestration in cloud-based environments.

Key Responsibilities:

Design and build ETL/ELT pipelines using Databricks, PySpark, and Spark-based frameworks.

Develop reusable and high-performance data processing workflows for batch and streaming workloads.

Integrate data from multiple structured and unstructured sources into cloud data platforms.

Optimize Spark jobs for performance, reliability, and cost efficiency.

Work with Delta Lake or similar lakehouse architectures for data versioning and ACID compliance.

Collaborate with data architects, analysts, and application teams to deliver clean, reliable datasets.

Write efficient code in PySpark, SQL, and Python.

Implement CI/CD for data workflows and follow best practices in data governance and security.

Troubleshoot data pipeline issues and support production workloads.

Required Qualifications:

5-7 years of experience in data engineering and distributed data processing.

Strong hands-on experience in Databricks, PySpark, and Apache Spark.

Proficient in SQL for transformations, profiling, and validation.

Experience with cloud platforms (AWS/Azure/GCP) and data lake architectures.

Familiarity with Delta Lake, table optimization, and partitioning strategies.

Hands-on experience with Git, CI/CD, and Agile development.

Strong problem-solving and communication skills.

Preferred Skills:

Experience with Airflow/ADF/Luigi for orchestration.

Knowledge of data governance, lineage, and cataloging tools.

Understanding of MLOps or integration with ML pipelines.

Familiarity with Docker or container-based execution.

Jobs at Hyderabad

Oracle

Senior Member of Technical Staff (I…

Professional

Hyderabad, Telangana

Data Engineer – Databricks / PySpark / Spark Apply Now

Job Description

Jobs at Hyderabad

More Jobs at Accolite

Actively Recruiting Companies at Hyderabad, Telangana

Data Engineer – Databricks / PySpark / Spark