Accolite
Data Engineer – Databricks / PySpark / Spark
Job Description
Job Summary:
We are seeking an experienced Data Engineer with strong hands-on expertise in Databricks, PySpark, and Apache Spark to design and develop scalable data pipelines for analytics and operational workloads. The ideal candidate will work across data ingestion, transformation, optimization, and orchestration in cloud-based environments.
Key Responsibilities:
Design and build ETL/ELT pipelines using Databricks, PySpark, and Spark-based frameworks.
Develop reusable and high-performance data processing workflows for batch and streaming workloads.
Integrate data from multiple structured and unstructured sources into cloud data platforms.
Optimize Spark jobs for performance, reliability, and cost efficiency.
Work with Delta Lake or similar lakehouse architectures for data versioning and ACID compliance.
Collaborate with data architects, analysts, and application teams to deliver clean, reliable datasets.
Write efficient code in PySpark, SQL, and Python.
Implement CI/CD for data workflows and follow best practices in data governance and security.
Troubleshoot data pipeline issues and support production workloads.
Required Qualifications:
5-7 years of experience in data engineering and distributed data processing.
Strong hands-on experience in Databricks, PySpark, and Apache Spark.
Proficient in SQL for transformations, profiling, and validation.
Experience with cloud platforms (AWS/Azure/GCP) and data lake architectures.
Familiarity with Delta Lake, table optimization, and partitioning strategies.
Hands-on experience with Git, CI/CD, and Agile development.
Strong problem-solving and communication skills.
Preferred Skills:
Experience with Airflow/ADF/Luigi for orchestration.
Knowledge of data governance, lineage, and cataloging tools.
Understanding of MLOps or integration with ML pipelines.
Familiarity with Docker or container-based execution.
Jobs at Bangalore
Amazon
WHS Officer, WHS
Freshers/Experienced
Bangalore, Karnataka
Last Date: Dec. 9, 2025
Amazon
Senior Software Development Enginee…
Freshers/Experienced
Bangalore, Karnataka
Last Date: Dec. 16, 2025
KPMG
Assistant Manager - Finance Advisory
KI Professional
Bangalore, Karnataka
Last Date: Dec. 17, 2025
KPMG
Manager - CloudSec-Architecture des…
KI Professional
Bangalore, Karnataka
Last Date: Dec. 17, 2025
Razorpay
Associate Director, Engineering - D…
Freshers/Experienced
Bangalore, Karnataka
Last Date: Dec. 13, 2025
Amazon
Investigation Specialist [ENG]
Freshers/Experienced
Bangalore, Karnataka
Last Date: Dec. 16, 2025
More Jobs at Accolite
Accolite
Databricks Engineer
5 - 10 Years Exp.
Bangalore, Karnataka
Last Date: Dec. 20, 2025
Accolite
Mobile App Test Engineer
5 - 10 Years Exp.
Gurgaon, Haryana
Last Date: Dec. 13, 2025
Accolite
Front End(AEM) Developer
4 - 6 Years Exp.
Bangalore, Karnataka
Last Date: Dec. 20, 2025
Accolite
Senior Test Engineer
6 - 12 Years Exp.
Gurgaon, Haryana
Last Date: Dec. 13, 2025
Accolite
Murex Back Office / Settlement BA
5 - 8 Years Exp.
Bangalore, Karnataka
Last Date: Dec. 20, 2025
Accolite
Full Stack Developer – .NET & Angul…
7 - 12 Years Exp.
Bangalore, Karnataka
Last Date: Dec. 20, 2025
Accolite
Java + AWS Developer
10 - 15 Years Exp.
Bangalore, Karnataka
Last Date: Dec. 20, 2025
Accolite
Test Engineer - Data
5 - 10 Years Exp.
Gurgaon, Haryana
Last Date: Dec. 13, 2025
Accolite
Data Architect – Payments (AWS & St…
10 - 15 Years Exp.
Bangalore, Karnataka
Last Date: Dec. 20, 2025
Accolite
AI Engineer – Python / Agentic AI /…
5 - 12 Years Exp.
Bangalore, Karnataka
Last Date: Dec. 20, 2025
Accolite
Senior AI Engineer – Python / Agent…
9 - 12 Years Exp.
Bangalore, Karnataka
Last Date: Dec. 20, 2025