We are looking for a skilled PySpark Data Engineer / Senior Data Engineer with strong expertise in Apache Spark, PySpark, Python, SQL, and Big Data technologies. Experience: 6+ Years Location: Hyderabad Employment Type: Full-Time Key Responsibilities: Design, develop, and maintain scalable Apache Spark pipelines in production environments Build reliable and reusable data ingestion frameworks integrating multiple source systems Develop and optimize ETL/ELT pipelines using PySpark, Python, SQL, and Shell scripting Process and transform large-scale datasets (500GB+) efficiently Perform Spark performance tuning including partitioning, caching, shuffle optimization, and resource optimization Work with HDFS, Oracle SQL, and different file formats (Parquet, Avro, ORC, CSV) Implement data cleansing, validation, transformation, and curated data layer creation Troubleshoot production issues and improve pipeline reliability and scalability Required Skills: Strong hands-on experience with PySpark, Python & SQL Deep understanding of Apache Spark architecture (Driver, Executors, DAG, Shuffles, Partitions) Experience designing and optimizing Spark-based ETL/ELT pipelines Strong knowledge of Big Data ecosystems (Spark, Hadoop, HDFS) Experience with BigQuery is preferred Understanding of data quality, governance, observability, and performance tuning Good debugging, problem-solving, and Agile delivery skills 🎓 Qualification: Bachelor’s / Master’s degree with 6+ years of Data Engineering experience and strong PySpark expertise.