$120K - 160K a year
Design, develop, optimize, and maintain big data pipelines and cloud applications using Python/PySpark within a Hadoop ecosystem, ensuring performance tuning and adherence to Agile methodologies.
5+ years experience with Python/PySpark, Hadoop ecosystem tools, cloud application development, distributed computing, relational and NoSQL databases, multi-threading, shell scripting, CI/CD tools, microservices, container orchestration, Agile/SAFe practices, and strong communication skills.
Job Requirements - Qualifications: Software Engineer Big Data 5+ years in Python/PySpark 5+ years optimizing Python/PySpark jobs in a hadoop ecosystem 5+ years working with large data sets and pipelines using tools and libraries of Hadoop ecosystem such as Spark, HDFS, YARN, Hive and Oozie. 5+ years with designing and developing cloud applications: AWS, OCI or similar. 5+ years in distributed/cluster computing concepts. 5+ years with relational databases: MS SQL Server or similar 3+ years with NoSQL databases: HBASE (preferred) 3+ years in creating and consuming RESTful Web Services 5+ years in developing multi-threaded applications; Concurrency, Parallelism, Locking Strategies and Merging Datasets. 5+ years in Memory Management, Garbage Collection & Performance Tuning. Strong knowledge of shell scripting and file systems. Preferred: Knowledge of CI tools like Git, Maven, SBT, Jenkins, and Artifactory/Nexus Knowledge of building microservices and thorough understanding of service-oriented architecture Knowledge in container orchestration platforms and related technologies such as Docker, Kubernetes, OpenShift. Understanding of prevalent Software Development Lifecycle Methodologies with specific exposure or participation in Agile/Scrum techniques Strong knowledge and application of SAFe agile practices, preferred. Flexible work schedule. Experience with project management tools like JIRA. Strong analytical skills Excellent verbal, listening and written communication skills Ability to multitask and prioritize projects to meet scheduled dea
This job posting was last updated on 8/29/2025