Rootshell Enterprise Technologies, Inc.

Compensation

Full Description

Lead Data Engineer Remote must be in USA Skill Set: Pyspark, Python, AWS, GIS, Palantir Foundry Need a SME in Python coding (Data structures and algorithms) Responsibilities • Develop and enhance data-processing, orchestration, monitoring, and more by leveraging popular open-source software, AWS, and GitLab automation. • Collaborate with product and technology teams to design and validate the capabilities of the data platform • Identify, design, and implement process improvements: automating manual processes, optimizing for usability, re-designing for greater scalability • Provide technical support and usage guidance to the users of our platform's services. • Drive the creation and refinement of metrics, monitoring, and alerting mechanisms to give us the visibility we need into our production services. Qualifications • Experience building and optimizing data pipelines in a distributed environment • Experience supporting and working with cross-functional teams • Proficiency working in Linux environment • 5+ years of advanced working knowledge of SQL, Python, and PySpark • Knowledge on Palantir • Experience using tools such as: Git/Bitbucket, Jenkins/CodeBuild, CodePipeline • Experience with platform monitoring and alerts tools