Jobgether

Compensation

Full Description

This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Data Engineer with IRS MBI Clearance in the United States. We are seeking a Data Engineer to design, build, and maintain scalable, high-performance data processing pipelines in a cloud-based environment. This role requires hands-on expertise in distributed computing frameworks, real-time data streaming, and large-scale data modeling. You will collaborate with data scientists and analysts to deliver reliable, high-quality datasets while optimizing performance and ensuring observability across pipelines. The position demands a proactive, detail-oriented professional with strong problem-solving skills and the ability to work independently or within a team. Applicants must hold an active IRS MBI Clearance with IRS GFE and be able to manage sensitive and secure data environments. This role offers the opportunity to contribute to mission-critical data solutions and advance your career in cutting-edge data engineering. Accountabilities Design, implement, and maintain scalable data pipelines using modern big data technologies such as Apache Spark. Optimize existing workflows for performance, reliability, and data quality. Support real-time data processing and streaming architectures, integrating with sources like Kafka or CDC feeds. Collaborate with data scientists, analysts, and cross-functional teams to meet data needs. Implement monitoring, observability, and data quality checks for all pipelines. Maintain comprehensive documentation for data architectures, workflows, and best practices. Ensure compliance with IRS security protocols and handle sensitive data with confidentiality. Bachelor’s degree in Computer Science, Engineering, or related technical field. 5+ years of professional software development experience. Strong programming skills in Python, Java, or Scala. Hands-on experience with Apache Spark, including Spark SQL and Spark Streaming. Proficiency in cloud-based Spark platforms (e.g., Databricks, AWS EMR, AWS Glue). Experience with Hive, Unity Catalog, Glue Catalog, and distributed computing concepts. Familiarity with SQL and NoSQL databases, data modeling, and ETL/ELT best practices. Knowledge of data quality frameworks, CI/CD pipelines, and observability tools. Strong problem-solving skills, attention to detail, and excellent communication. Must hold an active IRS MBI Clearance with IRS GFE. Preferred Qualifications: Experience with Delta Lake or similar data lakehouse technologies. Knowledge of AWS, Azure, or GCP cloud platforms. Experience with real-time data streaming architectures. Contributions to open-source projects. Competitive salary and incentive packages. Full-time remote work flexibility. Professional development and career growth opportunities. Collaborative and innovative work environment. Health, dental, and vision benefits, as well as retirement plans. Opportunity to work on mission-critical data engineering projects. Jobgether is a Talent Matching Platform that partners with companies worldwide to efficiently connect top talent with the right opportunities through AI-driven job matching. When you apply, your profile goes through our AI-powered screening process designed to identify top talent efficiently and fairly. 🔍 Our AI evaluates your CV and LinkedIn profile thoroughly, analyzing your skills, experience, and achievements. 📊 It compares your profile to the job’s core requirements and past success factors to determine your match score. 🎯 Based on this analysis, we automatically shortlist the 3 candidates with the highest match to the role. 🧠 When necessary, our human team may perform an additional manual review to ensure no strong profile is missed. The process is transparent, skills-based, and free of bias — focusing solely on your fit for the role. Once the shortlist is completed, it is shared directly with the hiring company, who will manage the final selection steps, including interviews and additional assessments. Thank you for your interest! #LI-CL1