Careerflow.ai

Compensation

Full Description

Careerflow Human Data Labs partners with AI companies to bring real-world professional expertise into their products. About the role: We are looking for a Strategic Project Lead with direct experience operating software-engineering data programs for frontier AI models. You will own production across coding data, RLHF or preference data, SWE benchmarks, model evaluations, agentic coding, code generation, and code repair. This includes contributor calibration, throughput, quality, rework, cost, timelines, and delivery recovery. The role requires strong technical judgment. You should be able to read code and tests, inspect final artifacts, identify invalid work, and make release or rejection decisions with minimal training. WHAT DOES DAY-TO-DAY LOOK LIKE? * Operate active software-engineering data and evaluation programs end to end. * Manage contributor sourcing, calibration, onboarding, reviewer capacity, and performance. * Track throughput, quality, rework, defects, cost, and delivery timelines. * Review code, tests, task metadata, and final artifacts for technical correctness. * Diagnose quality failures and improve task design, rubrics, instructions, and review processes. * Lead recovery plans when programs fall behind on quality, throughput, cost, or timelines. * Coordinate distributed software engineers, reviewers, and team leads. * Communicate progress, risks, and corrective actions to customer research and program teams. * Use Python, SQL, or similar tools for quality analysis, defect tracking, and operational reporting. REQUIREMENTS * Direct experience operating production software-engineering data programs for frontier AI models, preferably involving coding training, coding RLHF, SWE benchmarks, model evaluations, agentic coding, code generation, or code repair. * Proven ownership of the full production system, including contributor sourcing and calibration, task throughput, quality, rework, cost management, and delivery recovery. * Strong software-engineering judgment, with the ability to read code and tests, inspect final artifacts, identify invalid technical work, and make release or rejection decisions. * Experience managing distributed software-engineering contributors, reviewers, and team leads while maintaining consistent quality and delivery performance. * Ability to take over an active program quickly, diagnose operational problems independently, and communicate clear decisions and recovery plans to demanding customers. FREELANCING * Work remotely with leading global technology companies and AI labs. * Flexible engagement options based on project requirements and availability. * Opportunity to work on high-impact AI training, evaluation, and data-generation programs. * Collaborate with experienced technical, research, operations, and delivery teams. * Build expertise at the intersection of software engineering, AI, and large-scale operations. OFFER DETAILS: * Commitments Required: 8 hours per day with at least 4 hour overlap with PST.