via Gem
$Not specified
Design and conduct evaluations of AI outputs for legal accuracy and relevance, collaborating with engineering teams to improve product quality.
Requires a law degree, U.S. legal practice license, 3+ years of legal experience, and familiarity with AI in legal contexts.
About The Role As a Legal Engineer, you’ll be instrumental in measuring and continually improving the substantive accuracy and quality of the legal information we provide. We're delivering a product that legal teams love, and in this role, you’ll leverage your legal experience to guide our AI toward practical excellence. You’ll set the bar for how legal AI behaves—optimizing system prompts, designing evaluations, and ensuring every output aligns with real-world legal standards. At a fast-moving, product-market-fit stage startup, your contributions won’t just be meaningful; they’ll help define our trajectory. The Impact You Will Have Shape the accuracy, reliability, and overall quality of AI outputs that hundreds of legal teams trust in their daily work. Lead the development of cutting-edge legal evaluations that set new standards for assessing AI in practice. Drive product enhancements by translating substantive expertise into features that improve usability, confidence, and outcomes for users. Help build the leading legal AI platform for in-house teams. What you’ll do Design and run rigorous evaluations measuring the quality of our AI-generated outputs in order to ensure they meet the high standards expected by top-tier legal professionals. Review and validate product outputs for accuracy, relevance, comprehensiveness, helpfulness, etc. Collaborate closely with the engineering and GTM teams to identify performance gaps, diagnose their causes, and develop solutions. Translate nuanced, real-world legal expectations into concrete, testable prompts and evaluation criteria. Draft and iterate on system prompts for our various AI systems. Assess foundational model performance and suitability for different legal tasks. Build and annotate complex legal datasets and document collections. Serve as the legal voice of quality, advocating for precision, consistency, and clarity in AI outputs. Act as the bridge between technical teams and legal domain expertise, translating complex legal concepts into clear product requirements and vice versa. Provide domain-specific legal expertise that guides feature development and product roadmap decisions. Ideate new products based on your deep understanding of legal teams’ pain points. Participate in customer calls and feedback sessions. Proactively develop and continually learn new technical skills. Take on additional projects and tasks as needed in response to the evolving needs of a fast-growing startup. Required Experience Law degree. License to practice law in a U.S. jurisdiction. 3+ years of legal experience, ideally gained in an in-house environment in a product, regulatory, or litigation counsel role. Deep familiarity with U.S. legal research and regulatory law. Experience using generative AI for legal work. Experience conducting evaluations of generative AI outputs. Strong quantitative and analytical skills. Ability to design and interpret evaluation metrics and to abstract legal reasoning and processes into logical and repeatable steps. Exceptional cross-disciplinary communication and collaboration skills. Technologically curious with an aptitude for learning new tools, software, and skills Meticulous attention to detail. Ability to work cross-functionally with both GTM and engineering/product teams. Nice To Have You are already a GC AI user. You’re nerdy about legal tech, especially in areas like prompt engineering and quality assurance. You care about user experience, especially how legal professionals interact with and trust AI assistants. You’ve worked closely with software engineers before. In addition to U.S. expertise, you are familiar with the law of other countries.
This job posting was last updated on 12/12/2025