Comrise

Compensation

Full Description

This is 10 week part time job with 5 hours per day and 2 days a week. Job description Company is partnering with an AI innovation initiative seeking Professional Software Engineers to support the evaluation and improvement of software engineering-focused AI tasks. In this role, you ll use your engineering and systems-level reasoning to assess model competency in real-world technical domains, debugging, architecture decisions, and system behavior analysis. What You ll Do • Create challenging SWE-related prompts based on visuals, datasets, or model outputs. • Evaluate AI-generated responses using provided rubrics. • Develop or improve rubrics to ensure they reflect accurate reasoning for complex tasks. • Review and stress-test model outputs for accuracy and logical consistency within your domain. • Provide clear, structured feedback based on your subject-matter knowledge. Ideal candidate profile Experience using an LLM is ideal. They want someone who has been working in their actual field most recently, rather than AI training, but experience working with LLMs is a plus. Daily tasks Design problems and evaluate AI prompts based on the domain expertise. Provide thorough feedback. Required skills Need a bachelor s degree or higher, experience in related field.