Staff Software Engineer (AI Evals)
G2i · Engineering · Lead · 24d ago
Anyone, anywhere in the world can apply.
Direct from the employer's own careers page
Who can apply
Read straight from the listing, so you know before you spend time on it.
Anyone, anywhere in the world can apply.
- Async-first
- The listing doesn't promise async-first working.
- Contractor / B2B
- The role is offered as an hourly contractor engagement at $100–$200/hour.
- Global stipend
- No home office or equipment budget mentioned.
- Annual retreat
- No company-paid get-togethers mentioned.
Read from the listing automatically. Always confirm the details with the company before you accept an offer.
About the role
In this contract role, you will assess how modern AI coding agents behave across real-world developer scenarios, judging whether outputs reflect strong engineering judgment, sound reasoning, and genuine utility. Rather than writing production code or performing low-level syntax reviews, you will apply seasoned engineering taste to distinguish subtle quality levels, evaluate developer guidance and explanations, and provide opinionated feedback on what works and what misleads.
What you'll do
- Evaluate AI-generated coding interactions end-to-end for correctness, utility, and reasoning quality.
- Assess whether agent explanations, preambles, and code align with how top-tier engineers think.
- Distinguish between varying quality levels in model responses and score them using rigorous standards.
- Deliver direct, highly opinionated feedback on agent behaviors, strengths, flaws, and failure modes.
- Help define the standard of excellence for interacting with AI-assisted developer tools like Cursor and Claude Code.
What you bring
- Staff- or Principal-level software engineering experience or equivalent seniority.
- Strong background in at least one of Go, Python, Rust, or TypeScript/JavaScript.
- Hands-on experience using OpenAI Codex, Claude Code, Cursor, and AI-assisted workflows.
- Ability to evaluate code and architectural reasoning without needing to execute every line.
- High engineering standards and comfort providing subjective yet rigorous feedback.
About G2i
G2i is a hiring platform and developer network that matches pre-vetted senior software engineers with tech companies worldwide.
View G2i profile & open roles →