AI Response Evaluator

Job description

<p>iMerit is looking for detail oriented analysts to evaluate and rank AI generated responses to image based prompts. You will judge answers on accuracy, relevance, clarity, conciseness, safety, localization, and how well they follow the user's instructions, then explain your reasoning in writing.<br>Much of the job comes down to this: look at the image, look at what the model said about it, and decide whether the two actually match.</p> <p><strong>What you will do</strong></p> <ul class="list-disc list-outside leading-3 -mt-2" style=""> <li class="leading-normal -mb-2" style=""> <p>Interpret conversational context and identify what the user really wanted</p> </li> <li class="leading-normal -mb-2" style=""> <p>Rate and rank responses against defined quality criteria</p> </li> <li class="leading-normal -mb-2" style=""> <p>Compare multiple answers and explain in writing why one wins</p> </li> <li class="leading-normal -mb-2" style=""> <p>Verify factual claims using approved research sources</p> </li> <li class="leading-normal -mb-2" style=""> <p>Flag tasks that cannot be reliably assessed rather than guessing</p> </li> </ul> <p><strong>What you bring:</strong></p> <ul class="list-disc list-outside leading-3 -mt-2" style=""> <li class="leading-normal -mb-2" style=""> <p>Strong critical thinking and sound judgment in ambiguous cases</p> </li> <li class="leading-normal -mb-2" style=""> <p>Solid research skills and attention to detail</p> </li> <li class="leading-normal -mb-2" style=""> <p>Excellent reading comprehension</p> </li> <li class="leading-normal -mb-2" style=""> <p>Self direction and the discipline to hit deadlines without supervision</p> </li> </ul> <p><strong>Good to know</strong></p> <ul class="list-disc list-outside leading-3 -mt-2" style=""> <li class="leading-normal -mb-2" style=""> <p>Independent contractor engagement for the length of the project. </p> </li> <li class="leading-normal -mb-2" style=""> <p>Fully remote and flexible.</p> </li> <li class="leading-normal -mb-2" style=""> <p>You choose your hours as long as volume and deadlines are met.</p> </li> <li class="leading-normal -mb-2" style=""> <p>Task volume varies with project demand. </p> </li> </ul> <img src="https://remotive.com/job/track/2091126/blank.gif?source=public_api" alt=""/>

Job locations

Compensation

Salary for this role is not disclosed by the employer. We don’t have enough benchmark data from similar roles to produce a reliable estimate. The actual compensation will be discussed during the interview process.