Interview questions · 24 with model answers

AI evaluation engineer

Proves whether an AI system is good enough, and keeps checking after launch.

How to use this page. Read the question. Answer it aloud first, in your own words, for about a minute. Then open the model answer and compare. Do not memorize the model answers. Interviewers can tell. Use them to see the shape of a good answer.

Your marks are saved in this browser. Questions tagged India cover language, rules, users and customers in the Indian market. Laws and rules change, so check the current text before you quote one.

Start a timed mock interview Roadmap Hands-on lab Hiring in India: a short guide

0 of 24 marked as known · 0 to review again

Level

1ConceptsFresher
Why does AI need its own kind of testing?
2ConceptsFresher
What is a rubric, and what makes a good one?
3ConceptsMid
How do you build a good test set?
4ConceptsMid
Explain precision and recall, and when you would favor each.
5ConceptsMid
What are the weaknesses of using a language model as a judge?
6ConceptsSenior
What is groundedness, and how do you measure it?
7ScenarioMid
Your judge scores are high, but users complain. What do you do?
8ScenarioMid
A new prompt raises the average score by two points. Do you ship it?
9ScenarioMid
How do you evaluate an agent that sends emails?
10ScenarioSenior
Retrieval looks fine, but answers are wrong. How do you find the cause?
11Hands-onMid
Design an evaluation plan for a UPI payment support chatbot.
12Hands-onSenior
How big should a test set be?
13BehavioralMid
Tell me about a time your evaluation caught a problem before launch.
14BehavioralSenior
How do you handle a product manager who wants to ship despite a failing gate?
15IndiaMid
How do you evaluate quality across Indian languages?
16IndiaSenior
You need large-scale human review. How would you organize it in India?
17HR roundFresher
Tell me about yourself.
18HR roundMid
Why do you want to leave your current job?
19HR roundMid
What is your notice period, and can it be shortened?
20HR roundMid
What are your salary expectations?
21HR roundFresher
What are your strengths and your weaknesses?
22HR roundMid
Are you comfortable with work from office, shifts or relocation?
23HR roundFresher
Where do you see yourself in five years?
24HR roundMid
Do you have any questions for us?

AI evaluation engineer

Why does AI need its own kind of testing?

What is a rubric, and what makes a good one?

How do you build a good test set?

Explain precision and recall, and when you would favor each.

What are the weaknesses of using a language model as a judge?

What is groundedness, and how do you measure it?

Your judge scores are high, but users complain. What do you do?

A new prompt raises the average score by two points. Do you ship it?

How do you evaluate an agent that sends emails?

Retrieval looks fine, but answers are wrong. How do you find the cause?

Design an evaluation plan for a UPI payment support chatbot.

How big should a test set be?

Tell me about a time your evaluation caught a problem before launch.

How do you handle a product manager who wants to ship despite a failing gate?

How do you evaluate quality across Indian languages?

You need large-scale human review. How would you organize it in India?

Tell me about yourself.

Why do you want to leave your current job?

What is your notice period, and can it be shortened?

What are your salary expectations?

What are your strengths and your weaknesses?

Are you comfortable with work from office, shifts or relocation?

Where do you see yourself in five years?

Do you have any questions for us?