How AI Engineering Interviews Work
What interviewers score, the answer framework, behavioral stories and eight fundamentals questions with model answers.
Q1What is the difference between training and inference?show answer
In plain words: Training teaches the model by changing its inner settings. Inference is just using the finished model to answer. Most app builders only work on inference.
Full answer · Training updates a model's parameters using data and an optimization procedure; inference uses the trained, fixed model to produce an output for a new input. As an application engineer you mostly work at inference time — prompts, retrieval, tools, state, evaluation, security and serving — while training/fine-tuning is a separate, much costlier activity you only choose when prompting and retrieval can't meet the requirement.
Q2Why does tokenization matter for cost and context?show answer
In plain words: Models read tokens, not words, and you pay per token. The same message can cost more in code or other languages, and the limit is counted in tokens too.
Full answer · Models read tokens, not characters or words, and both billing and context limits are counted in tokens. Code, JSON and non-English text usually tokenize less efficiently, so the same information can cost far more than its character count suggests. It also affects chunking: split by tokens, not characters, and count tokens on your real prompts, tool schemas and tool results — in an agent loop they're re-sent each turn.
Q3What is an embedding and why is it useful?show answer
In plain words: An embedding turns text into numbers that capture its meaning. So you can find text that means the same thing, even if the words are different.
Full answer · An embedding maps content such as text to a vector so that semantically similar items sit near each other; similarity (often cosine) lets you search by meaning rather than keywords. It powers retrieval, clustering and deduplication. An embedding model is different from a generative model, and quality depends on the model, chunking and metadata filtering as much as on the vector database. Changing the embedding model means re-embedding the corpus, so version it.
Q4Why can a model be confident and wrong?show answer
In plain words: The model writes likely-sounding text, and nothing in it checks the facts. So it can sound sure and still be wrong. That is why you verify.
Full answer · It generates likely continuations, not verified facts, and nothing in the generation process checks truth; fluent text is produced whether or not it's correct. It may lack current or private information, misread an ambiguous request, or fill gaps plausibly. So production systems treat output as an untrusted proposal: ground it with retrieval and citations, constrain it with schemas, verify it with deterministic checks or tests, and escalate when uncertain.
Q5What factors influence model selection?show answer
In plain words: Test the models on your own tasks and compare quality, speed, cost and safety. Use small models for easy jobs and bigger ones for hard jobs.
Full answer · Quality on your actual tasks, tool-use and structured-output reliability, latency, cost per successful task, context length, privacy and residency requirements, availability and rate limits, and safety behavior. The method matters more than the list: build an evaluation set from your workload, compare candidates, route by capability (small model for classification/extraction, stronger for hard reasoning), pin versions, and test fallbacks. Don't pick from leaderboards alone.
Q6Is a bigger context window a replacement for retrieval?show answer
In plain words: No. Reading more text costs more and can hide the important part. Search finds the right pieces, so you use a small, relevant set.
Full answer · No. Long contexts cost more, add latency and can bury relevant facts among noise or conflicting text, so relevance degrades. Retrieval selects what matters, supports access control and freshness, and lets you cite sources. Long context helps when you genuinely need to reason over one large, coherent document; it isn't a substitute for a search system over a corpus.
Q7Explain transformers conceptually, without pretending attention is magic.show answer
In plain words: Each word looks at other words in the text to understand them. That is why longer text costs more, and why more text is not always better.
Full answer · A transformer represents each token as a vector and, in each layer, lets tokens exchange information through attention: each token computes weights over other positions and mixes their representations, with multiple heads learning different relationships. Feed-forward layers transform the result; residual connections and normalization keep deep stacks trainable. The practical consequences are that cost grows with context length, order and position handling matter, and more text isn't automatically more information.
Q8When is an LLM the wrong tool?show answer
In plain words: When there is one exact right answer you can compute, or when you need strict, checkable rules. Then use normal code, not a language model.
Full answer · When there is a single right answer you can compute (arithmetic, exact lookups, date logic), when strict determinism or auditability is required (eligibility rules, permissions), when a classical model or simple rules suffice for a prediction task, or when latency and cost budgets rule out a model call. A good answer names the smallest technique that fits and shows where an LLM might still help at the edges, such as extraction or explanation.