AI Engineer Interview Questions for 2027

Interview loops for AI engineers are still settling down. Some companies run a classic coding round plus one "AI question"; others have rebuilt the loop around a take-home that ships a small LLM feature. The formats differ, but the questions cluster around the same handful of skills, because those are the skills that decide whether an AI feature works in production.

Below are the questions I would expect, organised by round, with what a strong answer covers and a practice prompt for each. If you are earlier in the journey, start with how to become an AI engineer; if you are hiring, my list of questions to ask before hiring an AI developer is the client-side version of this post.

What does an AI engineer interview loop look like?

Most loops cover five areas, whatever the company calls the rounds: LLM fundamentals, retrieval, agents and tools, evaluation and debugging, and system design, usually wrapped in a recruiter screen at the start and a behavioural conversation at the end. A take-home or live build often replaces one of the middle rounds.

The later rounds carry the most weight, because they are the hardest to fake. Anyone can memorise what an embedding is; fewer people can explain why a retrieval pipeline that scored well last month is now returning stale answers.

LLM fundamentals: what will they ask?

The fundamentals round checks that you understand the component you are building on, well enough to predict how it fails. Expect questions like these:

• "What happens when a conversation exceeds the context window?" A strong answer covers truncation strategies, summarising older turns, retrieval instead of stuffing, and the cost of long contexts, not just the definition of a token.
• "How do you get reliable JSON out of a model?" Mention provider-native structured outputs, schema validation in code, and handling the cases the schema cannot cover. Anthropic's structured outputs documentation is a good reference: it describes constrained decoding against a JSON schema, and also lists caveats such as refusals and responses cut off at the token limit, which "may be incomplete".
• "When would you fine-tune instead of prompting or retrieving?" Good answers start with the cheapest option and explain what evidence would justify moving up. My post on RAG vs fine-tuning covers the trade-offs.
• "Why does the same prompt give different answers?" Talk about sampling, model updates, and why you pin model versions and test against a fixed set.

Practice prompt: take one feature you have built and write down every way the model could produce a wrong output. Then write the code-level check that would catch each one.

Retrieval and RAG: what will they ask?

Retrieval questions test whether you can ground a model in the right data and know when the grounding is the problem. Typical questions:

• "Walk me through a RAG pipeline end to end." Cover ingestion, chunking, embeddings, indexing, retrieval, re-ranking, prompt assembly and citation. Then say where each step tends to break.
• "Users say answers are wrong. How do you tell if it's retrieval or generation?" The strong answer separates the two: check whether the right passage was retrieved before blaming the model, and measure retrieval on its own.
• "How do you enforce permissions in a RAG system?" Filter documents by the user's access before they reach the model. Retrieval that ignores permissions turns the assistant into a data leak.
• "When is keyword search better than vector search?" Exact identifiers, product codes and names often favour keyword or hybrid search. Saying "it depends, and here is how I would test it" is the right instinct.

Practice prompt: build a small RAG app over 50 documents, then write 20 questions with known answers and measure how often the right document comes back in the top five.

Agents and tool use: what will they ask?

Agent questions test judgement more than framework knowledge: when an agent is the right tool, and how to keep one safe. Anthropic's essay on building effective agents draws the line clearly. Workflows are "systems where LLMs and tools are orchestrated through predefined code paths"; agents are "systems where LLMs dynamically direct their own processes and tool usage". It recommends "finding the simplest solution possible, and only increasing complexity when needed."

Expect:

• "Would you build this as an agent or a workflow?" Interviewers want to hear you choose the workflow when the steps are known, and justify the agent only when they are not.
• "How do you stop an agent doing something harmful?" Least-privilege tools, input validation, human approval for high-impact actions, step limits and logging. OWASP's guidance on prompt injection lists "require human approval for high-risk actions" and least-privilege access among its mitigations, and notes that "it is unclear if there are fool-proof methods of prevention".
• "An agent loops forever. What do you do?" Step and budget limits, a clear stop condition, and traces to see where it went round in circles.
• "How would you design the tools for this agent?" Small, well-named tools with typed inputs beat one giant "do anything" tool.

Practice prompt: implement one task twice, once as a fixed chain of prompts and once as an agent with tools. Compare reliability, cost and debuggability, and be ready to explain which you would ship.

Evals and debugging: what will they ask?

The evaluation round is where many candidates struggle, and where strong ones stand out. Hamel Husain puts it bluntly in his post on AI product evals: "unsuccessful products almost always share a common root cause: a failure to create robust evaluation systems." Interviewers who have read that will probe for it.

Expect:

• "How do you know your feature works?" Describe a test set of real examples, assertion-style checks that run on every change, human or model-graded review for subjective quality, and production metrics.
• "You changed the prompt and one metric improved but another dropped. What next?" Look at the actual examples that changed, not just the averages.
• "How do you use an LLM as a judge without fooling yourself?" Calibrate the judge against human labels and check agreement before trusting it.
• "What do you log for each model call?" Inputs, outputs, model version, tokens, latency, cost and the trace of tool calls, with personal data handled properly.

My post on AI agent evaluation goes deeper on building the test set.

Practice prompt: take 30 outputs from a feature, label them good or bad by hand, write down why, and turn the reasons into automated checks.

System design: what will they ask?

The system design round asks you to put everything together under real constraints: cost, latency, security and failure. A typical prompt is "design a support assistant for a company with 10,000 help articles" or "design an agent that triages incoming invoices".

A strong answer usually covers:

• Requirements and success metrics before architecture.
• The simplest design that could work, then what you would add and why.
• Data flow and permissions: what the model sees and who is allowed to see it.
• Cost per request, with levers such as model routing, prompt caching and batching. Anthropic's pricing page, for example, prices a cache hit at 10% of the standard input price, and lower still on some newer models, which changes the design of prompts with long, stable prefixes.
• Failure modes and fallbacks: provider outages, bad outputs, prompt injection, cost spikes.
• Evaluation and monitoring from day one.

Practice prompt: design one system in 40 minutes out loud, then list three things you would cut to ship it in two weeks.

Questions, signals and red flags by round

Work with me

If you are hiring and want a second opinion on an AI engineering loop, or need the feature built while you hire, I can help. See AI agent development and AI integration, browse my work, and book a scoping call.