Data Preparation and Retrieval Quality — Databricks Generative AI Engineer practice questions

A team reports recall at k=5 of 0.62 and precision at k=5 of 0.55. Which interpretation is MOST defensible?

1. Retrieval is fine; the generator is the problem and should be replaced
2. About a third of questions never see the right passage
3. The evaluation set is too small to be meaningful at these values
4. Precision below recall proves that top-k is set too low

Answer: B — About a third of questions never see the right passage

B) Correct — recall of 0.62 means the supporting passage is absent for about 38 percent of questions, and no generator can answer those correctly, so retrieval is the binding constraint. A) With recall that low the generator is being blamed for missing evidence it never received. C) Set size is a fair general concern but nothing in the numbers indicates it, and the recall gap is large enough to act on. D) The relationship between precision and recall values does not by itself imply a top-k setting; raising k usually raises recall and lowers precision.

A team evaluates retrieval and finds recall at k=10 is 0.94 but the generator still answers poorly. Adding which stage is MOST likely to close the gap?

1. A cross-encoder re-ranker over the top candidates, passing the best few onward
2. A larger embedding model to raise recall further above 0.94 at k=10
3. A higher top-k so that more candidate chunks reach the generator
4. A summarization step that compresses all ten chunks into one paragraph

Answer: A — A cross-encoder re-ranker over the top candidates, passing the best few onward

A) Correct — high recall with poor answers is the signature of a precision problem: the right chunk is present but buried, and a re-ranker reorders candidates so the strongest evidence leads. B) Recall is already near ceiling, so spending on a bigger embedder targets the metric that is not the bottleneck. C) Raising top-k adds more distracting context and typically worsens an ordering problem. D) Summarizing all ten chunks propagates the noise into the summary and can drop the specific detail the answer needs.

For chunking, what does overlap solve?

1. It reduces the total number of tokens embedded
2. It reduces the storage the index consumes
3. It speeds up the embedding computation step
4. It preserves context across chunk boundaries

Answer: D — It preserves context across chunk boundaries

D) Correct — repeating tokens at the seam keeps an entity mentioned near a split retrievable from either neighboring chunk. A) Overlap increases the token count. B) It increases stored data. C) Embedding more text is marginally slower.

What is the impact of duplicate documents in the corpus on RAG quality?

1. Recall rises, since the same fact is reachable through more chunks
2. Embedding cost falls because identical text is encoded only one time
3. No effect at all, since the index deduplicates identical vectors itself
4. Near-identical chunks crowd out top-K, biasing context and raising cost

Answer: D — Near-identical chunks crowd out top-K, biasing context and raising cost

D) Correct - duplicates occupy multiple top-K slots, so the context repeats itself and genuinely different evidence is pushed out; deduplicate at ingestion. A) More copies of one fact inflate its apparent support rather than improving coverage. B) Duplicates are embedded repeatedly, so cost goes up. C) The index stores what you write; it does not silently collapse duplicates.

Optimal RAG chunk size range is typically:

1. 10-50 tokens, one short phrase per chunk
2. 5,000+ tokens, several full pages in each chunk
3. 200-1,000 tokens with a small overlap window
4. Exactly one sentence per chunk, no overlap

Answer: C — 200-1,000 tokens with a small overlap window

C) Correct — this range balances retrieval precision against enough surrounding context, and modest overlap preserves continuity at boundaries. A) Too small to carry meaning. B) Too large, so retrieval returns mostly irrelevant text. D) Single sentences strip the context the model needs.

What is the purpose of chunk overlap, and what is the cost of setting it too high?

Overlap repeats tokens at chunk boundaries so an idea split across a boundary still appears whole in at least one chunk. Too much overlap inflates index size, embedding cost and duplicate retrieval hits, which crowds out diverse context.

Common RAG failure modes?

Poor retrieval (wrong docs), too few/many chunks, conflicting sources, model ignoring context, hallucination, latency overruns.

Retrieved chunks are individually relevant but the model answers without needed surrounding context. What chunking technique addresses this?

Parent-document or context-enrichment retrieval: embed small precise child chunks for matching, but return the larger parent section to the LLM. This keeps retrieval precision while restoring the context the answer needs.

Why chunk documents?

Foundation models have limited context windows; small focused chunks improve retrieval relevance + reduce hallucination.