Design Applications — Databricks Generative AI Engineer practice questions

When should you fine-tune vs prompt + RAG?

1. Always fine-tune before attempting any prompting
2. Prompt and RAG first; fine-tune for style needs
3. Always fine-tune for any production deployment
4. Never fine-tune; prompting always suffices

Answer: B — Prompt and RAG first; fine-tune for style needs

B) Correct — start with prompting and retrieval, then fine-tune when output style, structure, or per-call cost at volume demands it. A) Skips the cheaper levers. C) Many production workloads never need it. D) Fine-tuning is genuinely the right answer for some format-critical cases.

A team is selecting a foundation model for a support assistant. Traffic is bursty, prompts average 3,000 tokens, and the assistant must reason over multi-step troubleshooting trees. Which selection criterion should carry the LEAST weight at design time?

1. The model's published context window relative to prompt plus retrieved context length
2. The model's parameter count as an absolute measure of capability
3. Whether the model is available through pay-per-token Foundation Model APIs
4. Measured quality on a task-representative evaluation set the team builds

Answer: B — The model's parameter count as an absolute measure of capability

B) Correct — parameter count is a poor standalone proxy for capability; models of similar size differ widely by training and instruction tuning, so it should not drive selection. A) Context window is a hard constraint: if prompt plus retrieval exceeds it, the design fails regardless of quality. C) Availability on pay-per-token matters directly because traffic is bursty and provisioned throughput would be idle much of the time. D) Task-representative evaluation is the strongest evidence available and should dominate the decision.

A GenAI proposal lists this success criterion: "users are satisfied with the assistant." Before development starts, the FIRST thing the engineer should do is:

1. Select the foundation model and provision a serving endpoint for prototyping
2. Stand up a feedback widget so satisfaction data accumulates during development
3. Ingest and chunk the full corpus first so that retrieval experiments can begin immediately
4. Define measurable targets and a labeled evaluation set that represents real questions

Answer: D — Define measurable targets and a labeled evaluation set that represents real questions

D) Correct — without measurable targets and a representative evaluation set there is no way to compare design options or know when the project is done. A) Choosing a model first commits to an implementation before the bar it must clear is defined. B) A feedback widget is valuable in production but collects nothing useful before there is a working assistant, and thumbs data alone is too sparse to steer design. C) Chunking before knowing what questions matter means retrieval experiments have nothing to be scored against.

A new GenAI workload's biggest design risk is unclear success criteria. The right first step is:

1. Train a custom model for the new workload
2. Select the largest model currently available
3. Define an eval set and target metrics first
4. Provision additional GPU capacity up front

Answer: C — Define an eval set and target metrics first

C) Correct — an evaluation set with explicit targets for groundedness, correctness, and latency is what every later design choice gets measured against. A) Training before knowing the target is premature. B) Model size chosen without metrics wastes budget. D) Buying infrastructure first inverts the order.

Which design choice MOST reduces hallucination risk in a RAG app?

1. Serve the largest available foundation model
2. Raise the sampling temperature on generation
3. Increase the max_tokens limit on responses
4. Ground answers and refuse without evidence

Answer: D — Ground answers and refuse without evidence

D) Correct — solid retrieval plus an instruction to cite context and decline when evidence is missing is the proven anti-hallucination pattern. A) Larger models still fabricate. B) Higher temperature increases invention. C) A longer budget gives more room to fabricate.

MoE (Mixture of Experts)?

Architecture where each token routes through a subset of expert sub-networks. Higher capacity at lower compute per token.

What is the main engineering tradeoff of adopting Agent Bricks over a hand-built Agent Framework agent?

Agent Bricks gives fast, auto-optimized, governed quality with far less code, but limited control over orchestration logic, custom tool calling and bespoke chain steps. Choose Agent Framework when you need arbitrary control flow or non-standard tools.

Name two reasons to prefer a fixed chain over an agent for a production workload.

Predictable latency and cost, because the number of model calls is known in advance • Easier evaluation, tracing and debugging, since the execution path does not vary between requests.

You must turn 200,000 scanned contracts into a governed Delta table of typed fields. Which Agent Bricks option fits?

Information Extraction. It is purpose-built to pull a defined schema of fields out of unstructured documents at scale and land them as structured output, rather than answering free-form user questions.