Trustworthy AI — NVIDIA NCA Gen AI LLMs practice questions

Indirect (cross-domain) prompt injection occurs when:

1. The user types the malicious instruction directly into the chat
2. The model is fine-tuned on deliberately poisoned training data
3. Malicious instructions hide in retrieved third-party content
4. The model process crashes from an out-of-memory condition

Answer: C — Malicious instructions hide in retrieved third-party content

C) Correct — the payload arrives through pages or documents the model later retrieves or summarizes, which is why it is called indirect. A) That is direct injection. B) That is data poisoning at training time. D) A crash is not an injection.

To reduce hallucination, the BEST mitigations are:

1. Raising the temperature to encourage more varied answers
2. Grounding with RAG citations, low temperature, filters
3. Always selecting the largest available model checkpoint
4. Accepting hallucinations as an inherent model limit

Answer: B — Grounding with RAG citations, low temperature, filters

A) Higher temperature means more randomness and more hallucination. B) Correct — combine grounding (RAG with citations), determinism (low temperature), structured outputs, guardrails and ongoing evaluation. C) Larger models still hallucinate. D) It is a critical issue to address, not accept.

A support assistant logs full conversation transcripts containing customer PII. The MOST appropriate control BEFORE storage is to:

1. Encrypt the log volume at rest and keep transcripts indefinitely
2. Restrict log access to the on-call engineering rotation
3. Detect and redact PII entities, retaining only what is needed
4. Hash the customer id field while storing the rest verbatim

Answer: C — Detect and redact PII entities, retaining only what is needed

C) Correct — minimization at the point of capture is the strongest control because unwritten data cannot leak. A) Encryption protects the medium but leaves the data present and retained forever. B) Access control alone still concentrates sensitive text in one store. D) Hashing one identifier leaves the PII in the free text untouched.

Which statement about LLM safety and capability tradeoffs is most accurate?

1. Stronger alignment always drives model helpfulness down to zero
2. Safety concerns can be ignored entirely once a model is large
3. Safety measures carry no measurable cost to a model's capability
4. Safety and helpfulness are co-designed; both refusal errors matter

Answer: D — Safety and helpfulness are co-designed; both refusal errors matter

D) Correct — over-refusal and under-refusal are both real failures, so current practice measures and balances them together. A) An unsupported extreme. B) Scale alone does not deliver safety. C) There are genuine, measurable tradeoffs.

A model card's PRIMARY purpose is to:

1. Document intended use, limitations and evaluation results
2. Store the serialized weights and optimizer state for reloading
3. Provide the container manifest used to deploy the model
4. Record per-request inference latency for capacity planning

Answer: A — Document intended use, limitations and evaluation results

A) Correct — model cards communicate scope, known limitations, evaluation and appropriate use to downstream adopters. B) Weights live in a checkpoint or registry artifact. C) Deployment manifests are infrastructure config. D) Latency telemetry belongs to monitoring dashboards.

The MOST effective way to reduce unsupported claims in a RAG assistant's answers is to:

1. Require sentence-level citations and reject answers lacking support
2. Increase max_new_tokens so the model can explain itself more fully
3. Raise top_p so the model considers a wider token distribution
4. Remove the system prompt so the model relies purely on retrieval

Answer: A — Require sentence-level citations and reject answers lacking support

A) Correct — enforcing attribution makes ungrounded statements detectable and blockable rather than merely discouraged. B) A longer answer gives more room for unsupported content. C) A wider nucleus increases, not decreases, drift from the evidence. D) Deleting the instruction removes the grounding directive itself.

Watermarking generated text aims to:

1. Embed statistical signals in output that detectors can spot
2. Reduce inference latency by shortening the generated sequence
3. Improve perplexity scores measured on held-out evaluation text
4. Encrypt the prompt so intermediaries cannot read user input

Answer: A — Embed statistical signals in output that detectors can spot

A) Correct — watermarking biases token selection so a downstream detector can flag text as machine-generated. B) It does not target latency. C) It does not improve modeling quality. D) Encryption protects confidentiality, a different goal.

Prompt injection attack defenses include:

1. Trusting user input and passing it straight into prompts
2. Structured prompts, input sanitization, and output filters
3. Disabling the LLM entirely for every user-facing feature
4. Restricting deployments to the smallest available models

Answer: B — Structured prompts, input sanitization, and output filters

A) That is the vulnerability, not a defense. B) Correct — defense in depth: clear delimiters, sanitized input, output policy filters, and role separation in the chat template. C) Defeats the purpose. D) Model size does not fix injection.

Which is the BEST defense-in-depth strategy against prompt injection in a tool-using agent?

1. Trust user input fully and execute every tool call it requests
2. Rely on a very large model whose scale is assumed to prevent attacks
3. Disable request logging so attackers cannot study system behavior
4. Layer input validation, role separation, and least-privilege tools

Answer: D — Layer input validation, role separation, and least-privilege tools

D) Correct — layered controls plus output filtering and policy guardrails give defense in depth, since no single check holds. A) Removes every control. B) Scale does not confer injection resistance. C) Logging is needed for detection and response.

Red-teaming an LLM application typically involves:

1. Running only automated unit tests against the application code
2. Adversarial probing for injections, jailbreaks, and data leaks
3. Recoloring and restyling the chat interface before it launches
4. Replacing the deployed model with a smaller, cheaper variant

Answer: B — Adversarial probing for injections, jailbreaks, and data leaks

B) Correct — manual and automated probing surfaces injections, jailbreaks, privacy leaks, and tool misuse before release. A) Unit tests do not explore adversarial behavior. C) Interface styling is cosmetic. D) Swapping models is a cost decision, not a security exercise.

Prompt injection?

Attack where user input overrides system instructions. Mitigate with input sanitization, structured prompts, output filters.

Where does bias enter an LLM pipeline, and name two fairness measurements you can apply to outputs.

Bias enters through skewed pretraining corpora, annotator judgments in fine-tuning and preference data, and the deployment context itself. Measurements include disparity in output quality or sentiment across demographic groups when only the group term in a prompt is swapped (counterfactual testing), and equalized error rates across groups on a labeled evaluation set.

What privacy risks are specific to LLM systems, and what controls address them?

Risks include memorization and regurgitation of PII from training data, sensitive data entering prompts and being logged or used for training, and leakage through retrieval indexes that ignore per-user permissions. Controls: PII detection and redaction during curation and at the prompt boundary, retention limits and opt-out on logs, and access control enforced at retrieval time rather than in the prompt.

What is NeMo Guardrails and what kinds of rails can it enforce?

It is an open-source toolkit that sits between the application and the LLM and enforces programmable policies on the conversation. Rail types include topical rails keeping dialogue in scope, input rails screening for jailbreaks and prompt injection, output rails filtering unsafe or non-compliant responses, retrieval rails governing what context may be used, and execution rails controlling tool calls.