Operational Efficiency and Cost Management — AWS Certified Generative AI Developer AIP-C01 practice questions

An application occasionally receives ThrottlingException errors from Bedrock during short traffic bursts that resolve within a few seconds. The team wants to handle this gracefully without manual intervention or overwhelming the service further. What is the BEST client-side pattern?

1. Implement retries with exponential backoff and jitter, capped at a small number of attempts before surfacing an error.
2. Immediately retry the failed request in a tight loop until it succeeds.
3. Increase the application's request timeout so throttled requests are given more time to complete.
4. Switch all traffic to batch inference so throttling no longer applies.

Answer: A — Implement retries with exponential backoff and jitter, capped at a small number of attempts before surfacing an error.

Right - because exponential backoff with jitter spaces out retries so the client gives the service room to recover from a burst rather than adding to the load, and capping attempts prevents the retry logic from hanging indefinitely. Retrying in a tight loop is a common mistake practitioners fall into - it increases request rate exactly when the service is already constrained, which can prolong or worsen the throttling. Extending the timeout does not help because a throttled request is rejected immediately rather than left pending, so waiting longer changes nothing about the outcome. Moving all traffic to batch inference sidesteps real-time throttling but abandons the latency-sensitive, real-time nature of the application, and batch inference has its own separate quotas that can also be exceeded.

A global application serving users in North America and Europe currently calls a Bedrock model hosted only in a single US region. European users report noticeably higher latency, and the team also wants resilience if that region has an outage. What is the MOST appropriate solution?

1. Use a Bedrock cross-region inference profile, or deploy to a nearby region, so requests can route to the closest or most available region.
2. Enable prompt caching so repeated requests from European users are served faster.
3. Increase the Provisioned Throughput model units in the US region to handle more concurrent requests.
4. Switch European traffic to on-demand inference instead of Provisioned Throughput.

Answer: A — Use a Bedrock cross-region inference profile, or deploy to a nearby region, so requests can route to the closest or most available region.

Right - because the described problem is geographic latency and single-region availability risk, and cross-region inference (or deploying closer to users) directly shortens network distance and adds a failover path if one region is unavailable. Prompt caching reduces token cost and reprocessing time for repeated content, a true benefit, but it does not shrink the transatlantic network latency that is the actual complaint here. Adding more Provisioned Throughput capacity in the same US region increases concurrency headroom but does nothing to reduce the physical distance data must travel or provide a backup region. Choosing on-demand over Provisioned Throughput is purely a pricing-model decision and has no bearing on where the model is hosted or how far requests must travel.

A team wants to gradually shift traffic from an existing SageMaker-hosted model version to a newly trained version, monitoring the new version's performance before it takes full traffic. Which SageMaker capability is BEST suited to this?

1. Enable auto-scaling on the existing endpoint so it can absorb load from both model versions simultaneously.
2. Use Provisioned Throughput to reserve capacity for the new model version before rollout.
3. Deploy both versions as production variants on the same endpoint and use variant weights to gradually shift traffic, in a canary or blue-green style rollout.
4. Delete the old endpoint and immediately deploy the new model to a freshly created endpoint.

Answer: C — Deploy both versions as production variants on the same endpoint and use variant weights to gradually shift traffic, in a canary or blue-green style rollout.

Right - because production variants on a single endpoint, with adjustable weights, are the SageMaker mechanism purpose-built for gradually shifting live traffic between model versions while both remain available for comparison. Deleting the old endpoint and cutting over immediately is an abrupt, all-at-once rollout, the direct opposite of the gradual, monitored transition the team wants. Auto-scaling adjusts the instance count behind a single variant's load; it does not split or route traffic between two distinct model versions, so it does not achieve a gradual shift. Provisioned Throughput is a Bedrock capacity and pricing concept for reserving inference throughput, not a SageMaker feature for routing traffic between endpoint variants.

An application sends the same 6,000-token system prompt and reference document with every user request to Amazon Bedrock, followed by a short, unique user question. Reprocessing the long static portion drives significant cost and latency. Which feature is the MOST direct way to reduce this cost?

1. Reduce the max_tokens parameter to limit how many output tokens the model can generate.
2. Switch to batch inference so the repeated context is processed as a single asynchronous job.
3. Enable prompt caching so the static system prompt and reference document are cached and billed at a reduced rate on subsequent calls.
4. Move the static content into the model's fine-tuning dataset so it no longer needs to be sent at inference time.

Answer: C — Enable prompt caching so the static system prompt and reference document are cached and billed at a reduced rate on subsequent calls.

Right - because prompt caching targets exactly this pattern: a large, unchanging prefix reused across many calls, which is billed at a reduced rate once cached instead of full price every time. Fine-tuning is a plausible-sounding but incomplete fix - it does not reliably reproduce exact reference text at inference time and is a far heavier undertaking than caching. Lowering max_tokens only caps output token spend, but the stem's cost driver is the repeated long input, not generation length. Batch inference suits asynchronous, non-latency-sensitive workloads; this is a live, per-request interaction, so batch does not solve the repeated-context problem here.

A platform team supports five product teams that all invoke the same shared Bedrock models. Finance wants to attribute GenAI spend to each product team for chargeback. What is the BEST way to enable this?

1. Use CloudWatch Logs Insights to query invocation logs and manually total tokens per team each month.
2. Enable Bedrock model invocation logging to Amazon S3 for long-term audit retention.
3. Apply consistent cost allocation tags, such as team, project, and cost-center, to the invoking resources and requests, and activate those tags in AWS Cost Explorer or Billing.
4. Create a separate AWS account for each product team so costs are isolated by account boundary.

Answer: C — Apply consistent cost allocation tags, such as team, project, and cost-center, to the invoking resources and requests, and activate those tags in AWS Cost Explorer or Billing.

Right - because cost allocation tags are the AWS-native mechanism for attributing shared-resource spend to the teams generating it, and activating them surfaces per-team costs directly in Cost Explorer or Billing reports. A per-team account achieves isolation too, but it is a much heavier organizational restructuring than the question calls for when tagging within the existing shared setup solves the same problem. Manually tallying tokens from logs is possible but labor-intensive and error-prone, and it is not the built-in AWS cost allocation mechanism. Invocation logging to S3 is a true and useful capability for audit and compliance, but logging alone does not attribute dollar cost to a team without an additional tagging or accounting layer.

A team wants a SageMaker real-time endpoint to automatically add instances as traffic grows and remove them as traffic falls, while keeping per-instance load near a target level. Which auto-scaling configuration is the MOST appropriate starting point?

1. A step scaling policy based on CPUUtilization, since CPU is always the limiting resource for LLM inference.
2. Manually increasing the instance count whenever the team observes rising latency in CloudWatch dashboards.
3. A target-tracking scaling policy on the SageMakerVariantInvocationsPerInstance metric, with a target value established through load testing.
4. A scheduled scaling policy that adds instances at fixed times of day based on historical traffic patterns.

Answer: C — A target-tracking scaling policy on the SageMakerVariantInvocationsPerInstance metric, with a target value established through load testing.

Right - because target tracking on InvocationsPerInstance directly maintains a chosen per-instance load level and reacts to actual traffic in either direction, matching the stated requirement to add and remove instances automatically as demand changes. Scheduled scaling is a legitimate technique for known cyclical patterns, but it does not adapt to unplanned traffic swings the way the scenario requires. Basing scaling on CPUUtilization assumes CPU is always the bottleneck, but LLM inference workloads are frequently limited by GPU or accelerator capacity rather than CPU, making this metric an unreliable scaling trigger. Manually adjusting instance count after observing latency is reactive and slow by definition, which defeats the purpose of automatic scaling.

A retrieval-augmented generation pipeline routinely stuffs 15 retrieved document chunks into the prompt, but manual review shows only 3-4 are ever relevant to the final answer. Input token costs are the largest line item in the GenAI budget. What should the team do FIRST?

1. Tune the retriever to return fewer, higher-relevance chunks, for example via re-ranking or a stricter similarity threshold, before sending context to the model.
2. Ask the model to summarize its own answer more concisely to reduce output tokens.
3. Switch the embedding model to a lower-dimensional model to reduce vector storage costs.
4. Increase the chunk size so fewer, larger chunks are retrieved per query.

Answer: A — Tune the retriever to return fewer, higher-relevance chunks, for example via re-ranking or a stricter similarity threshold, before sending context to the model.

Right - because the identified waste is irrelevant input context, so improving retrieval precision (re-ranking or a tighter similarity threshold) removes the unnecessary tokens at the source before they ever reach the model. Shortening the output targets a different cost line - output tokens - and does nothing about the 11-12 irrelevant chunks being sent as input on every call. Lowering embedding dimensionality can cut vector storage and search cost, a true but irrelevant fact here, since the stated problem is prompt token spend, not storage. Increasing chunk size without improving relevance just packs the same irrelevant content into fewer, larger chunks, which can increase rather than decrease total tokens sent.

An engineer wants to detect when Bedrock is rejecting requests because the account is exceeding its request-rate quota. Which CloudWatch metric should they monitor?

1. Throttles, which increments when Bedrock rejects a request due to rate limiting.
2. InputTokenCount, since a spike in input tokens indicates the account is nearing its throttling limit.
3. InvocationLatency, since throttled requests will show as latency spikes in this metric.
4. InvocationServerErrors, which tracks 5xx errors returned by the model.

Answer: A — Throttles, which increments when Bedrock rejects a request due to rate limiting.

Right - because Throttles is the metric Bedrock emits specifically when a request is rejected for exceeding rate limits, making it the direct signal for this failure mode. InputTokenCount reflects usage volume and can correlate loosely with load, but it is not a direct indicator of whether requests are actually being throttled. InvocationLatency measures how long successful or in-flight calls take; a throttled request is typically rejected quickly with an error rather than appearing as a slow call, so latency is the wrong signal here. InvocationServerErrors tracks server-side 5xx failures, a distinct failure category from client-facing rate-limit rejections, so it would not reliably surface throttling.

What does 'model right-sizing' mean as a cost optimization strategy?

Selecting the smallest or least expensive model that still meets an application's accuracy and latency requirements, rather than defaulting to the largest available model for every task.