Home › Certifications › GCP Professional ML Engineer › Serving and Scaling Models
60 multiple-choice questions and 21 flashcards on Serving and Scaling Models, about 20% of the GCP Professional ML Engineer bank. Every one carries a written rationale.
Serving and Scaling Models is one of 8 chapters in CoStudy's GCP Professional ML Engineer bank, and it holds 60 of the bank's 300 multiple-choice questions — roughly 20% of the total. That proportion is not arbitrary: chapters follow the certifying body's published exam outline, and the number of questions in each is set by that domain's published weight, so the share of your practice time this chapter takes matches the share of the real exam it accounts for.
Studying by chapter is worth doing once you have a diagnostic score. A single overall percentage tells you whether you are close; it does not tell you which domain is dragging. Working a weak chapter in isolation, and re-testing it in isolation, is the fastest way to move a score that has stalled — and it is why the mock exams in CoStudy report by domain rather than as one number.
10 questions drawn from this chapter, with the full rationale shown — the controlling principle behind the right answer, and why each wrong option tempts and fails.
Which statement about batch prediction and online prediction is LEAST accurate?
Answer: D — Online endpoints are the cheaper way to score a very large static dataset
D) Correct as the inaccurate statement — scoring a large static dataset through a request/response endpoint pays per-request overhead millions of times, which is the more expensive path. A) Accurate: batch compute is transient and billed only for the run. B) Accurate: warm replicas are exactly what buys low response latency. C) Accurate: scheduled bulk file input is the canonical batch scenario.
Cost trap: a model is deployed with one A100 to serve 10 QPS. Better choice?
Answer: D — Right-size to a T4 or L4 with batching, since idle A100 time is costly
D) The controlling idea is matching accelerator class to actual throughput: at 10 QPS an A100 sits mostly idle, so a smaller T4/L4 with batching serves the same load far cheaper. A tempts because scaling out is the reflex for load problems, but the problem is over-provisioning, not shortage. C tempts because newer hardware sounds like an upgrade, yet it raises cost for a workload that is already over-served. B tempts because retraining is a familiar lever, but model quality is not what drives this bill.
A custom TensorFlow model needs serving with high throughput and dynamic batching. Best container?
Answer: A — A prebuilt TF Serving container, or a custom Triton container
A) Correct — TF Serving provides dynamic batching for SavedModels, and Triton adds multi-framework support with batching. B/C) Poor fit for high-throughput model serving. D) A warehouse, not a prediction server.
A model registry holds versions 1 through 6. Version 6 is deployed to 100% of traffic and version 5 remains deployed at 0%. What is the MAIN benefit of leaving version 5 deployed?
Answer: B — It allows rollback by changing traffic weights, with no redeploy wait
B) Correct — a deployed-but-unweighted version is warm capacity for instant rollback, which is the operational reason teams accept its cost. A) Lineage lives in the registry independently of whether a version is deployed. C) Keeping a second deployment running costs more, not less. D) Monitoring baselines derive from configured reference data, not from idle deployments.
Which factor MOST strongly argues for serving a model with an accelerator rather than CPUs?
Answer: C — The model performs large dense matrix operations per request
C) Correct — accelerators pay off when the per-request arithmetic is large, dense, and parallel, which is a property of the computation rather than of traffic or storage. A) Artifact size drives memory requirements; a large sparse or lookup-heavy model may not benefit at all. B) High request rates can often be met more cheaply by scaling CPU replicas horizontally. D) Training and serving have different economics, so accelerator use in training does not carry over automatically.
Cost trap: which accelerator for low-cost inference of small CNN at moderate QPS?
Answer: D — T4 or L4 GPUs, paired with TensorRT for higher throughput
A/B/C) Overkill for a small CNN at moderate request rates; you pay for capacity you never use. D) Correct — the low-cost inference GPUs, with an optimized runtime, fit this profile.
A fraud team must return a decision within 80 ms at the point of card authorization. Which serving design BEST meets the requirement?
Answer: D — A managed online prediction endpoint with a low-latency feature lookup at request time
D) Correct — synchronous authorization needs a request/response endpoint paired with an online feature source, because the decision must exist before the transaction completes. A) Precomputed scores cannot reflect the transaction being authorized right now, so the fraud signal is stale by construction. C) Warehouse-side inference is analytical: query latency and concurrency behavior are not built for an 80 ms synchronous path. B) Asynchronous scoring returns after the authorization decision was already made, which defeats the control.
For batch prediction over 10M rows in BigQuery, the BEST approach is:
Answer: B — Batch prediction or ML.PREDICT into a table
A) Ten million sequential online calls is slow and costly. B) Correct — the managed batch prediction service, or ML.PREDICT in place, is built for high-volume offline scoring and writes results back to BigQuery. C) A single function cannot sustain that volume. D) Manual scoring is not operable at this size.
Cost trap: 'Spot/Preemptible nodes save cost on Endpoints.' Correct?
Answer: D — No — Spot fits interruption-tolerant training and batch work
A) Wrong — reclaimed nodes break user-facing latency guarantees. D) Correct — interruptible capacity suits jobs that can restart from a checkpoint. C) Wrong; endpoints scale across replicas. B) Wrong; Spot is discounted, not free.
Push vs pull serving — direction reversal: which is appropriate for a streaming personalization use case where features change every second?
Answer: D — Push precomputed features to the online store for low-latency reads
A) The offline store is built for training reads, not request-time latency. D) Correct — push fresh values into the online store and serve them at request time. C) Stale by a day. B) Discards the personalization signal.
4 cards from the 21 in this chapter.
What are the inputs and outputs of a managed batch prediction job?
Input is a registered model version plus a source in Cloud Storage (JSONL, CSV, TFRecord) or a BigQuery table, with a machine type and replica count. Output is written back to a Cloud Storage prefix or a BigQuery destination table, including per-row errors. No endpoint is created and compute is released when the job finishes.
When do you use batch prediction instead of an online endpoint?
When predictions are consumed asynchronously — scoring a full table nightly, backfilling scores, or feeding a downstream job — and no per-request latency requirement exists. Batch reads from Cloud Storage or BigQuery, writes results back in bulk, provisions compute only for the run, and costs far less than keeping an endpoint warm.
Vertex AI Endpoints?
Real-time serving infrastructure. Supports traffic splitting, autoscaling, GPU/TPU acceleration.
Vertex AI Private Endpoints?
Endpoint reachable only via VPC peering. Required for sensitive workloads.
These are a sample. The full Serving and Scaling Models chapter runs 81 items with per-chapter progress tracking, on the web and in the iOS app.
Open GCP Professional ML Engineer in CoStudy →