MLOps interview questions

This page gives MLOps interview questions and answers, written the way an engineer who has run models in production would answer. Candidates can use the page to prepare, and interviewers can use it as a question bank that notes what a strong answer includes.

The questions move from fundamentals to system design. Each answer gives the core points first, and strong candidates add an example from a system they ran. For background, read What is MLOps?. For a study plan with projects, read Learn MLOps: a practical learning path.

What an MLOps engineer interview tests at each level

LevelTypical questionsWhat a strong answer shows
JuniorDefinitions, versioning, batch versus online, basic testsCorrect terms and one worked example
Mid-levelDrift detection, registries, CI for models, rollbacks, debugging latencySpecific tools, thresholds and failure cases
SeniorPlatform design, real-time systems, label delay, what not to buildTrade-offs, cost, staffing, and how the design fails

Fundamentals

What is the difference between MLOps and DevOps?

DevOps manages the lifecycle of code: build, test, release and run. MLOps keeps all of that and adds data and trained models as versioned artifacts. A model's behavior depends on its training data, hyperparameters and code together, so all three need versions and tests. A model can also get worse with no code change, because the input data shifts. MLOps monitoring therefore covers input data and prediction quality as well as uptime and latency. A good answer names reproducibility, drift and training-serving skew.

What is training-serving skew, and how do you prevent it?

Training-serving skew is a difference between the feature values a model saw in training and the values it receives in production. Common causes:

  • separate code for training features and serving features, such as pandas in training and Java in the service,
  • different data sources, such as a warehouse table in training and a cache in serving,
  • leakage in the training set, where a feature uses data from after the prediction time,
  • different library versions in the two environments.

To prevent it, compute features from one shared definition, or package the preprocessing with the model as one artifact. Build training sets with point-in-time joins, so each row only uses data available at that moment. Log the feature vectors the service receives and compare their distribution with the training data.

What is the difference between data drift and concept drift?

Data drift, also called covariate shift, is a change in the input distribution, such as a new customer segment after a marketing campaign. Concept drift is a change in the relationship between inputs and the target, such as fraudsters changing tactics so that the same transaction pattern now has a different label.

Data drift can be detected without labels by comparing input distributions. Concept drift usually needs labels or a good proxy, because the inputs can look unchanged. Drift alone is not a reason to retrain, because the decision depends on whether quality metrics got worse.

What would you version in an ML project, and how?
  • Code in Git.
  • Data with a tool that keeps pointers in Git and the content in remote storage, such as DVC, or with immutable dated partitions and a recorded hash per run.
  • Models in a registry, with a version number, a link to the run that produced the model, and an alias. An alias is a movable name, such as champion, that points to the version in production.
  • Configuration and hyperparameters stored with the run record.
  • Environment as a lockfile and a container image digest.

The setup passes when you can rebuild any production model from these records alone.

Training is not reproducible even with the same code and data. What do you check?

Check these sources of variation in order:

  • random seeds for Python, NumPy and the ML framework, plus data shuffling and train-test splits,
  • library versions, which a lockfile and a pinned image control,
  • non-deterministic GPU operations, which PyTorch and TensorFlow can turn off with a setting that slows training,
  • the data query itself, which may return new rows or a different order when the source table changes,
  • multi-worker data loading and distributed training, which change the order of updates.

Some variation is acceptable, so agree on a tolerance for the evaluation metric across repeated runs rather than demanding identical weights.

Pipelines and tooling

What does an experiment tracker give you, and what should every run record?

An experiment tracker gives you a searchable history of what the team tried, and a link from a deployed model back to the run that made it. Each run should record:

  • parameters and the config file,
  • metrics, including per-segment metrics where segments matter,
  • the Git commit,
  • the data version or hash,
  • the environment, as a lockfile or image digest,
  • the output artifacts, such as the model file and evaluation plots.

MLflow Tracking is a common open-source choice. A tracker without the commit and the data version cannot reproduce a model.

What is a model registry for, and how do aliases work in MLflow?

A model registry records which model versions exist, where each one came from, and which one serves production. The registry keeps an audit trail of each promotion to production, and rollback becomes a pointer change. The serving code asks for "the production version of model X" and does not need to know which run produced it.

In the MLflow Model Registry, each registered model has numbered versions, and an alias is a named, movable pointer to one of them. Promotion is a call such as MlflowClient().set_registered_model_alias("fraud", "champion", 7), and the service loads models:/fraud@champion. Rollback moves the alias back to version 6. MLflow deprecated the older Staging and Production stages in version 2.9 and recommends aliases instead. Interviewers may ask about this change.

When would you use a feature store, and when not?

A feature store helps in three cases:

  • several models share the same features,
  • features must have the same values in offline training and online serving,
  • point-in-time joins for training sets are hard to write by hand.

Feast, an Apache 2.0 project, is one example. It keeps an offline store for training data and a low-latency online store for serving, and it builds training sets with point-in-time joins. Each training row then uses only feature values that existed at that row's timestamp.

For one or two models with simple features, a feature store adds a system to run and a layer to debug. A strong candidate names the problem before naming a product.

How would you automate retraining, and what triggers it?

Put the steps in a pipeline that runs without manual work: data validation, preprocessing, training, evaluation and registration. A run can start on a schedule, when enough new labeled data arrives, or when monitoring reports degradation.

Retraining alone is not enough: the pipeline must compare the new model with the current production model on the same held-out data and on key segments. Only a model that passes is registered as a candidate. Promotion to production requires those checks to pass, with a human approval step when the cost of a bad model is high.

Deployment

Batch or online inference: how do you decide?

Ask two questions. Is the prediction needed at the moment of the request? How old can the input data be before the prediction is wrong? If scores can be computed on a schedule and looked up later, batch is cheaper and simpler to validate and roll back. Online serving is needed when the inputs only exist at request time, or when a stale score causes harm.

Online serving requires on-call duty, latency budgets and autoscaling. A hybrid design computes heavy features in batch and runs a light model online. The page Model deployment strategies compares batch, online and hybrid serving.

Explain shadow, canary, blue/green and A/B deployments.
  • Shadow. The new model receives a copy of live requests, and its outputs are only logged. Users see no effect, so shadow testing cannot measure the effect on user behavior.
  • Canary. A small share of real traffic goes to the new model, with automatic rollback when metrics get worse. In KServe, the canaryTrafficPercent field of an InferenceService sets that share.
  • Blue/green. Two full environments run side by side. Traffic switches from one to the other at once, and rollback switches it back.
  • A/B test. Users are split at random into stable groups, and a business metric decides which model wins.

The first three methods control release risk, while an A/B test answers a causal question about which model performs better.

A model is fast in staging and slow in production. How do you investigate?

Measure before changing anything. Split the latency into stages: network, deserialization, feature lookup, preprocessing, inference and post-processing. Then compare production with staging on these points:

  • payload size and batch size,
  • cold starts, meaning the delay when a new container starts and loads the model,
  • CPU throttling and memory limits on the container,
  • hardware, such as a GPU in staging and CPU in production,
  • occasional very slow calls to dependencies such as the feature store.

Look at p95 and p99 latency, not the average. Typical fixes are request batching, a smaller or quantized model, caching, and moving feature computation into a batch job.

How do you roll back a bad model?

Deploy by an immutable version, keep the previous version loaded or ready, and make rollback a config or traffic change instead of a rebuild. For a batch model, rerun the job with the previous version. Set rollback thresholds before the release.

Also handle the predictions the bad model already made, because scores written to a table or acted on may need correction. Logging the model version with every prediction makes that cleanup possible.

How do you serve a large model on GPUs efficiently?

GPU cost depends on the share of time the GPU does useful work. The main techniques:

  • dynamic batching, which groups requests that arrive within a few milliseconds,
  • quantization to lower precision, such as INT8, after checking the accuracy loss on the evaluation set,
  • a smaller distilled model when the larger one is not needed,
  • autoscaling on queue length or GPU use rather than on CPU,
  • keeping model weights on local disk or in memory to shorten cold starts.

A strong candidate states the latency target and the cost per thousand predictions before choosing among these techniques.

Monitoring and testing

What do you monitor for a model in production?

Monitor four layers:

  • Service health: latency, error rate, throughput and resource use.
  • Data quality: schema violations, null rates, value ranges and freshness.
  • Drift: the distributions of inputs and predictions compared with a reference window.
  • Model quality: accuracy-type metrics once labels arrive, plus business proxies that arrive sooner.

Every metric should split by model version and key segments, and every alert needs an owner and a runbook.

How do you detect data drift, and which statistical tests do you use?

Compare a recent window of production inputs with a reference window, usually the training data or a stable period. The test depends on the column type and the sample size. The open-source drift library Evidently uses these default tests:

  • numeric columns with up to 1,000 reference rows: two-sample Kolmogorov-Smirnov test at a 0.95 confidence level,
  • categorical columns with up to 1,000 reference rows: chi-squared test,
  • numeric columns with more than 1,000 reference rows: Wasserstein distance,
  • categorical columns with more than 1,000 reference rows: Jensen-Shannon divergence, with drift at 0.1 or above.

Statistical tests on large samples flag tiny, harmless shifts, which is why distance measures with thresholds are used at scale. A dataset-level alert usually fires when a fixed percentage of columns drift, such as half of them. Thresholds need tuning against past incidents, or the alerts get ignored.

Labels arrive three months late. How do you know the model is still good?

Use leading indicators, signals that are available before the labels arrive:

  • monitor input and prediction distributions against the training reference,
  • track proxy outcomes that arrive sooner and correlate with the final label,
  • send a sample of cases for fast manual labeling.

Also run a backtest when the labels arrive: compute quality for each past period and model version, and check how well the leading indicators predicted the real outcome.

How do you test ML code and models in CI?

Use several layers of tests:

  • unit tests for data transformations and feature logic,
  • data tests on the schema and distributions of the training set,
  • a training smoke test: a short run on a small sample that checks the loss decreases and outputs have the right shape and no NaN values,
  • model tests on a fixed evaluation set, with minimum thresholds per segment and a comparison with the production model,
  • behavioral tests, such as checking that changing a customer's name does not change a credit score, and tests for known edge cases,
  • serving tests that start the container, check the API contract, and run a short load test.

pytest runs most of these layers. Google's ML Test Score paper lists 28 tests and monitoring checks a candidate can cite.

System design

Design an ML platform for ten data scientists with a few models in production.

Start from the current problem: which step is slow or risky today? A reasonable first design has these parts:

  • Git projects created from a standard template,
  • one shared tracking server and model registry,
  • containerized training jobs started by CI or a scheduler,
  • a standard serving template, such as a container with a validated API,
  • shared monitoring dashboards and alert routing.

Add an orchestrator when pipelines depend on each other, and a feature store when duplicated features cause real inconsistencies. Name the build-or-buy choices and the people needed to run each part. Shared platform parts break over time, and if nobody owns them, every team that depends on them is blocked.

Design deployment and monitoring for a real-time card fraud model.

Clarify requirements first: the latency budget, the availability target, the cost of false positives and false negatives, and the label delay, since chargebacks arrive weeks later.

  • Serving: a low-latency online service. Account-level features are precomputed into a fast key-value store, and request-time features are computed in the service. A rule-based fallback runs when the model is unavailable.
  • Release: shadow first, then a small canary with automatic rollback on errors and latency, plus monitoring of approval and decline rates.
  • Monitoring: input drift, score distribution, decline rates by segment, and delayed precision and recall as chargebacks arrive.
  • Audit: store every decision with the model version and the feature vector.

Fraudsters adapt to the model, so concept drift is expected, and the retraining plan should assume it.

Design a recommendation system that retrains daily.

Split the work into a batch path and an online path. The nightly batch job builds training data from the previous day's interactions with point-in-time joins, retrains, and evaluates on a held-out recent day. It then precomputes candidate lists per user. The online service reranks the candidates with request-time context, such as the current session.

Promote a daily model only when its offline ranking metrics match or beat the current model. Measure the real effect with an A/B test on clicks or purchases. Monitor the catalog coverage of recommendations, because a feedback loop can narrow the items the model shows.

LLM and generative AI operations

How does operating an LLM application differ from operating a classical model?
  • The versioned artifacts are prompts, retrieval indexes, tool definitions and evaluation sets. Many teams train nothing.
  • In an LLM application, a failure is usually a wrong or unsupported answer, so evaluation needs programmatic checks, model-graded scoring and user feedback in addition to service metrics.
  • Cost per request is an operating metric, because it grows with input and output tokens.
  • Providers update and retire model versions, so every model change needs a regression run on the evaluation set.
  • Traces of each step, including retrieval and tool calls, are the main debugging tool.

The page LLM monitoring and observability covers tracing and evaluation for LLM applications.

How would you evaluate a RAG application before launch?

Evaluate retrieval and generation separately. Build a question set from real documents and record the source passage for each question. Then measure two things:

  • retrieval: whether the right passage appears in the top results,
  • generation: whether the answer is correct and supported by the retrieved passages.

Have people review a sample of answers to calibrate any model-based judge. Include questions whose answers are not in the documents, which the app should decline. Run the set in CI for every change to prompts, models or retrieval settings.

Experience questions

Interviewers learn the most from questions about real systems. Expect these:

  • Describe a model you ran in production. What broke, and how did you find out?
  • What would you do first with a model that only one person knows how to retrain?
  • What did you decide not to automate, and why?
  • How did you decide a model was good enough to ship?

Candidates who have run real systems answer with a specific incident, the signal that caught it, the fix, and the change they made afterward.

Frequently asked questions

Which tools should I know before answering MLOps interview questions?

Know one tool per stage well enough to explain how it works: Git and DVC for versioning, MLflow for tracking and the registry, Docker, one CI system, one orchestrator, one serving approach, and one monitoring stack. Interviewers care more about why you chose a tool than about how many tools you list.

Do certifications help in MLOps interviews?

A cloud certification shows platform knowledge, which helps for roles on that cloud, but a public repository with a working pipeline gives the interviewer more to discuss. The page Learn MLOps: a practical learning path lists the current certifications and their status.

Need this built?

We build training pipelines, experiment tracking, model registries, CI checks for models, serving and drift monitoring as part of our MLOps services. Tell us about your project, and we will reply with questions or a rough plan.

Privacy engineering notes by email

When we publish a new guide on data mapping, DSR automation or AI privacy, we send it to this list. Nothing else.