Most personalization systems start out simple, and they should. A scheduled job computes recommendations overnight, writes them to a table, and an API serves them on request. It is cheap, predictable, and for a lot of products it is entirely good enough.
The problem or mismatch shows up when someone’s recent behavior stops matching their history. Imagine a listener who almost always reaches for high energy music, but who opens the app one evening, searches for something acoustic, and skips the upbeat tracks the system keeps offering. Nothing is technically broken, still the model is accurate, the API is fast, last night’s recommendations were computed correctly. They just describe a person who isn’t there anymore. Offline metrics stay green while the experience quietly feels off.
The fix is not to rip out the batch and stream everything. It is to find the specific places where freshness changes the decision, and to spend real-time complexity only there.
Why Batch Still Earns Its Place
Batch is the right tool for anything that changes slowly: long-term preferences, historical engagement, item similarity, segments, embeddings, daily popularity. None of these need recomputing on every request, and running them on a schedule costs little and rarely surprises you. Precomputed recommendations are also cheap and boring to serve, a fast lookup with no inference in the request path. Boring is a compliment in production.
Batch starts to hurt only when the current moment matters. A run of recent searches, clicks, or skips can signal an intent that has nothing to do with a user’s long-term profile, and trending content can move faster than an overnight job can see. So let the recommendation type decide the machinery: a daily personalized list can stay batch, trending content fits nearline updates, session-aware suggestions need streaming, and context-aware ranking wants request-time data. One platform usually runs several of these at once.
Batch Gives You Memory; Streaming Gives You Awareness
A mature stack is really a few pipelines running at different speeds, each matched to how fresh its signals need to be.
| Layer | What it captures | Profile |
| Batch | Long-term preferences, history, embeddings, daily popularity | Lower cost, lower freshness |
| Nearline | Recent activity over seconds to minutes — session summaries, trending velocity | Medium cost, medium freshness |
| Real-time | The current request: surface, device, time, the preceding action | Higher cost, highest freshness |
Batch pipelines are the long-term memory. Nearline pipelines handle the recent past without dragging that work into the live request. Real-time is reserved for the present. The layers are meant to cooperate: pushing every feature into real time buys you cost, latency risk, and complexity, and often buys nothing back. In systems I have worked on, some of the largest reliability and latency improvements came from how features were computed, cached, and served, rather than from introducing a more sophisticated model. A simpler model using fresh, trustworthy features can outperform a more sophisticated one working with stale context that is the one idea worth designing around.
An Event-Driven Reference Architecture
The real-time loop starts when the application emits behavioral events like searches, clicks, plays, skips, saves, purchases etc. An event platform like Kafka or Kinesis carries them to stream-processing jobs that validate, enrich, aggregate, and reshape them to features.
Events and feedback flow from the application into an event stream and stream processing, which write to an online store (low-latency serving) and an offline store (training). A batch pipeline supplies long-term features and models. At request time, retrieval narrows the candidates and ranking orders them behind a serving API — and the outcomes become the next round of events.
Those features land in two places, and the distinction matters. An offline store keeps historical values for training and backtesting; an online store keeps recent values available for low-latency lookup at inference time. The two views have to stay meaningfully aligned. A model trained on features computed one way in batch may behave unpredictably when production computes them differently in the stream.
Jayakumar Ramalingam, author of this blog post is going to be a speaker during the upcoming Applied AI Summit. Secure your spot now!
When a request arrives, the system first gathers candidates like trending, similar items, editorial picks, precomputed results, vector search and then ranks them, combining model scores with user features, item attributes, session context, availability, diversity, and live experiments. I find it clarifying to keep the stages honestly separate: retrieval answers “what could we recommend?” and ranking answers “what should we recommend right now?” The API in front works best as a stable decision layer, so sources, models, and fallbacks can change underneath without breaking the product surface.
Keeping Training and Serving Honest
Running two feature stores signs you up for a consistency problem. Small differences in time windows, filtering, defaults, or event ordering are enough to make the same feature take on different values in each. Your offline pipeline might count skips over a clean thirty-minute window while the online one counts whatever arrived in the last thirty minutes; add some delay or a few out-of-order events and the numbers drift apart, quietly, with no error anywhere.
You will not make them perfectly identical, and that is fine but the goal is differences that are small, explainable, and watched. A few habits cover most of it: write each feature’s logic once and share it across training and serving, key aggregations off the event’s own timestamp rather than when you processed it, log the feature values used at each decision so you can reconstruct any call, and compare live feature distributions against the offline reference so a divergence shows up as a number rather than a mystery.
Designing for the Day It Breaks
Real-time dependencies fail in ordinary ways: the stream falls behind, the online store gets slow, ranking blows its budget. A production system should already know what it does when that happens, because deciding during an incident can produce stalled requests and unreliable results. Serve cached features when the store lags, fall back to precomputed recommendations when the model is down, return popularity results when ranking times out. The system should bend in a way you chose and tested.
That only works if observability reaches past API uptime. An HTTP 200 tells you the endpoint answered; it says nothing about whether the answer was fresh or relevant. You want eyes on stream lag, feature freshness, missing values, schema changes, gaps between the online and offline stores, fallback frequency, and whether the recommendations are staying diverse alongside the usual engagement and conversion numbers, since a model metric can improve while the experience does not.
Freshness also tends to mean collecting more behavioral data, so privacy belongs in the design rather than bolted on later: collect what you need for a stated reason, keep sensitive attributes out of pipelines just because they are available, and maintain lineage so you know where data came from and which models use it.
None of this argues for making everything real time. It argues for putting freshness where it changes the decision and leaving it out where it does not, while holding onto reliability, privacy, and cost. The strongest personalization architectures I have worked with tend to pair long-term memory with present-moment awareness, learn from feedback, remain observable, and degrade gracefully when real-time components fail.
Author: Jayakumar Ramalingam
Jayakumar Ramalingam, is going to be a speaker during the upcoming Applied AI Summit. Secure your spot now!