Personalised Product Recommendation Engine

Project overview

We built a personalised recommendation engine for an online retailer selling across many product categories. The engine decides which products appear in the home page carousels, the "customers also bought" block on product pages, cart add-ons and the product slots in marketing emails, and it adapts to what a shopper does during the session.

The engagement covered event tracking, data pipelines, model development, a low-latency serving layer and a console for the merchandising team. It was delivered by our machine learning development team and complements the storefront work we do for e-commerce and retail businesses.

The client and the challenge

Recommendations on the store were hand-curated or based on simple most-popular lists. Every shopper saw the same products, new arrivals rarely surfaced, and slow-moving stock was promoted through manual banners that merchandisers had to keep updating. A previous plug-in suggested items that were out of stock or already purchased.

The retailer wanted suggestions that reflected each shopper's interests, fast enough to render with the page, while keeping merchandisers in control: pinning campaign products, excluding restricted items and respecting stock and margin. Many visitors were anonymous or new, so the system also had to do something sensible with very little history. The catalogue also changed daily, with seasonal ranges arriving and selling out quickly.

Solution architecture & delivery

We replaced the ad hoc tracking with a consistent event stream for product views, searches, add-to-cart, purchases and returns. A nightly Python pipeline cleans these events, joins them with the product catalogue and builds training data, while recent session events are written straight to Redis so the engine can react to what a shopper is doing right now.

The core model is a two-tower retrieval network in TensorFlow: one tower embeds shoppers from their history and context, the other embeds products from their attributes, category and text. Candidate products are retrieved by embedding similarity, then a ranking model re-orders them using price band, freshness, stock and the placement they will appear in. Anonymous shoppers are served from session-based and category-level signals until enough history exists. Product-to-product similarity from co-purchase patterns powers the "customers also bought" placement, and brand-new products borrow embeddings from their attributes until they collect interactions of their own.

Serving is split from training. Product embeddings and precomputed candidate lists live in Redis, so a request only has to look up the shopper, fetch candidates and run the lightweight ranker. A business-rules layer then removes out-of-stock, restricted or already purchased items and inserts any products the merchandisers have pinned for a campaign. If the model service is ever unavailable, the API returns cached category lists so the page still renders a sensible carousel.

The merchandising console lets staff set rules per placement, preview what a given shopper segment would see and run A/B tests between model versions and the old lists. Model releases pass automated offline evaluation first, following the approach in our CI/CD and testing pipeline.

Key features we built

• Personalised home page, product page, cart and email recommendations
• Real-time session signals so suggestions change as a shopper browses
• Fallback recommendations for anonymous and first-time visitors
• Automatic exclusion of out-of-stock, restricted and already purchased items
• Merchandiser pinning and boosting rules per placement
• Segment preview showing what different shoppers will see
• A/B testing between model versions and rule-based lists
• Scheduled retraining with offline evaluation before release

How we delivered it

1. Discovery. Workshops with merchandising, marketing and engineering teams to agree placements, business rules and how success would be measured in A/B tests.
2. Data foundations. Event tracking fixes, catalogue clean-up and the first training datasets, with simple popularity baselines to compare against.
3. Model and serving build. Two-week sprints on the retrieval and ranking models, the Redis-backed serving layer and the console, with the storefront integration built alongside.
4. Controlled launch. Recommendations enabled on one placement at a time behind an A/B test before replacing the old lists, much like the phased releases on our electronics store build.
5. Optimisation. Ongoing retraining, new placements and model tuning under a support agreement.

Technology choices

• Python — the data pipelines, feature preparation, evaluation scripts and recommendation API share one language and one set of libraries.
• TensorFlow — flexible building blocks for two-tower retrieval and ranking models, and straightforward export of trained models for serving.
• Redis — in-memory storage for embeddings, candidate lists and live session events, so recommendations render with the page.

Outcomes

• Each shopper sees products related to their own browsing and purchases instead of one most-popular list
• New arrivals and long-tail products surface without manual banners
• Out-of-stock and already purchased items no longer appear in recommendation slots
• Merchandisers keep control of campaigns through pinning and exclusion rules
• Model changes are judged through A/B tests rather than opinion
• The same engine serves the website and marketing emails from one place

FAQ