Between demo and production
Everyone is talking about AI assistants in retail now. The demo, where the chatbot “understands human language” and selects goods, receives applause at conferences. But there is a huge distance between a spectacular demo and a system that handles millions of requests per day, doesn’t drop sales on Black Friday, and brings measurable benefits.
We are currently building such a system for a large federal retailer. This is not a pilot on the knee, but an industrial AI platform: an intelligent assistant that helps the buyer find a product, place and track an order, return a purchase and get advice - all through a normal conversation, as with a live seller. In this article, we will analyze the project from two sides at once: what the end customer and the business actually receive - and how it is arranged internally to work in production. And in the end, we’ll show you how to calculate the economics of such a solution so as not to overpay for the model.
What does the buyer get?
Let's start with the reason for which everything is started - with benefit. The assistant understands the request formulated in ordinary words and does not force the person to guess the “correct” wording.
The buyer can write “laptop for working with graphics up to 80 thousand”, “coffee machine with automatic cappuccino maker” or “what is from Samsung at a discount” - and receive a list of suitable products with prices, photos and availability status in his city. If the required model is not in stock, the bot will suggest analogues. For any product, he will provide a full description and characteristics, and when asked “how does the Galaxy A54 differ from the A55 and which is better for photography”, he will build a comparison table and give a reasonable recommendation.
Next is the entire transaction within the chat. “Repeat my last order” - the bot finds a previous purchase, checks availability, offers replacements for missing items and places a new order. Before confirming, he always states the result: “The order contains 3 items worth 45,780 ₽. Delivery to Moscow - 390 ₽. Total 46,170 ₽. Shall we arrange it?” Then - tracking (“where is my order”, “when will it arrive”, “change address”), processing payment and returns without calling support: the buyer gives the reason, the bot checks the possibility of a return by date and category of the product, confirms and creates a request.
A separate layer is consultations. “Will this filter fit my coffee machine?”, “How to get warranty repairs?”, “Which laptop is best for AutoCAD?” The bot answers such questions not with fiction, but on the basis of manufacturers’ instructions, warranty and return policies, FAQs and certificates - that is, according to verified company documents. And it remembers preferences: it recommends relevant items based on purchase and browsing history, and suggests discounted accessories for already purchased equipment.
All this works 24/7, responds in seconds to a simple request and up to several tens of seconds to a complex one - showing progress, and if the service fails, it honestly says “I can’t check availability now, try again in a minute” or transfers it to the operator.
Why is this more difficult than “screwing on GPT”
And now - the second half, for which demo and production are significantly different. The most common mistake when implementing such an assistant looks like this: Browser → FastAPI → GPT. The user writes, the backend keeps the connection open while the model generates a response for 25 seconds, and all this time the worker is busy. Under load, this scheme goes down instantly. Therefore, our platform is based on several strong architectural principles.
It is not the model that plans, but the rules. The sequence of tool calls is built by a rule-based Planner - deterministically, based on recognized intent and predefined rules. LLM does not “adjust the plan” on the fly, because adjusting the plan is, in fact, a business decision that we do not trust to the model.
Complete asynchrony. There are no long HTTP requests in the system. When a buyer sends a message, the request is completed in 20–50 ms: the client immediately receives a task_id, and processing goes to the RabbitMQ queue. Then the Celery worker picks it up, runs the entire AI pipeline, and the progress is published via Redis Pub/Sub and comes to the browser as a stream via Server-Sent Events. The user sees the statuses (SEARCH_PRODUCTS → RERANK → LLM_GENERATION) and receives an answer as it is generated, rather than looking at a spinning spinner. This event-driven approach allows you to independently scale AI processing, search and business services: during catalog import, you can raise 20 import workers without touching AI, and during a promotion, you can add recommendation workers.
Inside, the system is divided into independent environments, each with its own area of responsibility: a user interaction environment (Web, Telegram, mobile application - without business logic and without database calls), an AI environment (understanding the request, selecting tools, generating a response), a business logic environment (Catalog, Order, Customer, Payment, Return, Shipment), a search environment and a knowledge environment.
Knowledge Loop: Where Transactions End and RAG Begins
The platform deliberately separates two fundamentally different flows of information. The transaction loop works with products, prices, balances, orders and payments - its source is PostgreSQL. And the knowledge loop works with PDFs, instructions, FAQs, manuals, certificates and return policies - and its source is RAG (Retrieval-Augmented Generation).
It is this environment that is responsible for consulting “like an experienced salesman.” Documents go through a separate indexing pipeline: Ingestion → Parsing → Chunking → Embedding → Indexing → Hybrid Search → Reranking. Hybrid search - full-text on Elasticsearch plus semantic on Qdrant, followed by reranking via cross-encoder. And, which is critical for trust, the answer is always based on a specific source, and not on the “fantasy” of the model.
This is exactly the problem that in RESTART we are already solving with a separate product - Ragify, enterprise-RAG for searching and answering corporate documents. Ragify turns regulations, knowledge bases and project materials into controlled AI search with answers by source, supports hybrid search, downloading from PDF, DOCX, XLSX, Confluence, 1C-Bitrix and ERP, role-based access model, query auditing and deployment on-premise or in a private cloud. In the retail platform, the knowledge environment is built on the same engineering base: Ragify’s developments in reliable retrieval and source control are transferred to the client scenario almost without changes. This is the main benefit of the product approach - not reassembling RAG for each customer, but bringing a proven kernel.
Economics: how to calculate your budget and not overpay for a model
The technology is impressive, but the customer asks another question: how much does it cost and when will it pay off? And here it is important to consider honestly.
Let's start with the order of the numbers. A RAG pilot for one process via an external API costs approximately 1.5–2 million rubles. Local installation of the same scale - from 3 million rubles, and this does not take into account servers and operation. Next comes the arithmetic of payback, which is sobering. Let’s say a referral receives 100 requests per month, and the consultant spends 7 minutes on each – that’s about 12 hours per month. At a rate of 2,500 ₽/hour, the savings will be about 30,000 ₽ per month. With a pilot cost of 1.5 million rubles, the payback will take more than four years - this does not work for one narrow scenario.
The conclusion is not “RAG doesn’t pay for itself,” but “you can’t run it for one thin thread.” When the same platform closes five comparable processes, the annual effect reaches 1.8 million rubles, and the payback is compressed to 10–13 months. The economy of the AI assistant lives on scale and reuse - this is why we are building a single platform for the retailer, and not a dozen isolated bots.
The second saving lever is the correct choice of model for the cost of error. There is no need to use the most expensive model for all tasks. Routine, standard questions are perfectly covered by cheaper models; premiums are justified where several sources need to be compared and the cost of error is high. In our architecture, this is built in at the design level: rule-based Planner knows the type of request in advance, so you can route a simple search and a complex multi-source consultation to different models without “guesswork” on the part of the LLM.
The third factor that is often underestimated is data preparation. Long instructions need to be broken down into completed steps, preserve the table structure, extract text from pictures and add metadata (division, status, effective date, owner). Garbage at the input is either incorrect answers or overpayment for the fact that the model heroically parses a poorly prepared document.
How to launch: the pilot who gives the answer
The hypothesis should be tested on a controlled pilot. We recommend starting with 50–100 real work questions—real, not made-up—and checking the bot’s answers against the standard ones. The pilot measures clear metrics: the accuracy of source selection, the percentage of correct attribution of a response to a document, time saved per request and the cost of processing one request. The baseline also needs to be fixed in advance: the volume of incoming questions, response time and the share of outdated materials - otherwise there will be nothing to compare with.
What this gives in practice is shown by already implemented projects. In one of the RESTART cases, development took three months, and the speed of searching for information increased fivefold - employees found what they needed in seconds instead of hours. The benchmark for implementation ceiling is set by Morgan Stanley: their internal assistant indexes about 100,000 documents, and it is used by more than 98% of financial advisory teams. This is a sign of a mature solution - not a one-time wow effect, but a tool that people actually use in their daily work.
Where it already works: not only retail
Retail is not the first area where we are assembling such a system. The engineering core is the same: controlled retrieval, responses with a link to the source, and a strict layer between the model and the data. The subject area, sources of knowledge and regulatory requirements are changing.
- AI platform and RAG agents for a bank from the top 5 of Uzbekistan is a corporate AI platform where three knowledge bases live in one environment. The project passed the pilot, received a positive assessment from the customer and moved on to support, development and replication.
- RAG assistant for Spina Bifida for the foundation — the assistant helps families, patients, doctors and fund employees quickly find verified information and reduces the burden on primary consulting support.
Retail assistant, banking environment and medical fund cover different tasks, but rely on the same proven core - the same one that we develop as a product Ragify. It is the reuse of this core that makes the economics of such projects converge.
What's the result?
An AI assistant in retail is not a “chatbot on top of GPT”, but a distributed production system, where the model is responsible only for the language, and strict engineering is responsible for data, money and reliability: event-based architecture, MCP as a single contract to business logic, a hybrid RAG with answers by source and well-thought-out economies of scale. It's this combination - clear benefits for the buyer plus industrial reliability under the hood - that turns a spectacular demo into a system that pays for itself.
If you are trying out a similar scenario for your business, start with a small and honest pilot on real issues - and we will help you build a platform from it that scales.
RESTART is a technology group that helps large companies navigate complex technological changes: AI and data, enterprise platforms, information security and integration. Read more about Ragify →
Let's discuss your environment
Describe the task, current systems, constraints, and expected results. We will offer a practical first step: diagnostics, pilot, audit, roadmap or project team.
