Why an AI project starts with data

A familiar trajectory: the pilot showed an excellent result while uploading three files, the decision was approved, real sources began to be connected - and the quality collapsed. The reason is almost always the same. In the pilot, the data was prepared manually: one person selected the current files, threw out duplicates, and corrected the date format. There is no such person in the product, but there are six systems, each with its own directory of counterparties and its own idea of ​​what an “active contract” is.

The model does not eliminate contradictions in the data - it strengthens them and makes them convincing. Where a report would show two inconsistent numbers and raise questions, the language model will select one and explain it in coherent text. This is worse than an error: an error is noticeable, but a confidently wrong answer is not.

1. Availability

Data should be retrieved programmatically, and not downloaded by a person upon request. If the only way to get digital is to ask a colleague to upload it, automation is impossible in principle.

2. Consistency

One essence - one definition. As long as “revenue” is calculated differently in reporting and in CRM and it is not written anywhere what exactly it is, any answer from the model will be controversial. You should start not with a complete dictionary, but with two dozen key indicators around which decisions are made.

3. Origin

Each value must have a traceable path: from which system, by what transformation, when. Without this, it is impossible to check the answer, analyze the incident, or explain to the regulator where the figure came from.

4. Distinction

Permissions should be a property of the data, not an application-specific setting. Otherwise, each new AI scenario requires a separate analysis of “who can do this,” and at some point the analysis is skipped.

5. Freshness

Each set has an acceptable lag and the user must understand it. The answer based on data from a week ago is not wrong - it is dangerous precisely when it is not said so.

Most corporate AI scenarios work not with tables, but with documents: regulations, contracts, protocols, correspondence. This has its own requirements, and the usual storage practices are not transferred directly.

• Slicing. The document is split into fragments. Too small ones lose context, too large ones blur the search. Partitioning by semantic sections is almost always better than partitioning by number of characters.
• Fragment metadata. Source, section, date, version, owner, stamp. Without them, the response cannot be attributed and rights cannot be applied.
• Deduplication and relevance. The situation is typical in corporate repositories: there are seven versions of the regulations, one of which is in effect. If all seven are included in the index, the system will confidently quote the canceled one.
• Fragment-level delimitation. A document may be publicly available, but its attachment may not.
• Index update. We need a process: how changes in the source reach the index and in what time. A one-time download becomes obsolete in weeks.
• Recognition quality. Scans and PDFs without a text layer give garbage at the input; This is a separate work that cannot be missed.

The engineering part of this work is Enterprise RAG. It is cheaper to test hypotheses on a limited environment: RAG pilot using one real knowledge base, it shows both the quality of answers and the state of documents.

The conclusion “let’s build storage first, then in three years we’ll work on AI” is just as wrong as trying to do without data at all. A working order is a narrow outline for a specific scenario, expanded as it becomes useful.

• Step 1: Select a scenario with measurable value. Not “implement AI”, but “reduce the time to prepare a response to a client’s request.” Inventory of scenarios and AI already used in the company - AI-discovery.
• Step 2: Parse the data for this scenario only. What sources, what quality, what rights, what acceptable lag. The amount of work becomes visible.
• Step 3. Close the found gaps. Typically these are two or three reference books, one metric definition, and one update process.
• Step 4. Pilot on real data, not on upload. This is the only way to check what really breaks.
• Step 5. production environment. The script is transferred to the common platform - Restart AI Enterprise Platform, — and the next scenario starts on a ready-made foundation.

Separately: if the script does not need real personal data, masking and depersonalization removes half of the approvals and significantly speeds up the pilot.

• The list of sources relevant to the scenario and the method of programmatic access to each are known.
• For key indicators there are agreed upon definitions recorded in writing.
• Directories are consolidated: one entity does not exist under four identifiers.
• The origin of values ​​is traced back to the source system.
• The acceptable lag is known for each set and is visible to the user.
• Permissions are described as a data property rather than an individual application setting.
• For documents: the rules for cutting, the composition of metadata and the process of updating the index are defined.
• Invalid document versions are excluded from the index.
• Scans and PDFs without a text layer are processed or deliberately excluded.
• It is defined what data cannot be sent to the model at all, and this restriction is technically enforced.
• There are regular quality checks, rather than a one-time cleaning before the pilot.