What happened
LLC "Invent," a company within the RESTART Group, has introduced Polygon—a test bench for acceptance and load testing of AI systems. The bench is deployed within the customer's environment and, in a single run, simultaneously answers two questions: how many concurrent users the AI assistant can support and whether its responses degrade under this load.
The product addresses a gap currently filled manually: load testing tools cannot evaluate response quality, quality assessment tools cannot generate load, and neither produces a document that can be attached to an acceptance certificate. Product landing page — polig-on.ru.
Two axes in one pass
Polygon loads the AI system according to a specified profile—from single queries to hundreds of simultaneous users—and simultaneously collects a sample of responses for quality evaluation. Both metrics are measured within a single test run on the same traffic, not by two different tools at different times on different data.
Load
Five profiles: single requests, one-time burst, stepwise concurrency ramp-up, constant background load with think-time, random arrivals over a time window. The test stand automatically determines the queue formation threshold and the failure threshold.
A failure that is not visible via HTTP
Failure is detected by the increase in the container restart counter, not just by the response code: a system that silently restarts under load won't be noticed by a standard load test.
Internal engine metrics
Queue and memory are read directly from the inference engine (vLLM): how many requests are actually being executed, how many are queued, and how much GPU video memory is occupied.
Quality of responses
Relevance, completeness, groundedness (alignment with sources), and hallucination indicators are assessed on the sample of responses. Each evaluation includes a textual explanation of the verdict, not just a bare number.
The test profile is the single source of truth: everything specified in it (load profile, reference dataset, acceptance thresholds) is quoted verbatim in the protocol.
Test protocol as an acceptance artifact
The main deliverable of a test run is not a web page with charts—which will disappear once access to the test stand is lost—but a test report in PDF and DOCX formats based on an editable template: title page, test program and methodology, test conditions, load and quality results, infrastructure status, acceptance verdict per criterion, conclusions written in plain language, appendices with examples of “question → answer → rating → explanation,” and a signature block.
| Criterion | Threshold | Fact | Result |
|---|---|---|---|
| Success rate | ≥ 99% | 97,4% | FAIL |
| P95 response time | ≤ 15 sec | 11.2 sec | PASS |
| Service downtimes | 0 | 0 | PASS |
| Overall quality rating | ≥ 80 | 84 | PASS |
| Simultaneous users without a queue | ≥ 12 | 9 | FAIL |
Example verdict lines are illustrative. In the case of FAIL, the report specifies the exact violated criterion, the expected value, and the actual result, rather than a general statement like "there are issues."
The stand is with you. The judge is too.
The main objection from the security team to any quality assessment platform is, “You’ll send our data to a third-party cloud.” This doesn’t happen in “Polygon” by design:
- The test environment is deployed using docker-compose in the customer's environment and installed from an offline archive of images—without any internet access at all stages, including installation;
- The LLM judge operates on a local model within the perimeter: the question, reference, actual answer, and criteria do not leave the perimeter;
- the judge's input data are methodologically limited—only the question, reference, answer, and evaluation criteria, without tokens, cookies, or session identifiers;
- Datasets are checked for personal data (phone numbers, email addresses, full names, card and account numbers) during import, with masking suggestions provided; the verification report is stored alongside the dataset version;
- Customer system access secrets are stored in environment variables or a secret vault, not in the test profile.
Minimum test bench configuration: 4 vCPU, 8 GB RAM, 50 GB disk; installation on a clean server is performed according to the instructions, without development.
Who needs this right now
Bank and financial institution
A release or major update of the RAG assistant, support chatbot, or corporate banking assistant is being prepared. The acceptance committee requires load metrics and confirmation that the bot does not start generating incorrect responses under load—all in a single document.
State sector
The GIS with an AI component is being prepared for commissioning. A "test program and methodology" document is required, with the result being a judge who physically cannot send data to the internet.
Major retail
The support chatbot is heading into its seasonal peak. We need to know in advance how many concurrent requests it can handle and whether it starts inventing non-existent promotions and prices during traffic surges.
How does this differ from k6 and Ragas?
Open-source tools each solve half the problem: k6 and JMeter handle load generation, while Ragas and DeepEval provide quality evaluation. There’s no integration between them, and building one yourself is a separate months-long project.
- none of them can see the queue and memory utilization inside the inference engine;
- container restart failure is not recorded—only the HTTP error is captured;
- Load and quality are calculated at different times on different data, not from a single run;
- There is no single document with a PASS/FAIL verdict suitable for an acceptance certificate at the output.
Polygon does not dispute that k6 and Ragas are good tools. It closes the loop of "load + quality + protocol + delivery into the environment," which these tools cannot address individually.
How the test is conducted
Application and call
We discuss the system: type (RAG assistant, chatbot, LLM API), available traffic of real user queries, and whether there is access to infrastructure metrics.
Test Profile
Together, we define the load profile, a dataset of reference questions and answers, and acceptance criteria—thresholds that determine PASS/FAIL specifically for your system.
Run in your environment
The testbed loads the target, simultaneously evaluates the response sample using a local judge, and captures infrastructure state if access is granted.
Protocol
A PDF/DOCX document with a verdict for each criterion—ready to be included in the acceptance certificate package.
The infrastructure metrics section is optional: if there is no access to containers and GPUs, the testbed still loads the target via HTTP, measures response and quality, and issues a verdict based on acceptance criteria.
Let's discuss your environment
Describe the task, current systems, constraints, and expected results. We will offer a practical first step: diagnostics, pilot, audit, roadmap or project team.
