Tricentis AgentScore: score the agent before you ship it

Tricentis announced three technologies at its Transform conference on August 20: Aida, AgentScore, and Release Risk Intelligence. The headline product is AgentScore, which the company describes as moving quality engineering "from deterministic testing to probabilistic evaluation." That phrase is doing real work. If you ship anything with an AI agent in it, agent testing is now a different discipline than the test suite you already own — and you need a version of AgentScore whether or not you ever buy one.

What actually happened

Per Tricentis' announcement, all three come out of Tricentis Labs — an innovation incubator that gives users early access. These are exploratory technologies under development, not shipping GA products, and no pricing or general availability date was published.

Aida is an autonomous exploration agent that surfaces defects and coverage gaps in web and Windows desktop applications without any pre-existing test suite or scripts. AgentScore observes AI agent behavior in real workflows and produces a composite quality score plus a recommendation on production readiness. Release Risk Intelligence surfaces release-scoped coverage gaps and prioritizes them by severity so a release decision has a number behind it. The announcement also ties in Tricentis' acquisition of Tabnine as an enterprise context layer.

Why probabilistic agent testing matters for your business

Your existing tests answer the wrong question. A unit test asserts that a function returns 4. An agent does not return 4 — it takes a path, and the same prompt takes a different path on Tuesday. Asserting exact output on a non-deterministic system produces a suite that fails constantly and gets muted, which is worse than no suite at all.

The unit of measurement is a run distribution, not a pass. Run the same task 20 times. Count how often it reaches the correct end state, how often it calls the wrong tool, how often it silently gives up. That is a score. A score has a threshold, and a threshold is something you can put in CI. This is what "probabilistic evaluation" means in practice, and you can build it with a JSON file of test cases and a script.

Define the block condition before you need it. The valuable half of AgentScore is not the number — it is the ship-or-block recommendation. Decide now what score blocks a deploy. 90% task completion? Zero destructive-tool calls in 50 runs? Write it down while nobody is waiting on a release, because during an incident every threshold becomes negotiable.

Coverage gaps are where agents actually hurt you. Release Risk Intelligence exists because teams ship the paths they thought about. With an agent, the dangerous paths are the ones nobody scripted — the refund it decided to issue, the record it decided to delete. Log every tool call in your eval runs, then look at the calls you did not expect. That list is your next set of test cases.

Sources: Tricentis announcement, VKTR.