CTGT releases open source tool Concordance to trace why AI agents make inconsistent decisions

VentureBeat · Carl Franzen · 2026-10-21 15:42 UTC

CTGT replaced the identifiers with stable labels, sorted the list and isolated each test in a fresh organizational environment. Five subsequent runs each produced the same 34 obligations.

Generative AI is an incredible business advance, giving companies the opportunity to handle a wide variety of knowledge work tasks with ever-changing, inconsistent input data. But what about ensuring the outputs remain consistent and are in-line with the business goals? That's a much harder task, given, by the fundamental way in which it is designed, gen AI is "non-deterministic" — that is, outputs are constructed on the fly and may vary wildly even with the same or similar input information.

In order to help businesses harness the tech and make it more consistent, the stealthy startup CTGT is releasing a new system called Concordance under the Apache 2.0 open-source license, giving enterprises free tooling to investigate why an AI application makes different decisions when asked to perform the same task.

The release includes the testing protocol, reference implementation and experimental data, according to CTGT. Businesses can use and modify the software commercially under the license; the company says it has no paid or hosted Concordance edition planned.

Read the full story at VentureBeat · All coverage of this story