Lyro scores 92% on policy compliance, ranks first in G2’s default leaderboard, and shares the highest overall score among nine evaluated agents.
Tidio announced that its AI customer service agent, Lyro, achieved the highest policy compliance score among nine agents tested in G2’s AI CX Agent Evaluations. Lyro scored 92% on compliance, measuring how consistently it followed the written support policy in G2’s simulated customer service environment.
For businesses using AI to process refunds, change subscriptions, or update accounts, completing an action is only part of the job. The agent also needs to determine whether that action is permitted under company rules. G2’s evaluation tests both what agents say and what they do when handling support requests.
As of today, Tidio ranks first in the leaderboard’s default “All Metrics” view. Lyro passed 40 of 46 tasks, earning 200 points and tying for the highest overall score. It also achieved 87% accuracy. Its compliance score was 15 percentage points above the average displayed by G2, while accuracy was seven percentage points above the displayed average. Overall points reflect passed tasks, with compliance and accuracy reported as separate measures.
“We focus on making it easier for customers to build and maintain the knowledge Lyro needs to handle support requests reliably. We also continuously test its performance across hundreds of scenarios, using our own LLM judge to identify weaknesses and guide improvements,” said Szymon Piechaczek, Senior ML Product Engineer at Tidio.
G2 evaluates products using shared, buyer-informed tasks in a simulated company equipped with customer and order data, a written support policy, and 38 working business tools. An AI judge assesses accuracy and policy compliance, while automated checks confirm whether required actions took place in the underlying systems. Evaluation scores are separate from buyer reviews.
Tidio participated at G2’s invitation and worked with the evaluation team on multi-step workflows. The process helped Tidio identify where those workflows broke down and uncover opportunities to improve Lyro.
“We’re investing in Lyro across the customer journey, with proactive shopping assistance to help ecommerce businesses generate revenue, new channels such as voice and SMS, and deeper Shopify integration. Making Lyro easier to set up is just as important, and conversational setup within the Tidio dashboard is another key area of investment for us,” said Sebastian Wilk, Senior Product Manager at Tidio.
The results represent performance within G2’s controlled test environment and reflect the agents evaluated to date. Additional products remain under evaluation, and G2 aims to refresh results several times a year or following material product changes.
View the full results and methodology: https://www.g2.com/categories/ai-agents/agent-evaluations.
About Tidio
Tidio is a customer service platform that brings AI and people together so businesses can scale support without losing the human touch. Used by over 300,000 businesses worldwide, Tidio combines a helpdesk, live chat, and AI-powered customer service. Learn more at https://www.tidio.com/.