Insights

Introduction

Insights enable you to automatically generate reports and gather data from your agent executions. If you need to evaluate certain aspects of your agent's interactions, classify main usages, or assess performance, Insights allow for easy configuration and sampling.

How to Use

In this example, we will use an existing agent to configure AI Insights and demonstrate how conversations will be analyzed and processed. For guidance on creating an Assistant Agent, refer to this guide.

Create an Insight

1. Navigate to the Insights & Data Collection tab.

   

2. Click the switch to activate Insights collection.

   

3. Define the following values:

   • Analysis Sample Rate: Set the percentage of conversations or agent executions to be analyzed. A higher sample rate means more conversations will be reviewed.
   • Model: Select the supporting model that will analyze and generate the data.
   • Additional Instructions: Provide extra context on how the analysis should be performed. This helps the supporting Model better understand the purpose of your agent's interactions.

   

4. Add custom fields and define how the agent should collect that data. You can choose between different data types and provide custom instructions on how each field should be completed during evaluation.

   

5. Once you have configured all the fields in AI Insights, save the agent and start using it.

   

6. After your agent is published, the defined Insights will be generated automatically. This process runs in batches in the background based on closed conversations, so it may take some time for data to be available.

Access Insight Data

Once your agent is published and data has been generated, you can access it from the agent card by clicking the Data Collection button. On this page, you can view all the AI Insights data collected by the agent and export it to an Excel file.

For each conversation analyzed, a new row will be generated in the Insights Grid. These rows display the following information:

• Channel: The channel where this instance was generated.
• Start date: The time when the first interaction of the conversation was recorded.
• Data columns (e.g., Mood and Products): For each insight field defined in the Agent Designer, a column is added to the Insights grid. Here you can see the data generated for each field.
• Details button: Quickly access the conversation that generated this data, allowing for easy auditing.

Insights Versioning

Insights are versioned: each change to the Insights definition creates a new version linked to the agent's version. If changes are made to the agent but not to the form definition, the form version remains the same when saving. From the Form grid, you can filter collected data by version.

You can quickly change and see the information generated for each Insight version by adjusting the filters on the grid.