Agent Copilot Rating

Agents within call centers are often under tight timelines, trying to provide customer service to as many people as possible, while still taking the time to take notes, follow up and make sure that all client needs are taken care of. 

Enter copilot, a product that aims to speed up this process through the power of AI. In order for copilot to help, however, copilot needs to get an understanding of the knowledge-base and needs to learn by continued teaching from agents and specialists to make sure answers are correct and trending in the right direction. The challenge was getting the right amount of feedback while still being mindful of tight agent timelines.

My original attempt, designed quickly for purposes of having a minimum viable product utilized common design patterns for thumbs up / thumbs down while still being able to provide the LLM feedback.

In our user testing, we identified several takeaways:

• Agents liked the idea that they would be directly contributing to teaching the AI. It gave them more confidence in the outcomes knowing that there was some human influence and feedback.
• Agents wondered if they would have time to complete this - Could they go back? Would it interfere with time taking notes? They also wondered if it would interfere with their calls with the customers.
• Agents liked the option to go into detailed feedback but admitted they would likely not do this for every knowledge-based answer card they received.

Taking into account that wrap-up mode only is about 30 seconds to a minute on average, we created a feedback experience that included tags - allowing an agent to quickly identify why a card was helpful or unhelpful, and limiting feedback to a line appeared to decrease the cognitive load and perception that this would take a lot of time based on our user testing. While other ideas have been tried for this experience, user testing was integral to creating an experience that agents could easily complete in a short period of time.