My Design Process

Workflow

My process is mostly based on the Conversation Design Institute’s recommended workflow, with a little more emphasis on testing and iterating, as my experience has involved more analytics and UX work than the a designer role might entail:

Clearly define use case & problem we’re trying to solve. Work with product manager and AI trainer to find these use cases.

Research extensively & identify friction points in current processes. It’s important to learn from subject matter experts (especially phone agents) about the use case here.

Design time! Create design canvas to define what the bot can and can’t do with the help of engineering team.

Understand customers’ needs and state of mind during the conversation. (If we’re building a bot from scratch, this is when persona creation would happen).

Gather sample dialogues and/or prod utterances and design happy path flow.

Conduct wizard of oz testing or use a prototyping tool (like Voiceflow) to test with real customers or at the very least internally.

Send design to Conversational Copywriter for expert review.

Design long tail for repair flows, etc.

Test again if possible - if it’s a large enhancement and/or happy path changed significantly from 1st round of testing

Get stakeholder(s) sign off.

Meet with developers to review design if we haven’t done so already in canvas stage.

Analyze results after deployment and report any findings for our other application teams as part of a continuous feedback loop.

Where to start…

“Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should.” - Ian Malcolm (Jurassic Park)

The first thing I always think about when presented with a new channel or intent for the virtual assistant (VA) is whether or not it will actually solve a problem. Too often people believe the VA is a magic wand that will eliminate all phone calls or that we must have an intent for absolutely every question that has ever been asked. Virtual assistants are powerful, no doubt, but as with every product, we need to think about:

  • Value for the customer (Do customers want this and will it solve their problem?)

  • Usability (Is it intuitive and attractive to them?)

  • Feasibility (Can our engineers build it?)

  • Viability (Will the benefits outweigh the costs of building it?)

  • Brand (Can we afford not to have it for the sake of our integrity?)

The last thing we want to do is to negatively impact the accuracy of the NLU by introducing a very niche intent or spend months expanding VA to a new line of business/application just to discover customers only want to speak about something that they cannot do online in general. I love collaborating with product managers, AI trainers, and stakeholders to ensure we’re very intentional about what we develop for our customers. In this way, I help the organization save time and resources.

Analytics

Everyone agrees that making data-driven decisions is important, but chat data can have a steep learning curve. That’s where I come in. As the VA analytics lead for GEICO, I created dashboards and reports that helped management understand the VA KPI’s and conducted audits to identify intents’ containment percentages and CSAT scores. This analysis led to many enhancements and expansions for the VA, and to date, we achieved approximately 80% call deflection (no post-chat phone calls within 15 days).

Additionally, I participated in accuracy assessments for our NLU and by collaborating with our AI trainer to design new intents and restructure old ones, we increased our accuracy percentage by 15 points.

I’ve learned that one of the most important parts of VA data is getting your taxonomy right from the beginning. Labeling your intents and dialogs in a way analysts can interpret later on helps so much in identifying areas of opportunity. And then relentless iteration after launch is crucial. There’s no way to design and test for everything pre-launch, and it can be faster to just see how the VA behaves in the wild. But an organization must be 100% committed to updating the bot quickly when inevitable issues arise.