Planning for failure

Updated: Jun 13, 2018

A deluge of telephone calls to the Knoxville Emergency Operations Center's recording system caused the loss of all recordings of state officials' emergency responses on the day of the 2016 Gatlinburg wildfire. Findings by a Nashville forensics firm hired to recover the lost calls and determine the cause of the system failure – one of which was system overload - did nothing to placate victims of the fire.

And when a catastrophic Vodafone technical fault shut down a raft of key phone services including police and NHS hotlines - and even RAC breakdown recovery, Barclays bank and First Great Western - callers were unable to reach the police non-emergency 101 number and NHS 24's medical advice line. Customers stranded at the roadside were unable to get through to emergency breakdown services and those with queries about trains were also stuck when trying to get through on the phone.

Statistics collated by conversocial show that when it comes to customer service, especially regarding the performance of IVRs in the commercial sphere, consumers vote with their keyboards. One survey found that 32 percent of customers found phone/ voice the “most frustrating customer channel.” But, inconvenience is nothing compared to potentially life-threatening scenarios as documented above. The bottom line: plan for failure.

Application Baselining

The good news is that these issues, both soft and technical, can to a large extent be mitigated by rigorous testing procedures of your IVR before you go live and every time changes or updates, however minor,are deployed.While the various types of standard (or what should be standard) IVR testing processes are well-documented,the value of dialog traversal testing is often underappreciated. In fact, it incorporates to a lesser or larger extent most sub-sets of traditional testing - from database testing to stress testing - by analyzing every path of a customer’s journey through your IVR. Most importantly, dialog traversal testing, together with load testing, enables you to create a baseline of your system’s performance and functionality so that you can constantly monitor, tune and update it; modern consumers have great expectations and little loyalty.

Dialog Functional Testing

Dialog refers to the various states in your IVR conversational flow – from expected responses to error prompts. Don’t expect customers to do what you expect them to do. Don’t assume you’re immune to cyber attack. Don’t leave utility outages for your partner and utility companies to deal with. Functional testing can be time-consuming but automated test sets that handle known conditions are multi-functional and can provide answers to questions like:

  • What are your system’s menu response latencies?

  • What percentage of customer responses are unrecognized (first time recognition rate)?

  • How many calls are incorrectly re-routed?

  • Are clients disconnecting within the IVR?

  • Can the IVR handle multiple web services backend calls and are the responses correct?

  • How many application warnings or errors are logged during a dialog?

  • How does the system stand up to unexpected call volumes?

A comprehensive automated IVR functional testing platform matrix can cover a lot more ground and save more project cost than you think.

Consistent functional behaviour: A full dialog traversal test enables you to combine feature testing with anomalous condition testing. For example, think of your IVR as a labyrinth and ask: what would happen if one of my customers decided to try and break through the hedge?

Customer experience: Similarly, usability testing, while not to be confused with quality assurance testing, enables you to assess whether the system’s design specifications correspond with real-world user requirements, like a consistent host connection and system load balancing.

Automation and capacity: Test sets for each potential path of a user’s journey through your IVR can be used to automate load, soak and stress testing. In addition, traversal testing enables the simultaneous testing of multiple vendor systems and databases.

Plan for Failure

Attributed to J.R.R. Tolkien, The Hobbit, is the quote: “It does not do to leave a live dragon out of your calculations, if you live near him.”When it comes to IVR, the live dragons are the customers who can’t get through to you or have a bad experience and report it on social and business media. They’re also the tragic victims of events like the Gatlinburg wildfire. When it comes to IVR testing, alwaysimagine the worst.The biggest question that’s left to be answered by organisations is….what is the real cost of failure?