In Part 1 we evaluated three banking chatbots in the financial services sector: TD Bank’s Clari, RBC Bank’s Ask Nomi, and Scotiabank’s Chatbot. Using our Chatbot Analysis Framework, we reviewed these conversational AI solutions in terms of the user experience and the solution’s maturity and complexity and discussed the results. In Part 2, we’ll continue to share our insights and outcomes for three more banks and share valuable Use Cases of Conversational AI in Financial and Banking services.
Table of Contents
CIBC Bank Virtual Assistant
CIBC’s Banking Virtual Assistant is accessible on both web and mobile app. It is an authenticated FAQ bot that helps answer common questions, manage and pay bills, send e-Transfers, check your balance, and more.
CIBC`s Banking Virtual Assistant Strengths
- Friendly tone of voice
- Leverages APIs for personalization
- Includes auto-escalation
- Utilizes both app and web capabilities
The CIBC Virtual Assistant demonstrates a professional and friendly tone of voice that stays consistent throughout the chatbot experience. It also utilizes APIs to pull customer data and confirm account information, which adds a touch of personalization.
The bot allows for auto-escalation based on natural language which is extremely beneficial for customers who are stuck or simply want to talk to a live agent at any point in the conversation. In addition to live agent handover integration it offers additional support options if the customer is chatting after hours. This is a great element to have in your banking chatbot as it helps set expectations of when live agents are available and doesn’t block the user from getting the help they need.
The banking virtual assistant also leverages both app and web channel capabilities, including structured content used to confirm account information, message level feedback, and quick reply buttons.
CIBC`s Banking Virtual Assistant Opportunities
- Update welcome message to help set user expectations
- Optimize user feedback structure
- Improve dialogue quick replies to help move the conversation forward
When initiating a chat with the CIBC Banking Virtual Assistant, it introduces itself as a bot and then informs the customer on what it can help with and to select from popular topics. However, at the time we were reviewing the bot, it did not provide any popular topics as options. This could be confusing for the customer as it doesn’t set user expectations on what it can and can’t help with. Having quick reply options or a menu at the start of the conversation can really help reduce cognitive load for the customer. Unless the solution has minimal intents (i.e. 2-3), navigating via only NLU could cause more fallbacks and errors.
Another opportunity we identified while testing the virtual assistant is optimizing the user feedback structure. User feedback in the CIBC Banking Virtual Assistant is at the message level and the bot responds to the customer’s feedback with a system error message, which could be a bug. A best practice when implementing feedback is to use message level feedback for new flows or FAQs to understand whether or not the content is helpful to the customer. Once enough feedback is collected to inform the best customer experience, it can be removed later once the flow is improved.
Lastly, the CIBC Banking Virtual Assistant offers irrelevant options in certain dialogues such as small talk and disambiguation, which can halt the conversation and result in customer frustration. When a user derails the conversation, it’s a best practice to always steer it back on track to ensure we help the customer finish their task(s).
Our Framework Results: Good use of automation.
Start a discussion on improvements
Wealthsimple Virtual Assistant
Wealthsimple’s Banking Virtual Assistant exists on the web and app. It is a simple FAQ bot that answers customer questions and offers live chat assistance.
Wealthsimple`s Banking Virtual Assistant Strengths
- Sets some user expectations
- Offers auto-escalation
- Meets accessibility standards
While the bot does not introduce itself as a chatbot or virtual assistant, it mentions at the top of the UI that it is one. Once the chat is initiated the bot greets the customer and informs them of what types of topics it can handle.
This banking virtual assistant offers live agent escalation, even offering the customer to type “agent” at multiple points throughout the conversation in case they need more help. Additionally, when a customer chooses to chat with an agent it brings up a UI block that informs them how many people are ahead of them in the queue. This is an excellent feature to have as it once again sets user expectations and doesn’t leave them wondering when an agent will become available. The customer can even cancel waiting in the queue and return to the bot if needed.
Lastly, it meets the accessibility standards and uses different colors to clearly differentiate which text bubbles are the bot’s and which are the customer’s.
Wealthsimple`s Banking Virtual Assistant Opportunities
- Reinforce that the bot is a banking virtual assistant in the welcome message
- Include additional suggestions in sub flows to help set user expectations
- Expand to personalized, authenticated flows
- Tune the NLU model
As mentioned, the Wealthsimple Banking Virtual Assistant doesn’t mention it’s a chatbot. However it’s important to state this to let the customer know they are talking to a virtual assistant versus an actual live human agent. While the word “Virtual Assistant” is at the top of the UI in the web chatbot, not many customers may notice it as their attention will be immediately focused on the dialog boxes that pop up, so it’s important to reinforce this in the welcome message. After the welcome message, the bot presents a menu of topics, however, once a topic is selected it then asks the user to “type a specific question”. It would be beneficial to add some more suggestions here as the customer might be unsure what kind of “specific” question the bot will know the answer to, resulting in more escalations, fallbacks, or errors.
There’s also an opportunity to expand beyond simple FAQ flows. Customers can use the chatbot on the web without authentication, however they need to be authenticated when using the app and therefore are already logged in when using the chatbot. Leveraging Wealthsimple customer data can make for a more personalized experience and allow customers to make transactions like setting up auto deposits or transferring money.
Lastly, during our test we observed that the bot handles most NLU free form responses, but struggles to comprehend capitalization errors or minor spelling mistakes (i.e. etfs vs ETFs, etc.) Users are only human so spelling mistakes or case sensitivity are something that should be taken into consideration when building the NLP model.
Our Framework Results: Good use of automation.
Start a discussion on improvements
EQ Bank Virtual Assistant
The EQ Bank Virtual Assistant exists on the web channel and deflects to mobile web chat when accessing from the app. This chatbot can respond to customer questions about EQ bank, including general inquiries about products and services and basic FAQs.
EQ’s Banking Virtual Assistant Strengths
- Provides excellent prescription and sets user expectations
- Handles free-form NLU well and has auto-escalation
- Includes global commands
- Leverages app and web channel capabilities
The EQ Bank chatbot introduces itself as such, informs the customer on what it can do, and tells them to not share any sensitive or personal information. In the banking sector, this is an important piece of dialogue to mention and provides prescription on what the user should, or in this case shouldn’t do in the chat experience.
This banking virtual assistant appears to handle free form NLU responses well and has auto-escalation based on natural language such as “agent”. It also has global commands which are helpful for the user if they’re stuck or want to go back to the main menu.
The bot makes use of a variety of channel capabilities for both app and web. It utilizes carousels for certain menu options and includes both structured and quick reply buttons. Carousels are another great feature to have as they incorporate images, links, and call-to-actions to guide customers down the flow and further engage them by showing off products or services in a stimulating manner.
EQ`s Banking Virtual Assistant Opportunities
- Shorten responses to reduce cognitive load
- Improve authentication experience and utilize customer data to develop personalized & transactional experiences
One of the biggest opportunities we’ve identified for the EQ Bank Virtual Assistant is the amount of content it provides to the customer all at once. As pictured below, one FAQ response was given 8 text bubbles. These were all sent without pauses so the bubbles ended up filling the screen faster than the customer could read them causing them to scroll back up. As a conversation design best practice, it’s crucial that we make the content short and concise so users can easily digest it. If you can’t summarize the content into smaller dialogue (i.e. too important to cut out), then break it out into steps, allowing the customer to read the first step then selecting a button to continue to the next one.
While testing this virtual assistant, we also saw an opportunity for an improved authentication experience. Currently, customers who are authenticated within the app are still required to log into their account when chatting with the banking virtual assistant whether an answer to their question would require it or not. Furthermore, the chatbot provides the same content for both new and existing customers. If a customer is authenticated, leveraging customer data will allow you to personalize the experience and tailor it to them. Additionally, by utilizing user authentication and APIs, EQ Bank could turn its wayfinding chatbot into a more transactional one.
Our Framework Results: Good use of automation.
Start a discussion on improvements
Valuable Use Cases for Financial Services Chatbots
From reviewing these 6 bank’s virtual assistants, we recommend the following Conversational AI Use Cases for Banks sector that will speed up current web experiences, offer more value and utility and reduce human customer support.
Recommended use cases:
- Make a payment. Whether it’s a mortgage payment or bill payment, the “make a payment” use case is a high volume one that all customers can benefit from.
- Sending and requesting e-transfers. Another high volume use case, e-transfers is the most popular method of sending or requesting money in Canada.
- Completing bank applications. This could include having the banking virtual assistant allow the user to start or complete credit card applications, account applications, or loan applications where they can guide users through the process and depending on APIs available even approve the user all in the chat experience.
- Setting up or managing auto deposits. Everyone has recurring bills, rent, mortgages they pay each month. Having a chatbot to help set up and manage auto deposits can make this task easier.
- Fraud detection / reporting fraud. Being alerted when your card has been compromised or being able to report fraud will help ease customers’ stress.
- Finance insights. If applicable, leveraging a customer’s financial insights can help them break down their spending habits and budget accordingly (i.e. “How much did I spend on food this month?)
Conclusion
Using the framework, we found most of these banking virtual assistants shared similar strengths such as strong welcome messages, consistent personas, good use of channel capabilities, and meeting accessibility standards. In terms of opportunities, we found the bank virtual assistants to be lacking in transactional use cases, context and personalization, despite customers already being authenticated. One of the biggest opportunities we see for these financial virtual assistants is offering live agent handoff right in the chat experience instead of pushing customers to phone support.
Our Chatbot Analysis Framework is a great tool to help analyze your chatbot from an end-user perspective, ensure best practices are being followed, and identify opportunities for improvement and new features.
Ready to put your bot to the test? Download our Chatbot Analysis Framework here or get in touch with us to learn how you can optimize your virtual assistant to demonstrate a strong use of automation.
