/w=3840,quality=90,fit=scale-down)
/w=3840,quality=90,fit=scale-down)
UX Audit: Unmoderated usability tests for an international bank
Executive summary
Although testers indicate they appreciate the information on the homepage, they mention that the header text placed over the image is hard to read. On top of that, testers indicate that the amount of reviews [Bank#1] has is low and that this makes them trust [Bank#1] less.
All testers are able to walk through the application procedure on the website and appreciate the use of the slide bar. However, testers do experience difficulties when filling out the form as they indicate the format is unclear and that the error messages are difficult to understand. Some testers mention this would be a reason to leave the website.
When seeing the website of [Bank#2], testers indicate they like the lay-out of the website. Most testers are familiar with [Bank#2] and notice it is part of BNP Paribas group, which helps to build trust.
When going through the application procedure of [Bank#2], testers mention they appreciate the icons of the different loan categories and the phone button. In addition to this, testers appreciate the progress bar and that the format of the field stays in place when filling out the details.
There was no preference for one of the websites as half of the testers preferred [Bank#1] and the other half preferred [Bank#2].
Introduction
To answer this question ten user test sessions were conducted between the period of May 1st and May 23rd 2023. Five tests were conducted on mobile and five on desktop.
Research Question
How do users experience [Bank#1]’s website and do they regard it as trustworthy?
How do users experience the application process for requesting a loan on the website?
How do users experience [Bank#2]’s website and do they regard it as trustworthy?
How do users experience the application process for requesting a loan on the website?
Methodology
The User Sense platform facilitates the execution of remote unmoderated usability testing (URUT). Key elements of this methodology are:
- The tester conducts the user test from his natural environment (e.g. his own home or office).
- There is no real time interaction between tester and moderator, which results in a minimization of potential biases.
- The tester performs tasks based on the test script and thinks out loud whilst doing so.
- The tester conducts the test via the User Sense platform and uses one of the applications on Android, IOS, Windows or Mac.
“A user interface is like a joke. If you have to explain it, it is not that good.” Martin Leblanc
“Your most unhappy customers are your greatest source of learning.” Bill Gates
“Usability is about people and how they understand and use things. Not technology.” Steve Krug
Ten testers have been selected to take part in this test. The criteria are presented below.
- Age: All testers are between 25 and 55 years of age.
- French speeking: All testers live in Belgium and speak French fluently.
Findings [Bank#1]
Homepage
How do users experience [Bank#1] website and do they regard it as trustworthy?
- Testers mention they appreciate the stucture of the website and mention it looks serious. The squared shape of the buttons make the website look professional.
- Testers appreciate the fact that the loan calculator has a prominent place on the website. They mention it is the most important feature and that it is what they are looking for.
- Testers appreciate to see the TrustPilot reviews, although they are expecting to see more reviews.
- Testers indicate that the text placed on the header image is hard to read due to low contrast between the text and the image. Testers mention that this makes them doubt the trustworthiness of the website.
50% to 75% of the testers:
- Testers expected to see more reviews on the TrustPilot section. They mention they find it suspicious that only one review is shown.
- Testers mention the first thing they see on the website is the cookie banner. Although testers try to reject the cookies, they are unable to do so. They say they do not understand the text in the cookie settings section and therefore accept the cookies reluctantly.
25% to 50% of the testers:
- Testers indicate that the red color is aggressive and assimilated with fighting and austerity. They mention it is not reassuring to them.
- Testers mention that the pictures do not evoke anything particular to them. They don’t recognize themself in the characters.
/w=3840,quality=90,fit=scale-down)
Session 8122“The website is clear and well presented, but the red color is very aggresive”.
Session 8182“It's good to have access to the calculator, it is the most important thing we are looking for.”
Request a loan
How do users experience the application process for requesting a loan on the website?
- All testers succeed in selecting the right type of loan for their request using the dropdown menu.
- Testers mention that the slidebar works intuitively and that they can easily select the amount of money they wish to borrow on the homepage.
- Testers indicate that the information is clear and that the tooltip is helpful. With the use of the tooltips testers are able to complete the flow.
- Testers mention that the information they are asked to share is in line with their expectations.
25% to 50% of the testers:
- Testers mention that they appreciate to see the details of their loan whilst going through the process.
- Testers encounter errors whilst filling out the form, especially for the date of birth and filling out the ID Card Number. They would prefer to see a fixed format in these fields to prevent them from making mistakes.
- Testers mention that they were expecting to first type in the name of the city they live in instead of the postcal code.
- Mobile testers have difficulty filling out the form as their keyboards is overlapping some fields. This makes it hard to see what they are typing and to use the dropdown menu.
50% to 75% of the testers:
- Testers mention they would like to see the amount of interest to be specified. This prevents them from having to calculate it themselves.
- When clicking on ‘Autres possibilités’, testers were surprised to see a pop up with the different variations of their loan. They indicate they find this feature helpful and mention it deserves to be highlighted on the page.
25% to 50% of the testers:
- When clicking on ‘plus d’infos’ testers expect to see the same loan value as they had previously selected. They mention they do not understand why the loan values are different.
- Testers mention they would expect the banner with ‘Borrowing money costs money’ to be placed on the website more prominently.
0% to 25% of the testers:
- Testers clicked on “Plus d’infos” and did not appreciate to be taken to another page. They do not understand why this is the case and ignore the content on the page.
- Testers mention that the text on the images is hard to read.
- Testers indicate they prefer to see the total duration of the loan expressed in years instead of months.
/w=3840,quality=90,fit=scale-down)
Session 8137“Oh no, I should not click on more information. I’m now on another page.”
Session 8115“I’m surprised to see other options. I though there was only 12, 36 or 42 months.”
/w=3840,quality=90,fit=scale-down)
Findings [Bank#2]
How do users experience [Bank#2] website and do they regard it as trustworthy?
Homepage
- Testers appreciate the lay-out of [Bank#2]’s website and mention that the design of the webpage is attractive, modern and welcoming.
25% to 50% of the testers:
- Testers appreciate the use of icons for the loan categories compared to the long list on [Bank#1] website.
- Testers saw that [Bank#2] is part of BNP Paribas, which they say helps to build trustworthiness for the brand.
- Testers notice there is a phone icon so they can easily call someone if they need help.
- Testers mention that the titles and text are easier to read when compared to [Bank#1].
50% to 75% of the testers:
- When using [Bank#2]’s tool, testers mention they preferred [Bank#1]’s slide bar over the ‘+’ and ‘-’ used by [Bank#2].
- Testers expected to see the loan simulator window when opening the homepage, but this was not the case.*
- Testers did not notice that they could change the cookie settings. They are upset about the fact that they have no choice but to accept the cookies.
* Caution: testers might be influenced by having seen [Bank#1]’s website first.
Session 8122“I don't feel like I'm on a website for a loan. It feels like I am on a gaming website, so it is positive.”
Session 5552“I notice the phone icon, and like the idea that I can immediately call someone.”
Request a loan
How do users experience the application process for requesting a loan on the website?
- Testers appreciate that the format of some fields are fixed while typing, as they indicate it prevents them from making errors. Testers mention they found this especially helpful for entering their date of birth and/or ID number.
- Testers mention they find the website easy to use.
- Testers appreciate the progress bar.
50% to 75% of the testers:
- Testers are unable to find the ‘renovation category’. On mobile devices, testers are unaware that the loans are presented in a caroussel. Therefore they select ‘personal’ instead of ‘renevation’.
- Testers struggle with how the different loans are presented and mention they had expected to see more options.
- Testers mention they are unclear on what the difference is between ‘classic’ and ‘sustainable’.
- When choosing the monthly payment, testers mention they expect to see the total amount of the loan including the interest. This is not the case on [Bank#2]'s website.
Less than 25% of the testers
- Testers are unsure on where the CTA button will bring them as they can also see the loan calculator on the page. They are unsure whether they should click on the button or stay on the page.
- When filling out their e-mail address, testers get an error message. Testers mention they do not understand the message and do not get why their email address would be incorrect.
- Testers expected to see suggestions when they type in the name of the city.
Session 8182“I don’t see the total amount of the loan, only how much I will pay back.”
Session 6939“I can’t find the difference between “classic and sustainable.”
Preference: [Bank#1] vs [Bank#2]
How does [Bank#1] compare to [Bank#2] and which website has users’ preference?
50% choose [Bank#1]
Top3:
- Simple, easy to use
- Clear information
- Loan calculator
“ Evreything is clear and well detailed, we don't even have to call anyone to have more informations. I like to have directly access to the simulator, it is what we are looking for.”
50% choose [Bank#2]
Top 3:
- Design
- Call button
- Trust mark
“It’s good to see the phone button. And the loan icons are great, it highlights the website” .”
Testers were asked this question after having completed the exact same tasks on both websites. This question has been asked to ten testers.
Usability metrics: SUS, NPS, SEQ
Usability metrics indicate how testers have perceived the website in terms of usability. They say something about this group of testers, but cannot be generalized due to the sample sizes of this study.
Mobile Usability metrics
System Usability Scale (SUS)
The System Usability Scale is a questionnaire designed to measure the usability of a product.
Net Promoter Score (NPS)
How likely are you to recommend this website to a friend or colleague?
Single Ease Question (SEQ)
How easy was it for testers to complete this task? (on a scale from 1 to 7)
Request a loan [Bank#1]: 5.8
Trustworthiness [Bank#1]: 4.2*
Request a loan [Bank#2]: 4.6
Trustworthiness [Bank#2]: 3.6*
*Trusworthiness is measured on a scale from 1 to 5
Dekstop Usability metrics
System Usability Scale (SUS)
The System Usability Scale is a questionnaire designed to measure the usability of a product.
Net Promoter Score (NPS)
How likely are you to recommend this website to a friend or colleague?
Single Ease Question (SEQ)
How easy was it for testers to complete this task? (on a scale from 1 to 7)
Request a loan [Bank#1]: 4.6
Trustworthiness [Bank#1]: 3.2*
Request a loan [Bank#2]: 5
Trustworthiness [Bank#2]: 4.8*
*Trusworthiness is measured on a scale from 1 to 5
Key Insights
What went well
- Testers mention they appreciate the stucture of the website and mention it looks serious. The squared shape of the buttons make the website look professional.
- Testers appreciate the fact that the loan calculator has a prominent place on the website. They mention it is the most important feature and that it is what they are looking for.
- Testers appreciate to see the TrustPilot reviews, although they are expecting to see more reviews.
[Bank#1] - Request a loan
- All testers succeed in selecting the right type of loan for their request using the dropdown menu.
- Testers mention that the slidebar works intuitively and that they can easily select the amount of money they wish to borrow on the homepage.
[Bank#2] - General impression
- Testers appreciate the lay-out of [Bank#2]’s website and mention that the design of the webpage is attractive, modern and welcoming.
- Testers appreciate the use of icons for the loan categories compared to the long list on [Bank#1] website.
- Testers saw that [Bank#2] is part of BNP Paribas, which they say helps to build trustworthiness for the brand.
[Bank#2] - Request a loan
- Testers appreciate that the format of some fields are fixed while typing. “it helps to prevent error. Testers mention they found this especially helpful for entering their date of birth and/or ID number.
- Testers mention they find the website easy to use.
- Testers appreciate the progress bar.
What can be done better
- Testers indicate that the text placed on the header image is hard to read due to low contrast between the text and the image. Testers mention that this makes them doubt the trustworthiness of the website.
- Testers expected to see more reviews on the TrustPilot section. They mention they find it suspicious that only one review is shown.
- Testers mention the first thing they see on the website is the cookie banner. Although testers try to reject the cookies, they are unable to do so. They say they do not understand the text in the cookie settings section and therefore accept the cookies reluctantly.
[Bank#1] - Request a loan
- Testers encounter errors whilst filling out the form, espcially for the date of birth and filling out the ID Card Number. They would prefer to see a fixed format in these fields to prevent them from making mistakes.
- Mobile testers have difficulty filling out the form as their keyboards is overlapping some fields. This makes it hard to see what they are typing and to use the dropdown menu.
- Testers mention they would like to see the amount of interest to be specified. This prevents them from having to calculate it themselves.
- When clicking on ‘Autres possibilités’, testers were surprised to see a pop up with the different variations of their loan. They indicate they find this feature helpful and mention it deserves to be highlighted on the page.
[Bank#2] - General impression
- When using [Bank#2]’s tool, testers mention they preferred [Bank#1]’s slide bar over the ‘+’ and ‘-’ used by [Bank#2].
- Testers did not notice that they could change the cookie settings. They are upset about the fact that they have no choice but to accept the cookies.
[Bank#2] - Request a loan
- Testers are unable to find the ‘renovation category’. On mobile devices, testers are unaware that the loans are presented in a caroussel. Therefore they select ‘personal’ instead of ‘renevation’.
- When choosing the monthly payment, testers mention they expect to see the total amount of the loan including the interest. This is not the case on [Bank#2]'s website.
- Testers are unsure on where the CTA button will bring them as they can also see the loan calculator on the page. They are unsure whether they should click on the button or stay on the page.
Conclusion
Based on the findings from this usability study, we conclude that testers are able to request a loan on the website of [Bank#1] but its user experience could be improved significantly by optimizing the website.
Although testers understand that [Bank#1] is a bank and can provide them with a loan, they mention they lack a feeling of trustworthiness when using the website. This is due to a lack of reviews on TrustPilot and the fact that some titles and texts are hard to read as they are place over an image where there is too little contrast between the text and the image.
This is different from what users experience on [Bank#2]’s website. Testers appreciate the layout and indicate that seeing that [Bank#2] is part of BNP Paribas helps to build trust.
All testers are able to request a loan with [Bank#1]. However, testers indicate that the error messages are unclear and that some fields should have a fixed format to prevent them from making mistakes in the first place. On mobile devices the keyboard sometimes overlapped the field or answers in the dropdown list, which had a negative impact on their experience on the website.
Similar to [Bank#1], all testers are able to go through the application procedure at [Bank#2]. Testers appreciate the way the loan categories are structured and the use of the progress bar. In addition to this, testers mention they like the fixed format of some fields as this prevents them from making mistakes.