Use test (usability studies)

Social Inclusion and e-Health
1

Hello ! I would like to propose this topic so we can gather here the usability studies/reports about Taler.

Please feel free to share your reports of a use test that you have already conduct, or your ideas/proposals of framework for use-test procedure.

People from ESEN had already conduct some usability test in several events ( Testing Taler wallet in France for Usability studies ).

Gathering the studies/reports is important for Taler community (programmers, users, etc) to make Taler an accessible system and to reach our social goals in term of social-inclusion.

Even if you conduct a “single person usability test” (for instance your own use of Taler :slight_smile:) you could also share this valuable experience here!

2

Hello,
I am uploading here the usability study report summarising the four pilot studies carried out by E-Seniors in France to date.

E-Seniors remain available for any questions regarding the pilot and results.

TALER workshop - Système Usability Scale_ESEN.pdf (251.9 KB)

1 Like
3

Thank you @Olivier-R. Can you please explain what does this mean?

System Usability Scale result
=46,70

4

Sure.
Just a reminder: The SUS score is used to measure the perceived usability of a product or system by users/testers.
The closer the score is to 100, the more the system product is considered very easy and pleasant to use.

The project proposal indicates that the SUS of the system must be above 80.

The result of 46.70 is the overall score based on the responses of the 22 people who participated in the pilot with E-Seniors (participant profile: elderly people who struggle with technology and digital devices).

These pilots were conducted on previous versions of the TALER Wallet application.
I think it will be interesting to compare the results with:

  • New pilots with the latest versions of the application (to make a before/after comparison)/
  • Other types of audiences.
5

Thank you @Olivier-R and @how for sharing informations and discussions in this topic!

I’m glad that this topic is getting attention!

The system usability scale was developed by John Brook (1984), one can find here a retrospective by John Brook:

I think that the following paper is also useful:

lewis2018SystemUsabilityScale.pdf (1,5 Mo)

Even if the formula is simple (paragraph 1.3 of the above paper) it will be useful for the final report (and also for Taler community in ich.taler) to put some explanation about this system usability scale in the futur reports.

And yes I agree, it’s important to compare this usability scale with other sample/audiences : open invitation to motivated people to achieve this!

1 Like
6

Hello everyone,

With @Amandine (petites singularités) we conducted a use test, we acted as if we knew very few things about Taler (just that Taler Gnu exist as payment system-being built- and it’s privacy friendly).

We share with you our experience with all the details.

We start with a query (duckduck go) : “Taler Gnu”:

first result in the list: GNU Taler: Accueil

Then we followed the steps in

GNU Taler Demo ?

Step 1 : Install the Taler wallet
Ok

Step 2 : Withdraw coins
Ok

Remark:
Account ID for authentication <> Name of the account holder

use the bank’s Web interface to authorize the transfer of KUDOS to your wallet.

The meaning of currency conversion is not very clear.
The counterpart ‘exchange’ not clearly identified!

Step 3 :
pay a chapter of a book.

or donation:
“Thank you, Anonymous Donor, for donating KUDOS:1 to taler”.

Step 4: Check money flow

The step as explained is not clear, does not correspond to what is expected!

This step corresponds to what was already done in step 1.

First Conclusion:
The wallet is not really visible and the bank page lacks clarity, it takes time to understand what corresponds to what…
Navigation is not obvious, API to improve from our point of vue…

Suggestions:

  • Explain from the start the exchange account and the wallet, what they represent…
  • Have the possibility to view the wallet (its contents and operations in both directions) in the menu of the main page. Currently we only see the operations of the exchange account.
  • You should have a “push” notification when receiving or withdrawing in the wallet.
  • Improve the navigation in the bank page, for example when you select “welcome, account name” you arrive on a given page of the account, the return to the bank page is not easy.
  • To clarify: should we know everything about Taler before using the API or make the interface more obvious (more explanations)?

We also answered the questionnaire (explained in previous posts of this topics) used to compute the system usability scale

System Usability Scale
“Ranking”:
1: Strongly disagree
2: Disagree
3: Neutral
4: Agree
5: Strongly Agree

Assessments of usability: Sentence used

  1. I think that I would like to use this system frequently. 3 - 2
  2. I found the system unnecessarily complex. 3 - 2
  3. I thought the system was easy to use. 2 - 2
  4. I think that I would need the support of a technical person to be able to use this system. 2 - 2
  5. I found the various functions in this system were well integrated. 2 - 1
  6. I thought there was too much inconsistency in this system. 3 - 2
  7. I would imagine that most people would learn to use this system very quickly. 2 - 2
  8. I found the system very cumbersome to use. 3 - 2
  9. I felt very confident using the system. 2 - 2
  10. I needed to learn a lot of things before I could get going with this system. 4 - 4

System usability scale = 41,25 (just te compute the scale, “““sample””” of 2 persons :smiley:).

1 Like
7

Hello again,

I share with you the formula (ready to use) in libre Office Calc sheet :

System Usability Scale Formula.ods (35,2 Ko)

1 Like
8

Since I stumbled over usability deficits in Taler myself already, I really appreciate your work

1 Like
9

Thank you @htgoebel, please feel free to share your suggestions, remarks on the usability issues in this topic.

I hope that people from Taler System will participate to this topic and share with us their concerns about usability issues.