Evaluating Trust when Sending Money Online via Web vs. Mobile

Evaluating Trust when Sending Money Online via Web vs. Mobile

Abdullah Noman and Eduardo Ferreira Fontes Mercer
Tallinna Ülikool - 2014

Click on the title above to view/download the article in PDF format

Abstract. This research used self-assessments on trust and emotional reactions (hedonics) as well as pragmatic measurements to determine if the UX of two different access methods to the same money transfer service also affected the perception of trustworthiness of said service. It confirms Chu & Yuan’s (2013) observations on how interactivity can affect trust, but further observations of how trust will be affected by completing an actual money transfer instead of simply simulating a transfer order are recommended.

1.     Introduction

TransferWise is an Estonian-British start-up that intermediates currency conversions between peers. The users never interact directly with other users with whom they are converting their currency – price is set at the interbank rate at the moment of transaction and the user only enters with his money in the sending currency and receives the converted money in the other end of the transaction without having to deal with third parties. This approach eliminates many of the trust issues with converting money without involving a bank, retaining the economic advantages.

The service has been operating successfully for three years but only recently launched mobile applications – first for iOS and then for Android in early 2014. The main object of this study was to evaluate trust issues with both approaches to the same platform and, through comparison of pragmatic and hedonic measurements, try to determine if those can be attributed to user interface issues or stem from the different modes of access.

2.     The object of evaluation

TransferWise shifts all of the money-conversion operations to the web, eliminating the human factor in most (or all) steps but for the users themselves. This eliminates trust issues but also creates new ones – people are used to relying on banks for these operations and banks are typical last-century human-centered institutions. Eliminating the human operator on the other side can be jarring for first-time users and this can lead to mistrust.

According to Chu & Yuan (2013), perceived user-control, interactivity, responsiveness and connectedness affect trust and consumer behaviour online. The main object of this study was to determine if completing the same tasks on the same service using different access methods would lead to different levels of trust.

1        

2        

2.1       Design procedure

2.1.1    Procedure

Participants were invited to fill in a form asking for background information containing Yamagishi & Yamagishi’s General Trust Scale (GTS)-like questions. They then proceeded to visiting the company’s website and making a short heuristics evaluation, then completing a series of three tasks – send money to a saved recipient, locate a previous transaction and modify personal settings – using the TransferWise platform, both on the web browser and on a mobile phone. Video was captured of all on-screen interaction as well as from the participants themselves. After completing each task, participants were invited to select an emotion and intensity on the Geneva Emotion Wheel (GEW). At the end of all three tasks, participants took a post-mortem questionnaire containing more GTS-like questions to asses a shift in trust. A post-mortem interview was arranged individually to better understand emotional reactions and shifts in trust indicated.

Fig. 1. The Geneva Emotion Wheel

2.1.2    Apparatus and Materials

Two different access points were used, a common Windows PC running Google Chrome for web access and an Android phone running the native TransferWise application downloaded from the Play Store. A second mobile phone was used to film the interaction to an external memory card. Printed copies of the consent form, the background questionnaire, the heuristics evaluation form, three Geneva Emotion Wheels and the post-mortem questionnaire were provided to all participants.

2.1.3    Tools and Methods

Koyote Software’s Free Screen to Video was used in the Windows machines to capture on-screen interaction, while on the mobile phone, the native screen video capture capabilities of Android 4.4 KitKat was used for the same purpose. Yamagishi & Yamagishi’s GTS evaluation and Bänziger, Tanja, Véronique, Tran, and Scherer’s GEW evaluation methods were applied to the information provided by the participants. The post-mortem interviews were used to make sense of this hedonic layer of information.

 

Fig. 2. Screen Capture of Mobile App vs. Website.

2.1.4    Participants

Participants were nine people with almost-normal gender distribution (five men, four women) aged 21~35 years old. Participants were screened previously for reason to send money abroad – only those with family in other countries or other reasonable expectation to convert money to or from another currency in the next 12 months were invited to participate.

3.     Results and Discussion

In general, there was a shift in trust for the worst after using both web and mobile versions of the system, with a more pronounced negative shift in women and when using the mobile application. Post-mortem interviews revealed that the generalized shift in trust was mostly related to questions of whether the money would be delivered to the intended recipient, in the time frame promised and in the amount expected. This points to the need for a more detailed study with “live” money transfers that follows participants during the whole money transfer process, which can take several days.

GEW emotions also showed a negative shift when comparing web and mobile, with less-intense positive reactions and more frequent and more-intense negative reactions when using the mobile application. These were also reflected in the pragmatic evaluations, showing greater difficulty in completing tasks and, according to the post-mortem interviews, impaired by some user interface bugs and glitches. This perfectly reflects Chu & Yuan’s results in comparing E-Trust and interactivity.

3        

3.1       Recommendations

Pragmatic UX evaluation was invaluable in interpreting the hedonics results. Especially metrics like number of clicks and time to complete tasks, which explained shifts in trust related to poor user experience.

The GEW is a wonderful but confusing tool, both for participants and for evaluators afterwards. After the pilot test showed the Wheel itself had some poor UX aspects, it was decided that evaluators would recommend participants to pick a single emotion that better represented the task to be assessed and mark its intensity. This makes comparing emotional reactions difficult, because there are less overlapping data points between participants. Once again, the post-mortem interview allowed us to better categorize these reactions.

The main recommendation to TransferWise is to take Chu & Yuan’s conclusions in interactivity to heart and trying to make the mobile experience as complete and fluid as the web experience, this way saving the service from negative hedonic UX aspects.

It is also recommended to follow this study up with a more in depth one that follows the participants through the entire process of an actual transfer, as many of the participants questions on trust were left unanswered as this was a non-live test where no money transfer actually took place. A follow up study could also gather more statistically significant numbers, as the sample of participants for this study was rather limited.     

4.     Conclusion

The main conclusion we can take away from this study is that the same platform can produce different trust responses if accessed by different methods. In a user base that’s constantly shifting from web to mobile and back again, this is an important observation as designers must carefully craft the mobile experience to mitigate or eliminate perceptual differences that can lead to perceived untrustworthiness.  

5.     References

1.  Chu, Kuo-Ming, and Benjamin JC Yuan. "The Effects of Perceived Interactivity on E-Trust and E-Consumer Behaviors: The Application of Fuzzy Linguistic Scale." Journal of Electronic Commerce Research 14.1 (2013).

2.  Yamagishi, Toshio, and Midori Yamagishi. "Trust and commitment in the United States and Japan." Motivation and emotion 18.2 (1994): 129-166.

3.  Yamagishi, Toshio. "The provision of a sanctioning system as a public good."Journal of Personality and social Psychology 51.1 (1986): 110.

4.  Bänziger, Tanja, Véronique Tran, and Klaus R. Scherer. "The Geneva Emotion Wheel: A tool for the verbal report of emotional reactions." Poster presented at ISRE (2005).

Analysing Eye-tracking Data - Sopo using iTunes

Unfortunately, from all the captures made by our team, only one did not go awry (the capture on the other two froze around 8 seconds into the video. The CSV files are there, but without capture, there's very little to make sense of them).

0:14 - Exercise starts

 - There is some hesitation as to how you create a playlist. User settles for right-clicking on the sidebar to the left and creating one from the drop-down context menu. Task takes 12 seconds total.

0:26 - Playlist created

 - Once settled with the right click context menu methodology, user clicks on the tab called Songs after scanning the sidebar for options and a quick glance at the tab list on top. She then proceeds to right click songs, scroll down the very long context menu and add songs to her playlist 

0:42 - Two files added to playlist

 - Then proceeds a long hesitation where the user scans the song list for a long while, seemingly unaware of the next task or unable to find how to go back to the playlist. This ends at 0:57, a full 15 seconds later, when user finds and clicks the Playlists tab. She quickly drags the top song to the bottom of the playlist, completing the task.

0:59 - Switched order of playlist

 - Again the user's eyes wander around the two axes of the application - the tab list and the sidebar - for indication on how to delete the playlist. She finally settles for the comfort of the right-click and deletes the playlist after 10 seconds.

1:09 - Playlist deleted 

The video seems to indicate a lack of controls for specific functions. After scanning the chrome, the user always settles for right clicking items and scanning through a (sometimes very long) list of actions until she finds the one she's looking for. These added steps and lack of affordance seem widespread in the app. 

Reading Assignment 2: Affective Diary

Affective Diary is the process of collecting information through the use of a portable computer (such as a mobile phone) and an array of sensors (cameras, biometric sensors etc) in order to make a detailed registry of all daily activities of a study's participant.

It is intimately related to the phenomenon of the Quantified Self Movement, on which I'm writing my thesis, so I thought this might be a good method on which to base at least some of my autoethnographic exploration of the topic.

The Affective Diary is, in a sense, an evolution of the blogging movement, which itself is an evolution of the need to keep diaries. While a person writing a diary has to remember, select and write down the data they wish to keep (or share, when blogging), the Affective Diary collects all: passive video capture, mobile context (GPS and GSM location, nearby wireless networks), mobile activity (calls, messages, emails), biometrics (heart rate, galvanic skin response, temperature, motion) and active capture (conscious decisions to note down a certain passage or to capture a picture, etc). It should also allow the user to capture emotions, that the system can the correlate to the context: happiness, gloominess, even hunger.

 Just like a diary, the Affective Diary is a good tool for self-reflection, but just like the diary, it can suffer from the WORN effect - Write Once, Read Never. While a diary can show itself useless because the self-editing can lead to the information you're looking for in the future not being captured, the Affective Diary suffers from the same effect for the opposite reason: too much uncategorized information can lead to an unreadable and unprocessable mass of information. 

It is also intimately related to sousveillance, the inverse of surveillance (sous meaning under while sur means over in French), the act of capturing video and audio of your environment (as well as other pieces of information). Sousveillance was a term coined by Steve Mann, who also invented Natural User Interfaces, High Dynamic Range photography, scratch input and many other HCI strategies.

In a sense,  the Quantified Self Movement, Affective Diaries and Sousveillance are different ways to look at the same phenomenon for different reasons: you sousveil to fight surveillance or to collect evidence (as in alibi sousveillance, where you generate evidence that no wrongdoing has taken place, like when installing a camera to your dashboard); you quantify to know more about your health and to understand the effects of your physical activities on your own body and you make an affective diary to keep track of your activities.

The Affective Diary was, not so long ago, a pipe-dream, impracticable because of the high price of the equipment involved and the extra weight. Nowadays it's much easier to make a comprehensive log with the stuff we already carry around and new products such as the fitbit and Google Glass are only expanding the capacity to collect more and more information about yourself and the environment, 24/7.

What remains constant is the researcher's capacity to analyse that data. The risk of the WORN effect is very big, the mass of information too impenetrable to process completely. Better tools and analysis strategies must be developed to make sense of all the information gathered through such methods, running the risk of turning all that collection moot.

References

Mann S, (2014). "Veillance: Beyond Surveillance, Dataveillance, Uberveillance, and the Hypocrisy of One-Sided Watching", Chapter 2, pp 32-45 in "Uberveillance and the Social Implications of Microchip Implants: Emerging Technologies", Michael MG and Michael K, editors, Information Science Reference

Machajdik, J, Hanbury, A, Garz, a, Sablatnig, R, (2011) "Affective computing for wearable diary and lifelogging systems: An overview" in "Proceedings of the ÖAGM / AAPR Workshop", Graz, Austria.

Ståhl, A, Höök, K, (2008). "Reflecting on the design process of the Affective Diary". In "Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges" (NordiCHI '08). ACM, New York, NY, USA,

Evaluating the User Experience Reading 1: Virpi Roto’s User Experience Building Blocks

In this review, I analyse Virpi Roto’s User Experience Building Blocks, a paper written in 2006 on the need to better define User Experience as a means to be able to evaluate UX (as opposed to simply evaluate usability, as this is a limited approach that does not address user experience as a whole).

The text begins by exploring established definitions of User Experience, such as Norman’s and Jordan’s goals. By drawing parallels between those two and Nokia’s own definition, he comes up with certain levels of experience: behavioral (functionality, usability), visceral (pleasure) and reflective (pride). Personally, I feel quite vindicated because, as an Industrial Design student, I felt there was too much value being put on the reflective aspects of  modern design (image, beauty, the need to belong) as opposed to the Bauhaus ideals of functionality dictating design (which I also feel is a limited view of things). A tripod of behaviour, reflection and pleasure makes so much more sense to me as the domain where a balance must be stricken.

The author moves on to attempt to identify the building blocks of experience, all the components that influence the subjective feel that experience is. By exploring different authors, he lists components that exist in different levels of abstraction, such as Hassenzahl & Tractinsky user’s internal state (experiences and expectations), system and context and, in a much lower level, Arhippainen & Tähti’s user, social factors, cultural factors, context of use and product. From Forlizzi, he extracts the reference period, as UX can be analysed as a long, multi-temporal attitude which can, itself, be broken in much smaller perceptions and sensations.

Using Hassenzahl & Tractinsky high level of abstraction as a scaffolding, the author, through empirical evidence collected during his analysis of mobile browsers, derives his own building blocks of User Experience, as follows:

• System is everything with which the user interacts, products, objects, services, people and the infrastructure. It avoids the reductionist pitfall of equating usability with user experience by, instead of evaluating a single product, making it part of a whole that builds the experience, hence system.

• Context is built of external systems and objects that, although not part of the system, affect the User Experience. It begins in the physical context (humidity, temperature, luminosity etc.) to social context (expectations, culture), temporal context (period dedicated to the Experience) and task context (what part the system has in the greater whole of achieving a goal for the user).

• The User itself is the last block, as its expectations and personal experience (which affect and are affected by the social context) will affect the User Experience itself. Its mood has great effect on the final Experience as well, because s/he might have lowered patience (affecting the perceived temporal context). Last but not least, he moves away from the user’s internal state to list user’s physical factors, both temporary and permanent, as affecting the Experience (i.e. the user can have only one free hand at the time, or only possess one hand, or none).

Bibliography

• Arhippainen, L., Tähti, M. 2003, Empirical Evaluation of User Experience in Two Adaptive Mobile Application Prototypes. Proceedings of the 2nd International Conference on Mobile and Ubiquitous Multimedia, Norrköping, Sweden.

• Hassenzahl, M., Tractinsky, N. 2006, User Experience – a Research Agenda. Behaviour and Information Technology, Vol. 25, No. 2, March-April 2006, pp. 91-97

• Forlizzi, J., Ford, S. 2000, The Building Blocks of Experience: An Early Framework for Interaction Designers. Proceedings of Designing Interactive Systems (DIS 2000). New York City, USA

• Jordan, P.E. 2000, Designing Pleasurable Products. Taylor & Francis, USA.

• Norman, D. 2004, Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books.
• Roto, V. (2006, October). User experience building blocks. In proceedings of 2nd COST294-MAUSE Workshop–Towards a Unified View (Oslo, Norway, 2006) (pp. 124-128).