CASE STUDY - AIRLINE MOBILE APP

NETAS | Istanbul

02 - 06 2014

OVERVIEW

Improving the design of an airline mobile application in terms of user experience builds a competitive advantage by improving customer satisfaction in their user experience. Even though each mobile operating system has specific UI guidelines and standards, creating a good mobile app in terms of usability, user experience, and human-centered design principles is a rather difficult task.

In 2014, I conducted a UX research for the mobile application of AtlastJet Airlines to identify the usability problems of the current mobile application.

I created my road map as follows:

Conducting a usability inspection (heuristic evaluation)
Choosing competitors
Designing user tasks and survey
Conducting a competitor benchmarking (competitive analysis)
Sharing findings with internal stakeholders

GOAL AND OBJECTIVE

The major challenge of this research was that it was difficult to evaluate the memorability measurement, one of the Shneiderman's usability attributes because the customers don't use airline applications frequently unless they want to schedule their flights. Thus there was no much learning factor in using an airline application. Hence, it was better to conduct a competitor benchmarking study to identify what was missing in the current application, if the usability was inferior compared to competitors’. In this way, I could have evaluated the competing applications, looking for relative strengths and weaknesses, trends, patterns and differences.

The objectives of this study were:

Identifying high-level usability problems of the application by conducting a heuristic evaluation
Conducting a competitor benchmarking based on the findings found by the heuristic evaluation
Revealing usability opportunities based on results found out by the competitor benchmarking

TEAM

Software Development Manager

Sales Manager

UX Designer

UX Research Consultant

Eylem Yilmaz: UX Research Consultant

Conducting a heuristic evaluation

Interviewing with internal stakeholders

Designing user tasks and survey

Conducting a competitor benchmarking

Coding and analysing the results

METHODOLOGY

In this study, I firstly conducted a heuristic evaluation to find out the major usability problems of the current app.

Heuristic Evaluation

For the heuristic evaluation of the application, I used Ben Shneiderman's heuristics - Eight golden rules of Interface Design:

Strive for Consistency: Consistent sequence of actions for similar situations, identical terminology (prompts, menus, help), consistent visual layout (fonts, colour etc.), standardizing the way information is conveyed ensures users are able to apply knowledge from one click to another.
Seek Universal Usability: Recognition of the needs of a diverse user group, design for plasticity (transformation of content)
Offer Informative Feedback: For every user action, the system should provide feedback. The user should know where they are at and what is going on at all times.
Design Dialogues to Yield Closure: Action sequences should have a beginning, middle and end. Feedback provides a sense of accomplishment.
Prevent Errors: Users should be provided with simple, intuitive instructions to solve the problem quickly. Spell-checker, choosing a date from a visual calendar instead of having them type it in or universal usability can help lower errors (large buttons or the contrast between text and background would help readability).
Permit Easy Reversal of Actions: Actions should be reversible as much as possible (trash can, saving history for undoing etc.), letting the user know that they can reverse an action.
Support Internal Locus of Control: Giving users the sense that they are in charge of the interface, designing interfaces which rapidly respond to the user, making user initiators rather than responders etc.
Reduce short-term memory load: Human attention is limited, therefore interfaces should be as simple as possible. Rule of thumb: Humans can remember 7 +/- 2 chunks of information.

Competitor benchmarking

The outputs of the heuristic evaluation formed the inputs of the competitor benchmarking. Based on the identified problems, I designed two user tasks and then compared the AtlasJet mobile app with two other competing airline apps to assess their success rates in performing these tasks.

In order to find out differences in performing the tasks, I conducted unmoderated remote user testing sessions. The performance variables were defined based on Shneiderman's usability attributes:

Time to learn
Speed of performance
Rate of errors
Retention over time
Subjective Satisfaction

HEURISTIC EVALUATION

Every tab on the left navigation bar was evaluated according to Shneiderman's 8 golden rules and the evaluation result was as follows:

1. Consistency

The visual layout (fonts, colours) and the standardisation of the prompts, menus were consistent. The user was able to learn from one menu to another one.

2. Seek Universal Usability

I observed a major difficulty in completing the 'finding a flight' task:

After the user made a search, to be able to make a new one, they had to achieve this task in 9 steps.

Click on “Change” button (circled in red)
Click on “from” for selecting departure location
Select “from” info
Click on “to” for selecting arrival location
Select “to” info
Click on “date”
Select a new date
Click on “OK”
Click “show result”

Evaluation 1: The system didn't keep the last research info after the user clicked on the Change button. They had to write the details again and this caused extra 6 steps in performing the task.

Evaluation 2: It was very likely that the user would select a new date which is close to the former one. Instead of having a “Change” button, the design should have been replaced with “Previous day/ Next day” buttons. In this way, the next day flights would have been just one click away.

3. Offer Informative Feedback

The system provided informative feedbacks after the user actions.

4. Design Dialogues to Yield Closure

The system provided accomplishments after the user finished a task such as buying a ticket.

5. Prevent Errors

I observed a major problem in handling errors during the filling ticket details task.

While the user filled the passenger details to buy a ticket, the gender field was displayed as Mr. by default (circled in red), which means If a woman user had forgotten to select the gender info, the system wouldn't have warned her that the gender field was mandatory, and then she would have bought a faulty ticket.

Evaluation: The user journey in 'buying a ticket' was open to occur errors as explained above.

6. Permit Easy Reversal of Actions

One of the major problems on the reversal of actions was the 'forgotten password' task.

When the user clicked on the “I forgot my password” link on the login page, they were redirected to another page to change their password. If the user entered a wrong username or email address on this page, the system was redirected them to the beginning - a login page, which means user had to perform all steps again. The user should have stayed on the same page and received a descriptive warning message telling the username or password they had entered didn't match.

7. Support Internal Locus of Control

The sense was mostly that the user was in charge except occurring some redirection problems during the error handling and reversal of actions.

8. Reduce Short-time Memory Load

The design was minimal and its simple steps were involved in consideration of short-time memory load.

USER TASKS

The heuristic evaluation showed that the current user journey of the design was required improvements. The main features of the application, 'finding a flight' and 'buying a ticket', had usability problems.

I created the following two user tasks to compare their user flows with the competitors'.

USER TASK 1

Finding a flight

USER TASK 2

Buying a ticket

Task 1: Find a flight and then repeat the task by changing the flight date.

Task 2: Find a flight and click on "buy" and then continue to the steps until the card details are asked for.

COMPETITOR BENCHMARKING

The competitor benchmarking I conducted had five main steps:

Identifying the goal
Choosing competitors
Usability inspection
User testing
Analysing results

1. Identifying the goals

Task 1 and Task 2 were designed based on the findings of the heuristic evaluation I conducted.

2. Choosing competitors

After the meetings with the project manager and the digital marketing specialist, we defined the competing airline companies like Pegasus and Turkish Airlines.

3. Usability inspection

A heuristic evaluation was performed to find out the major usability problems of the design.

4. User testing

User testing was performed with 15 real users via unmoderated remote sessions. Each participant completed two tasks for the three different applications. They were asked to save their screen via the software we had provided for them before the session. All sessions were recorded to observe participants' behaviour and motivation. At the end of the session, they were also asked to comment on the products used and on how they compared, what they liked, what they found confusing etc.

5. Analysing results

We created an affinity map to make sense of the findings of the sessions.

SURVEY DESIGN

The survey contained a questionnaire for the participants to fill out upon completion of each task of the usability test. After the completion of each task, the participants rated the ease of each task using a Likert scale (-2-1 0 1 2; where -2 corresponds to “very difficult”, 2 corresponds to “very easy” and 0 corresponds to “neutral”).

Additionally, the participants were asked two subjective evaluation questions to rate the usability of the mobile app on a Likert scale (-2-1 0 1 2; where -2 corresponds to “strongly disagree”, 2 corresponds to “strongly agree” and 0 corresponds to “neutral”) and were asked open-ended questions about their experience with the interface.

Google Docs was used to obtain the collected usability rating data of the mobile app.

Debriefing

The participants were asked open-ended questions about their experience with the website interfaces.

RECRUITING

15 real users were recruited for the study based on the target audience. The distribution of the participants with respect to gender and education level is shown as follows.

Participant internet use frequency data stratified by education level is depicted as follows.

It can be said that most of the participants were graduate people who mostly use the internet more than 1 hour up to 8 hours a day.

USER TESTING

User testing sessions were performed with 15 real users via one-to-one unmoderated remote sessions. Each participant completed the same two tasks for each mobile application.

The mean of the task completion times was as flows:

The mean of the first task completion time in AtlasJet application was nearly 30 sn. While the participants performed the 9-step user flow on the app, they completed the same task for Pegasus in 5-steps. This result also proved the heuristic evaluation findings as true: time to find a flight was too long.

Considering the overall usability attributes, the mean scores of the apps are as follows:

KEY INSIGHTS

Add buttons showing previous and next flights in order to reduce retention time in finding a flight

Keep username info in username field after an incorrect password attempt

Partition long data items such as credit card numbers into four parts to aid users in detecting entry errors, and reduce erroneous entries.

Mandatory input fields should be null by default to avoid faulty entries

Experienced users usually look first at the top item in a menu or list. So place Istanbul, which is most used, in the first row to save the user time.

Improve contrast font size, style, and colour: The colour should be darker and more apparent. Contrast should be increased.