Prototypes and usability testing.

The main aim of creating a prototype is to allow successful usability testing, through which important data can be gathered and problems with the product can be identified and removed, thus improving the product. The question then becomes: what should the nature of the prototype be? Naturally we want to constantly move towards a more sophisticated model, one that better represents an iteration of the final product, but most design processes cannot sustain (financially or time-wise) a constant stream of high fidelity prototypes. What ought we then consider when we think about usability testing without a high fidelity prototype, can such testing even be done? The literature says yes! Sauer et al presents us with a framework for conducting usability testing, and notes that the fidelity level of the prototype is not superior in importance to other aspects of usability testing (Sauer, J. Seibel, K. Ruttinger, B). The framework consists of four main areas: user characteristics, system prototype, testing environment and task scenarios. It is clear that that the system prototype is an important part of the equation however it is not the only one and the other three can be strengthened to compensate for a low-fidelity prototype.

Boothe et al provides us with an interesting study in relation to the effects of the fidelity of a prototype on the ability of the user to detect usability problems and the user's perception about the usability of the product. Boothe et al finds that the medium of a prototype does not in fact alter the ability of users to detect usability problems. The study also notes that the medium of the prototype only affects the ability to detect severe usability problems in particular scenarios. Finally the study finds that the medium does not greatly affect the users perception about the usability of the product (Boothe, C. Strawderman, L. Hosea, E.). Booth et al focus on the two mediums our group used as well, paper and computer, and as such these results are very useful to us.

In light of these two studies we decided to focus mainly on the appropriate user characteristics along with good task scenarios to allow us to get away using a simple paper prototype rather than having to make a high fidelity one as Booth et al seems to imply that the medium of the prototype is not paramount. We worked to find people close to our targeted user base, that is: close to our personas. The relative user competence was also gaged by having different variations of our of personas (CS student, Arts student etc) to test our product. We designed the task scenarios to have a narrow breadth, that is we expect the users to not use our product in an environment where the user has multiple tasks to consider, but rather can focus on operating our product. The depth of the task scenarios was comparatively quite large, as we wanted the user to try to exhaust all the options and features of our product. Sadly we were unable to use the proper testing environment since we didn't manage feel comfortable asking museum visitors to work with our prototype at the museum as to not make a disturbance (and our presence was already somewhat frowned upon by the staff).

Boothe, C. Strawderman, L. Hosea, E. (2013). The effects of prototype medium on usability testing. Applied Ergonomics. 44 (6), 1033-1038.

Sauer, J. Seibel, K. Ruttinger, B. (2010). The influence of user expertise and prototype fidelity in usability tests. Applied Ergonomics. 41 (1), 130-140.

Think-aloud

The think-aloud took place at a cafeteria with a liberal arts student (English literature). The person did not quite fit the persona:s we made since he was neither an avid museum visitor nor does he have the spare time both of our persona's have but he has a great interest in the arts and writes a lot so he is at least in the same ballpark as the “Jonas” persona. The first aspect the user noted was the overall concept, wondering if he would ever really use one of these interactive maps since he really didn't care about the number of people in the galleries.

After persuading him to take the product out on a test run anyways he started pressing the floor buttons. He commented on how it seemed odd to display the museum from a side view but noted that the symbols and the buttons were easy and intuitive to use. The reward system caused some problems as the user seemed dismissive towards the idea, but conceded that the system too was quite easy to understand. Further than this we didn't go, partly because of a lack of time and partly because our product really didn't have too many other functions than the dynamic map.

The user seemed to fare well in the use of the system, but the attitude towards the necessity of such a system, and some of the features (side-view and reward system), was less stellar. The main lessons from this exercise were 1) the overall layout is clear and intuitive and 2) some features are not exciting enough. Thus work continues on making the reward system and side-view better.

Iteration based on the personas and requirements

After working with our personas we turned to the literature to find out how we ought to use the personas in the design process. One major area of discussion for us was the area of requirements, in particular user requirements in relation to our newly created personas. The main user requirements we established for our product were: 1) The user should not require prompts and instructions to use the product and 2) The user should not have to spend a lot of time with the product. 1 and 2 are close related but still decisively different. We identified 1 as a requirement because the nature of the intended product: we want the product to be clear enough so that the user doesn't have to have any particular skills to use the product nor should the user have to spend time learning to use the product through prompts. 2 on the other hand deals with the fact that as the product is to be used while in a museum we don't want the use of the product to take up too much of the time the user could be spending admiring the artworks at the museum. Both 1 and 2 are in line with our personas, neither one has any particular interest in learning to use a new system and both personas have come to the museum to be calm and escape the stress of normal life, not to spend a long time operating our product.

Thus, based on the user requirements identified above, we iterated our design. The main change of the product was decreasing the amount of information on the map screen and making all the symbols as easy to recognize as possible. This made it so that the user only needed to take a short glance at the main map to see which halls were clogged with people and which halls were not, thus helping to balance the load of visitors in the museum.

As we were working with user requirements we also realized that due to the specific environment where the product was to be used (the museum) there were also some environmental requirements to consider. Chief among these was the fact that the product should not interfere with the audio and visual atmosphere of the museum. To do this we made the whole product completely quiet and reasoned that the panels ought only be installed in specific locations outside the galleries as to not disturb the visitors.

After exercise 1

Exercise 1, field studies and understanding the user, and the subsequent interviews.

In the first exercise we were introduced to the idea of field studies. We discussed with our group different methods of gaining an insight into the potential user-base of our product. We were instructed to do interviews and we set out to defining how we were to conduct these interviews and to what extent we should rely upon them. The most important consideration we tackled was that of establishing the goals of our interview. The literature tells is that rather than going in trying to figure out potential product ideas from open-ended questions to the public we ought to set clear goals, although we ought not be afraid to modify these goals if something new and exciting pops up. The goals we settled on were the following: identify when, why and how often the interviewee visits the chosen museum, and how the interviewee would characterize the nature of his visit. An added goal, and a more specific one relating to our product idea, was to get an understanding of how often the interviewee eats at the museum restaurant and to what extent the possible queues affect the interviewee's eating habits at the restaurant.

We also discussed how we ought to observe the potential users outside the formal interviews. We concluded that we ought to be as casual as possible as not to disturb the field environment and as such try to make a good “quick and dirty” observation. We chose to focus on an outsider observation since our skills as observers were not capable of conducting a participating observation and we didn't want to disturb the museum goers with such either. We decided to do a single interview each and then spend the rest of the time acting as normal museum visitors to observe the other museum visitors. The goals for the observations were to focus on how the visitors dealt with crowded spaces, in particular their emotional reactions, along with investigating the nature of the visitors that chose to eat at the restaurant.

Finally we decided on using audio recordings in addition to written notes. The Audio was used in the interviews while the written notes were used to catalog observations. We chose against using a camera due to the intrusive nature of such and the ban on filming inside the museum itself.

The Final Design

The Clickable Design
(Needs to be downloaded to be clickable)

Here follows the final designs most important pictures with discussion about it's clickable buttons

This is a sideview of the museum which shows all the exhibits present in the museum at once.
This was chosen as a starting screen which goes back to the choice of
museum which is going to be discussed in the design decisions blogpost.

The human symbols represent the amount of people in all the rooms. The meaning of the different ones will be discussed on the help page.

If you want to know something about a certain exhibit, you press on it,
and then you will get specific information about that exhibit

Left Floor Buttons: Sends you to the floor overview of the that floor, birdview.

Rooms: Sends you to specific information about the different exhibits.

Question Mark: Sends you to the help screen, showing the meaning of the symbols.

Refresh button: Updates the amount of people in the different rooms.

Pastry Button: Sends you to the Bull-point screen.

The symbol page gives a quick overview over the icons that are in the birdview: Stairs, elevator and emergency exit. It also shows what the different human symbols represent: Green is for few people, yellow for more and red is for many people in that room. This was, again, used on the sideview screen, the main screen, to show the amount of people in the different rooms.

Back Arrow: Back to the sideview screen.

Information screen about the different exhibits. As mentioned, the human represent the current amount of people in the room.

Back Arrow: Goes back to the sideview screen

Read More: Goes to the museums website to show more information about the exhibit.

Birdview of the differenet floors, this is the first floor. Shows helpful information for when you are at the museum, like where the stairs and elevators are. Note that the floors are color coded with the sideview so the same room always have the same color.

Sideview: Goes to the sideview, mainscreen.

Floorbuttons: Goes to other floors, cant click on the floor that you are currently at.

Icons up in right corner are the same as for the sideview.

An optional feature to use in the application is the feature of gathering
points from different exhibits around the museum. The points are
gathered by scanning QR codes at the exhibits and the fewer people that
are in your current room, the more points you get.

The idea goes back to that we want to balance out the museums visitors as
much as possible around the museum and with this way, some people might
be encouraged to go to places where it currently are less people, like
they might wait to go to the cafeteria if it’s a lot of people there.

If you gathered enough points you will be rewarded with a pastry in the cafeteria.

Some information about this can also be read on the app.

Our Product after the evaluation session.

Our product is a simple map application/interface with load-balancing properties. The map gives the visitors a clear, easy to use graphical interface which not only gives a simple representation of the current population levels in each of the galleries and in the restaurant, but also gives some information of each of the exhibits. The interface is essentially a top-down view of the museum floors, with each room containing a coloured symbol (pictured below) representing the current population status. This button is colour coded and changes depending on the population level in each gallery. The pictures below shows the different colours and what they represent.

The product comes in two forms. A larger map form, which can be displayed on panels in the museum, and a smartphone application. Another feature of the app is a collect-points-get-reward system where the user scans QR codes that are present in each exhibition. The codes are worth different amount of points depending on the current people load (e.g. a “green” room is worth the most and a “red” one the least) and when the goal is reached a coupon is earned which can be redeemed for a pastry in the cafeteria. This is probably not something for every app user but will hopefully promote migration to less populated exhibitions.

From the evaluation session we got a number of great suggestions for improvement ( these can be read about in an earlier blogpost). The suggestions were particularly related to the area of  buttons and availability. One of the main questions brought up was how the map was to be used if the user was colour blind. After reading on wikipedia we decided to keep colorblind people in mind when deciding the final color palette and layout but not to create an entirely separate “colorblind mode”.

Another upgrade we made was adding a statistic button (a very small one in the corner) which displays past information relating to the population levels in the current exhibits. This lets people see which exhibits have been very popular in the past, a piece of information some people might really enjoy. We want to keep the overall display very simple and clean, but we felt that adding a small button to the corner would not be too intrusive. A button we added to the smartphone version if the refresh button. This button refreshes the population indicators and gives the user a sense of tactile control over the application that an automatically updating solution wouldn’t provide.

The final upgrade that was proposed was a floor button which allows you to change between floors in a multiple story museum. This was a great suggestion since our group had completely forgot that many museums actually have multiple floors and having all the floors constantly shown on the screen would make the maps not only small but also leave the screens cluttered. This addition will no doubt make the map more legible and clearer to use.

All things considered the evaluation session was a great boon for our group. It gave us a chance to improve on key features and add vital upgrades that we might have missed had we not been able to have our product evaluated.

We noted this specific things in our design process.

We focused on one museum which meant that are designed was adjusted to fit that museum. Since our choice of museum was Fotografiska museet which has three different floors we thought an overlook using a sideview would be a good way to show the entire museum at once. If we would have chosen another museum, the option of only having the birdview, view from above, would have been more of a viable option, but in this scenario, sideview fitted us better.

We were thinking about adding more symbols to the prototype in the beginning, for example symbols representing if exhibits were new or are going to expire soon, but we skipped this in the final design to make the design look more clean and to avoid unnecessary clutter that do not help with are load balancing goal.

 

Reading Seminar 2 Conclusions

During this reading seminar we discussed the aspects of the evaluation process that are most relevant to our project. In particular we identified the following areas: Bias control, controlled testing, different evaluation methods and usability paradigms. In particular interest during the seminar was the area of bias control. To what extent ought we accept bias in the evaluation process? Can bias even be avoided?

As engineering students we are naturally attuned to abhor bias, it mucks up data and worsens not only our efficiency but also our productivity. However this was an eye-opening seminar for many of us as, in the field of  human-computer-interaction, avoiding bias, personal or user bias, is almost impossible. As such we had to start thinking about alternative models in which we not only acknowledge the bias but also work around it. In our case we are to let students from other groups evaluate our product and as such we are unable to get a truly unbiased evaluation (seeing as it is highly unlikely that our peers would speak their mind if they thought ill of our product) and thus our teacher assistant advised us not to waste our precious design-time worrying about bias but rather focus on making the prototypes better so that bias would be diminished.

Another area of discussion during the reading seminar was the extent to which already established frameworks could be applied to our evaluation process. As the efficiency and reliability of our evaluation process is paramount it seemed, to our group, that using a already widely tested evaluation framework would be highly fortuitous. Among the frameworks we investigated it was the DECIDE framework In particular that stood out accepted to our group as the best mode of evaluation and we aim to continue using this framework.

During the main introductory phase of gathering information about our target group we visited Fotografiska Muséet. The time of our field study happened to coincide with a rather idle period at the museum and therefore the number of visitors was quite low. This presented a problem for us since the idea for our product is mainly intended for periods of high pressure when there are queues and visitors might have a hard time finding a place to sit in the cafeteria for example.  Bearing this in mind, we’ve had to make sure that we kept the “intended” situation in mind when designing our prototype rather than the one we actually found ourselves in when conducting the research.