Conceptual Models II, Usability Inspection Methods

From CS160 Spring 2013
Jump to: navigation, search



Finish the reading you started for Monday:

Reading Responses

Alvin Yuan - 2/10/2013 0:08:04

When I use the Google Maps app, my goal is to find a set of directions to reach a desired location. Let's say my goal I want to find a way to get to Soda Hall from where I am currently only using public transit. The intention in this case is the following: a Start Point of My Location; a Destination Point of Soda Hall, Berkeley CA; a Transit Option of Public Transit Only; a Departure Time of Now. The action specification is: the first line is already filled in with My Location; press on the second line to bring up the keypad and type in “Soda Hall” and then press on the suggestion that pops up; press on the icon that shows a bus; press on the bottom where it says “Get Directions”; departure time is already set to the current time; press on the first row under “Select a trip”. I execute the actions and then the app outputs rows with the following information: two times, icons of buses and a walking man, and arrows pointing from left to right between the icons. Pressing on the first row brings up text regarding bus information and walking instructions and the destination I input at the end. I perceive the output and interpret the following: the times to be refer to the time to begin the trip and the time I reach the destination; the series of icons and arrows to be a summary of directions of the trip, where a bus icon means I bus and a subsequent walking man icon means I walk after bussing; the additional text regarding bus information and walking instructions as a detailed set of directions to get to the destination at the end of the output screen. I evaluate my interpretation of the output with respect to my goal and conclude that my goal has been achieved; the app has provided me a set of directions to get to Soda Hall using only public transit. My new goal is to find where the first bus stop is in the directions so that I may take the trip.

It seems that heuristic evaluation's strength is that it can provide detailed analysis of interface issues. As mentioned in the reading, an evaluator should note each distinct problem with the interface, even if they deal with the same user task or interface element. In user testing, the observer only can only see where the user struggles and not the exact problems themselves; the observer may not even infer that there are multiple problems. Evaluators can also point out more subtle issues or suggest improvements where there is no immediate problem, such as cosmetic fixes or having better consistency, things that users may glance over because it did not impede them. Finally, evaluators are much better at gauging severity. User testing is better at gauging typical user experience. This includes being able to get feedback from actual target users related to target user expectations (ex: an interface for doctors may be well designed in general but not what doctors expect). Another benefit of user testing is that the test users are not specialized in interacting with interfaces. Evaluators may have different expectations and thought processes when learning a new interface, one that is less realistic than that of test users.

Shaun Benjamin - 2/10/2013 21:58:41

Out of Milk is an app that creates a virtual shopping list and the user checks items off as they are placed in the cart. Initially my goals are to create a list of items that I am searching for in the store, and check them off as I shop. I can say the name of an item out loud, and the voice recognition software will attempt to interpret it. This is the intention and action specification stage. It's interpretation will be the input expression, and the execution will add the input expression to the list. The form of output expression will be the appearance of an additional item on the list, and I will interpret the item and evaluate whether the app correctly achieved my goal of adding a specific item to the list. My next goal will either be to add another item, or correct a mistake if there was an error in the Gulf of Execution.

Heuristic Evaluation would be good for deciding certain things that you know you want to the user to see, or certain features or actions that the user must have or expects to have available to them. It may not be as good as answering questions about where or how these features should be accessed. User testing will also answer questions about how intuitive an interface is, whereas heuristic evaluation may not be able to quantify that.

Alice Huynh - 2/11/2013 18:52:14

1) Look at Figure 6 on page 331 in the Direct Manipulation reading - this is a very important diagram. Give an example of you interacting with a mobile application of your choice and describe at least one full roundtrip through this action cycle when you work with that application. Start with your goals, and describe all steps until you end up at new goals.

I’m going to use “Fun Run” as an example because I just recently downloaded this game.


Here is the initial screen when I first downloaded the game. I notice that I need to make an account first so my goal is to create an account. Using that goal, there are 3 different options “log in”, “use facebook”, or “register”. I want to use Facebook because it’s faster than plugging in information to register so I press Facebook. The feedback from the application is that an external Facebook login window opens up and I’ve done this many times before with other applications so I do what comes natural and log in.

Now I am brought to this screen: IMG_0651.jpg

My new goal is to add my friend who told me about Fun Run. I notice a play button, a button that looks like a store and 3 buttons on the bottom left. The gear is very familiar to me because my android device has a “settings” button with that gear as well. The middle button with the “standings” pictures of first, second or third is probably not what I’m looking for. The last button is a picture of many people. I assume that this means interacting with other people so I click it, with hopes that I’ll be able to add friends.

I am then brought to a screen that allows me to add friends by username. I have finished my goal of adding a friend and now my goal is to play a game with them. (etc.)

2) What kind of questions do you think Heuristic Evaluation can answer that regular user testing or usability studies cannot? Vice versa, what questions cannot be answered by Heuristic Evaluation?

Heuristic Evaluation can pinpoint specific problems with a user interface that will cause a lot of users to make mistakes on. Most of the time, serious usability flaws aren’t obvious to the designer. User testing might warrant a lot of mistakes from users, but heuristic evaluation will help to pinpoint the exact usability principles that they violate. Heuristic Evaluation will not be able to tell the designers whether it is easy to perform a certain task because that is not the job of the evaluator.

Heuristic Evaluation will not help to figure out common errors that everyday users may come across. Evaluators are told to not dawdle on small issues, but to push through and find the usability issues. Evaluators who are not used to the technology of the system can gain help from an observer which is usually not advised for user testing. Evaluators will get a hint as to how to approach a system in order to reach their goal, but in user testing a user is left to struggle and hope that the system itself will lead the user to the original goal. User testing will help the designer to realize how easy or how difficult it is to perform tasks on their system. If a task that must be repeated many times is hard, then user testing can help the designer to realize where they should make changes.

Something that both can help the designer to answer is the severity of a usability problem. If a lot of users are making the same mistakes in the same places then it helps the designers to know the severity of the problem. Heuristic evaluation has these same methods to figure out the severity of a problem as well. Both user testing and heuristic evaluation can help to pinpoint the most dangerous usability issues.

David Seeto - 2/12/2013 2:02:03

Goal: Tune guitar. Intention: Use the tuner app to tell me if the E string is hitting the right pitch. Meaning of input: Detected sound means I want to measure the pitch of that sound. Action specification: Strum the E string to produce sound. Form of input: Sound waves picked up by mic. Execution Inter-Regerential I/O Perception Form of output: A letter, E. Interpretation: It is not the expected letter. Meaning of output: The note the pitch corresponded to. Evaluation: My E string is not tuned. New goal: Continue to tune guitar.

Heuristic Evaluation focuses on inspecting a UI based on a standard set of guidelines, and often times, it is the set of popular heuristics for UI design. It can answer questions like "is there flexibility provided for users of different mastery levels." This inspection sheds light on whether or not a UI is in line with generally accepted practices for UI design or other compliances expected of for your UI type. However, a UI might not need to tailor to different usage mastery levels or more generally, the standard guidelines are irrelevant to that UI. Instead, other means of evaluation like usability studies and provide data to base design decisions on. They help determine what is important, thereby creating a set of individualized guidelines for what your UI needs to accomplish.

Haotian Wang - 2/12/2013 13:26:32

1) I want to wake up at 9:00 am. This is my goal/intention. I know that I can set an alarm using buttons and scrolling-lists in my iphone's alarm clock app. The transition from knowing I can do this to actually figuring out the series of actions to take is the semantic distance. The taking of the actual actions, and my evaluation of how clearly the buttons are actually labeled/ how they look is the articulatory distance. After I input the last option and click ok, the phone immediately gives me an entry in the alarm clock at 9:00 am, and that entry has a big blue "ON" marker. My perception is simply the bar with the labels "9:00 am" and the big blue "ON" marker. My interpretation is that the alarm clock has been set to ring at 9:00 a.m. This is the articulatory distance for evaluation. My evaluation of the interpretation is that I will be able to wake up at 9:00 a.m., and my task is complete. This is the semantic distance of evaluation.

2) I feel like Heuristic Evaluation would be particularly good at evaluating the severity of tests, since it gives a defined and specific way of evaluating severity which the usability studies that I've had experience with doesn't have. Normal usability studies are not iterative (where the previous test results and given to the new evaluator in order for severity testing to happen), and so previous knowledge is not able to be used during the new tests.

Sihyun Park - 2/12/2013 19:09:48

1. Here is an example of one action cycle in an Android built-in application Google Now. The goal of the user when using Google now varies depending on the context, ranging from what's the weather to nearest restaurants and how to ask out a girl. In this example, I will set my goal as "finding out the nearest Chinese restaurants." Having set my goal, I move on to the execution, by pressing the home screen and pushing it upwards, launching Google Now. Among various input options, I use the voice command option, speaking "Where is the nearest Chinese restaurant." Google Now interprets the speech, and presents the user with a multiple options for Chinese restaurants nearby. I evaluate the results given, comparing them with what I had expected. My expectation was to find a nearby Chinese restaurant with a satisfactory rating, and I can find this information by looking at the cards presented. I choose the most suitable result in terms of both distance and ratings, which may not always be the first one on the list. Having satisfied my goals, I close the application until I have another question to ask.

2. Heuristic evaluation allows testers to answer questions related to specific design principles. For example, you might look for whether an interface has good visibility, by answering a set of question that ask the user to locate an interface element and executing the task. Or, you can conduct a similar test with a checklist for feedback. As such, heuristic evaluation allows the tester to evaluate the design principles employed in the product using a minimal amount of funds. However, heuristic evaluation cannot answer questions related to pure aesthetics or user experience. You can't answer whether a logo looks good, or if a layout is fancy. Similarly, heuristic evaluation is not an adequate measure of an actual user experience, as the results are likely to be biased by the tester.

Shujing Zhang - 2/12/2013 19:59:46

1) The application I use is called CamScanner. My goal is to scan the picture through camera and upload the picture to any cloud services such as Dropbox for archive.

i) Intention: Take a picture.) -> Action Specification: The action of pressing the “camera” button on the button of the application. -> Execution: the camera interface shows up and take a picture e -> Interpretation: The picture taken is show on the interface with the intended area that the user wants to archive. -> Evaluation: The output(meaning the picture taken) satisfies what user wants to scan.

ii) Intention: User wants to document the picture -> Action Specification: press the “check mark” -> Execution: the picture will be documented -> Interpretation: the picture is shown in the “new doc” section -> Evaluation: the output satisfies the intent that picture is archived in the local memory under “new doc” section.

iii) Intention: Choose the upload service. -> Action Specification: press the “upload” button -> Interpretation: Dropbox icon is shown. -> Evaluation: the intended cloud service is shown, satisfying the user’s intent.

iv) Intention: Upload to the Dropbox. -> Action Specification: press the file that you want to upload to, and press “Upload Here” -> Interpretation: the dialogue saying “upload completed” is shown in the next interface. ->Evaluation: the user finally uploaded the scanned picuture to the Dropbox.

2) i)

  • Heuristic evaluation can be conducted at very early usability engineering lifecycle. It can be performed even on paper. So it suits design that is at very early stage.
  • Heuristic evaluation is a very efficient usability method. It is inexpensive and very straight forward.
  • Heuristic evaluation can answer those questions that are very specific to the list of recognized usability principles (the heuristics).
  • Heuristic evaluation can even develop category-specific heuristics.


  • Heuristic evaluation cannot evaluate the usability when it comes to perform real tasks. It is not good at assessing the user experience while preforming real tasks.
  • In order for the heuristic evaluation to be efficient, evaluators must be equipped with expert knowledge to identify most of the usability issues.
  • The testing result may be biased because the evaluator is not 100% representative of the target users.

Zhaochen "JJ" Liu - 2/12/2013 20:25:57


Goal: Search for a best-rated Mexican restaurant near me using Yelp on my iPhone


  • Forming the intention: find a Mexican restaurant
  • Specifying the action: type in the search bar of Yelp to see a list of restaurants, the first one is the best-rated one
  • Execution the action: Given Yelp already opened, click on the search bar to see the cursor and the virtual keyboard, type the term ‘Mexican’, click search


  • Perceiving the state of the world: the Yelp app received the actions initiated by the users
  • Interpreting the state of the world: the application interpret the term ‘Mexican’, thinking the user wants to find a ‘Mexican restaurants’
  • Evaluating the outcome: the application represents the result to the user and the user come the expected outcome and the evaluation and found.

The user is satisfied with the evaluation and move on to a new goal.


What kind of questions do you think Heuristic Evaluations can answer that regular user testing and usability studies cannot?

  1. questions that will make the testing users feel embarrassed or uncomfortable
  2. questions that will lead the testing users to give inaccurate answers or make them feel reluctant to give out the truth
  3. questions that are specifically about the domain knowledge: these are perfect questions for the experts

Vice versa

  1. questions about the possibilities of users making mistakes
  2. questions that had caused disagreement among the experts should leave to the testing users
  3. whether the user like this idea of the new application or not. The experts cannot represent the general public.
  4. Some new products that are very never-seen-before: the experts may hardly have no experience in dealing with those products

Colin Chang - 2/12/2013 21:08:29

1) Look at Figure 6 on page 331 in the Direct Manipulation reading - this is a very important diagram. Give an example of you interacting with a mobile application of your choice and describe at least one full roundtrip through this action cycle when you work with that application. Start with your goals, and describe all steps until you end up at new goals.

An example that comes to mind is our current drawing app. If I have a goal of drawing a dot, I might press on the screen in the location I want the dot drawn, with an intention of drawing a dot there. The app will take some input from my gesture (the x, y coordinates of my press, perhaps the duration of my press and other information) and do some internal calculations. In order to supply me (the user) with sufficient feedback, the app will kick out some output in the form of a dot in the location I expect. The meaning of this dot to me will mean to me that my attempt at drawing the dot (and implicitly, registering it to the underlying system) has been successful. At this point, I'll likely form new (but similar) goals (drawing more dots, or even lines!)

2) What kind of questions do you think Heuristic Evaluation can answer that regular user testing or usability studies cannot? Vice versa, what questions cannot be answered by Heuristic Evaluation?

Heuristic Evaluation can offer general case usability solutions. In some sense, these may not be found in user testing (the user may not know that they prefer visible system statuses, since, perhaps, the ubiquity of its presence suggests that it obviously be implemented. So obvious that it never even crosses the user's mind). On the other end of the spectrum, there are specific user insights that cannot be obtained from the fundamentally general heuristics. For instance, the insight that many users of a measuing cup by using a 'fill and check' method is so specific and inspired that it would likely not come from a heuristic.

Lishan Zhang - 2/12/2013 21:16:43

1). The action cycle when I use Apple map to search for a route: ""Goal:"" Search for a CVS pharmacy in Berkeley and give me the guide to walk there.


""Intention to act:"" I want to buy some drugs in CVS.

""Sequence of actions:"" I search for the nearest CVS pharmacy around my current location and use the map to guide me there.

""Execution of actions:"" I click on the Apple map app, and type CVS in the search bar, and choose walk in the direction bottom.


""Perceiving the state of the world:"" I perceive what happened to the app and check if it is the result that I need.

""Interpreting the perception:"" I try to understand the map and determine where should I go.

""Evaluation of interpretations:"" Compare the result and what I my initial goal is.

The Apple map app successfully guide to the CVS pharmacy.

2).Heuristic Evaluation is used for finding usability problems from a small set of evaluators examine and judge the interface with the heuristics.

Heuristic Evaluation can answer:

  1. Some questions about common usability problems.
  1. Some questions related to evaluators’ domain knowledge.
  1. Some questions that make the evaluators feel uncomfortable.
  1. Some questions need to provide hints.
  1. Some questions need to be answered independently.

Heuristic Evaluation cannot answer:

  1. Some questions about the product rather than the user interface.
  1. Some questions about uncommon usability problems.
  1. The real problems that the user may encounter when using the interface.

Jeffery Butler - 2/12/2013 21:17:57

1) I decided to use my finger drawing app for this question. My obvious goal was to paint something on the screen. Therefore in the world of painting you touch a brush to a surface, so with the app the meaning of expression would be to paint something and the form of expression would be touching the screen with my finger. There's a gap between using an actual paint brush and using my finger especially since I haven't finger painted since high school... in addition I am not putting my fingers in paint for this app so I don’t know how much paint I have left to paint with. Fortunately, on the app its an unlimited amount. When I push my finger on the screen and move it, the interface keeps track of the points where I touched. From here the interface recognizes my touch and consequentially puts little circles in the spaces where I touched the screen demonstrating its interpretation of my intent. Thus, completely the cycle and moving on to my next goal which is how do I change the color?

2) By the Heuristic Evaluation involving more evaluators, more usability problems become prominent. Therefore, are there small corky bugs a first time user will run into? The Heuristic Evaluation also treats each individual evaluator as an equal, (I can definitely buy into this because for some reason my mom always finds problems with my apps than I do, even though I think I am better at evaluating apps than she is)... This observation answers the question: can a non-computer science major work this interface? Lastly, tThe Heuristic Evaluation does a great job of figuring out exactly how many users have to evaluate this interface in order to get thorough data. On the other hand, the Heuristic Evaluation gives the users hints when they use the interface. The user should be talking out loud when the observer is taking notes therefore, the Heuristic Evaluation would be able to answer the question: where in this interface do people cognitively get confused? Also, the Heuristic Evaluation has very little input on where the interface can be improved upon since they fail to debrief the evaluators after their use of the interface.

Winston Hsu - 2/12/2013 21:53:12

1.) I will use communicating with a friend via sending a text message on Google Voice as an example for explaining this cycle. Initially, my goal is to communicate to a friend. For the intention, I must realize that I need to start a conversation by sending a text message. The action specification is that I must first find the contact listing for my friend, and then click on the text message button to open the Google Voice App. From there I type my message and hit send. Once I have formulated this list of steps I then execute it. The app then has a status under my message which I can percieve. By reading the status, I can interpret whether the app is saying the message when through or if it fails. From there I can evaluate whether I need to retry to send the message or to wait for a response. Ideally my new goal would be to hear back from my friend.

2.) Hueristic Evaluation will be better at answering questions that users may not be aware are problems. For example, non-designer users may not notice inconsistencies in typography unless specifically asked to check for them. On the other hand, questions that the designer never thought to ask are more likely to be answered by regular user testing. For example a designer might not realize they are using labels or wording that is not within their user's vocabulary.

Tiffany Lee - 2/12/2013 22:18:14

1) Application: Spotify Mobile Goal: Search for a song. Intention: I want to tell the application the song I want to search for. Activity Specification: I see a menu bar on the bottom of the screen with a search tab on the bottom of the screen. Execution: I press that tab. Perception: I see that the screen has changed. There is now a text input bar on the top and 3 tabs below that: Tracks, Albums, and Artists. Interpretation: I understand that the text edit bar on the top is for typing in a search query. The tabs are to categorize the search results into Tracks, Albums, and Artists. Evaluation: I understand that in order to achieve my goal of finding a song, I must type in the song I want. I then should be able to find the song I want under the Tracks tab. New Goal: Listen to the song.

2) Heuristic Evaluation can answer questions that involve: the evaluator's direct opinion on the usability of the interface and the degree of usability based on the specified heuristics.

Heuristic Evaluation, however, cannot answer questions that involve mistakes the users make because sometimes the users aren't aware that they had made a mistake; neither can they answer some questions outside the given heuristics if the evaluators don't think of them themselves. They also can't answer questions that involve exactly how designers can fix the usability issues.

Andrew Gealy - 2/12/2013 22:18:59

When using snapchat, it is possible to draw on the picture you have taken before sending it. Once I have taken the picture, which is displayed on the screen, I have the goal of drawing on the screen. In order to do this, I must first first intend to enable this feature. My action specification specifies that I must tap on the pencil icon in the corner to enable drawing. I execute the tap, and the screen outputs an icon of different color and a rainbow bar below. I interpret this change to mean that I have activated the drawing feature. I then evaluate my interpretation with respect to my goal, confirm them to be the same, and continue with the process,

Heuristic Evaluation should be able to directly assess usability of an interface, where traditional user testing requires a second level of analysis and interpretation between the user's actions and comments and the usability problems themselves. Employing a small group of expert evaluators seems more likely to cover a broad range of problems than a similar number of normal users. The evaluators will be thinking about the problems of an entire user base, where layman users can only really present their own experiences and issues. Heuristic Evaluation cannot directly answer questions about how layman users will interact with the system in question. They may be able to make predictions, but by nature they possess an expertise that will not allow for the observable user problems that traditional user testing seeks. They may not be able to perfectly emulate the needs or actions of disabled users, for example.

Joyce Liu - 2/12/2013 22:23:09

1) My goal is to listen to a song that I enjoy on my Pandora radio station. I start my opening up the application via clicking the app’s launcher icon, which leads me to the most recently played Pandora radio station. I don’t like the current song that’s playing, so I click on the ‘next’ button to skip to the next song. The next song isn’t that great, either, so I click on the ‘next’ button again. The song that’s playing now is pretty good, and since I haven’t up-voted this song yet, I click on the thumbs-up icon to upvote it. Clicking on the thumbs-up icon is the form of input expression. The form of output expression is depicted by a little thumbs-up icon that now appears next to the album cover on the top right of the screen. The meaning of this output expression is that I enjoy this song. My new goal is to listen to more songs that I enjoy.

2) In Heuristic Evaluation, the observer only needs to record the evaluator’s comments, whereas in user testing the observer also needs to interpret the actions of the user. Heuristic Evaluation helps to find usability problems. Heuristic Evaluation can answer questions about design, but it cannot answer questions about user needs or usage patterns. Heuristic Evaluation cannot answer questions about mistakes that users make when using the interface. Heuristic Evaluators can more readily give advice on design because they are informed on design and usability principles.

Cory Chen - 2/12/2013 22:39:18

1. A pretty simple example is putting an entry into the calendar application. The goal is to write down my appointment and remind myself about it later. From there, I have to perform the action to reach that goal, which is opening up the app, typing in the name and time of my event, selecting done. The phone then accepts that and outputs the full calendar screen again to show that the "adding event" action is finished. I interpret this output and realize that my goal has been reached, and then I am free to go to the evaluation stage, where I affirm that I am done and start thinking about what I will do next.

2. Heuristic Evaluation is good for answering a number of different questions. The main advantage I see is that you are able to find usability issues that do not impair the use of an application in significant ways. During regular testing, the impairing issues are more obvious while the secondary issues are harder to find in comparison. By fixing these side issues, you are able to avoid usability issues that may grow in the future. Heuristic evaluation also has the advantage of using people who are used to evaluating systems for their usability, giving you the advantage of looking at things from a skilled person's point of view instead of from a layperson's view. From this, you can find design flaws very easily.

Heuristic evaluation's downsides come from the fact that it requires people unfamiliar with the system to evaluate it. When you do heuristic evaluation and guide the evaluator through the process of using your application, there is a chance of missing the strange workaround habits that normal users often adopt when using a system low in usability. By missing out on these workarounds, you not only lose information about what needs fixing, you also lose insight on why the alternate method seems to be more intuitive. The experience of using a system over an extended period of time is also not answerable from using heuristic evaluations

Bryan Pine - 2/12/2013 23:21:17

1) I'll describe the interaction with the "maps" app (Google or Apple variety) that is on almost every phone. I start with the goal of getting from where I am to someplace else. First, I need to switch from the "search" view to the "directions" view. There is a button on the bottom labeled "Directions", which is helpful because the function that button performs is semantically labeled just as it is in my thoughts. This action is also articulately direct because the buttons for search and direction are right next to each other and look as though they are the same button, which gives the feel of actually moving the view to the directions screen. Once I am on the directions input screen, the system clearly accepts input in the form of addresses. It nicely grabs my current location for me (handy for most searches, but potentially confusing if I want directions from a different starting point). This input form definitely does not exhibit articulatory directness because there is no clear mapping between the line of text and a physical location. However, the system is semantically direct because location is often described and thought about in terms of addresses. Once I have inputted everything, I click "Route" and it displays my route on a map. As an initial feedback mechanism, this is pretty good at bridging the gulf of evaluation. I can see my start and end points displayed on the map and make sure that they are at least generally where they ought to be, and I can drag the route around by touch to change it. The system also allows me to change between output styles for different purposes (sometimes I want to see a list of what turns to make rather than the general outline of the path). Once I have this task completed, I turn my attention to driving and only refer back to the map occasionally. Eventually, the process starts over when I want directions to somewhere else.

2) Heuristic Evaluation seems helpful for identifying more subtle problems that the user might not otherwise be able to articulate. For example, a user in a user test situation might find an interface uncomfortable or confusing, but be unable to say why. However, after looking through a list of heuristics and thinking about each of them, the user may realize that the difficulty is the result of a poor color scheme choice. In other words, heuristic evaluation can give reliable information about why an aspect of an interface is problematic and what might be done to correct it, while user testing is more suited to simply identifying problems. However, heuristic evaluation seems limited by the list of heuristics; you won't get insights about things that you haven't already identified as potential problems. Part of the beauty of user testing is that you can discover the "unknown unknowns" by watching users interact with the interface with little to no guidance. Heuristic evaluation requires the tester provide more guidance, so it is less useful for figuring out how users will react to the interface in real-world situations.

Claire Tuna - 2/13/2013 1:42:36

1) Look at Figure 6 on page 331 in the Direct Manipulation reading - this is a very important diagram. Give an example of you interacting with a mobile application of your choice and describe at least one full roundtrip through this action cycle when you work with that application. Start with your goals, and describe all steps until you end up at new goals.

When I use Lockitron, my goal is to lock or unlock the front door. There is a button that looks like an undone lock and a button that looks like a locked lock. The goal, “unlock”, the meaning of the expression, “unlock”, and the form of the expression, the picture of the lock being either locked/unlocked, are all very similar and thus, the gulf of execution is small, making the application feel like magic.

The semantic gulf is minimal, because the goal (lock/unlock) maps directly to the meaning of all possible forms of expression (lock/unlock). The command, pressing the button, also has a minimal articulatory distance, because the button with the undone lock clearly maps to the unlock action. The output, which also spans the articulatory distance, is a green highlight around either the locked or unlocked door, meaning the action has been taken. If I am not in front of the door, the articulatory distance is small but present, because I have to interpret that green outline as meaning “the door was indeed unlocked”. The output, if I am in front of the door, also includes the sound of it unlocking or locking, which directly spans the articulatory and semantic gap by clearly achieving the goal.

In a recent version of Lockitron, the application syncs with the user’s bluetooth and unlocks the door automatically as the user approaches, completely demolishing the gulf of evaluation. Magic.

What kind of questions do you think Heuristic Evaluation can answer that regular user testing or usability studies cannot? Vice versa, what questions cannot be answered by Heuristic Evaluation?

I think heuristic evaluation would provide designers with a more organizable set of data, clearly divided into the chosen heuristic categories. The heuristic evaluation would directly answer the question “How good was our error prevention?”, whereas normal user testing would require synthesis/analysis to arrive at that answer. However, I think the user testing with mental walkthrough provides insight into the user’s mental model in a way that heuristic feedback would not. “How does the user think the fridge works?” would probably not be answered through a heuristic survey. Also, questions like “how long did it take the user to complete the task from front to end” and “where did spend the most time” are not captured by the survey the way they would be in regular user testing.

Minhaj Khan - 2/13/2013 2:08:55

ive opened the facebook messenger app on my phone and my goal is to identify the person A i'de like to message and click on their name. to begin the cycle, after initiating the app the screen displays a list of names of facebook messenger users who i can message. this is the form of output expression, which i perceive and interpret to be this list of names. i evaluate the meaning of this output expression to mean that the screen is ready for the execution of my goal. my intention is to then click on person A's name on the screen (the meaning of input expression), and i identify the action to be a tap on the screen where the user's name is located (the form of input expression). having tapped the screen i have executed the action, which then completes the cycle as the app then gives me new output in the form of my chat log with person A, my new goal then being to type in a message and send it.

Heuristic evaluation can answer questions that regular user testing cannot because the heuristics are specific usability principles (such as appropriate meshing of font colors) which are in some way quantifiable or at least discernible between bad and good designs by evaluators who are adept at applying such heuristic principles to interface design. this can be seen as a kind of checklist of design principles which are tested by evaluators for compliance, whereas such expertise or specific design principles may not be pointed out by regular user testing, even if the user experiences any negative effects of bad design principles. some questions that cannot be answered by heuristic evaluation are those that involve not specific design principles but intuitive usability that cannot be easily quanitified or labeled as bad or good. in this perspective, regular user testing will be superior because users will be able to point out discomfort or unintuitive designs which cannot be categorized by any specific heuristic as it may be a subtle or subjective detail.

Brian L. Chang - 2/13/2013 2:20:51

When I use Flipboard my goal is to be more informed of interesting news. I intend to find a good article so I open the cover articles and Flipboard shows me the top article. I look at the headline, think if I actually want to read the article and then decide if I want to move on to the next article or read the entire article. My next goal then becomes finding more articles or finish reading the current article.

Heuristic Evaluation gives the evaluators a better guideline for testing as opposed to regular user testing. The evaluators are given heuristics in which to evaluate the product and based on these the observers can find out which principles were violated and are able to fix these areas. Heuristic Evaluation cannot give a systematic way of fixing these problems and does not give the probable success of a redesign. Heuristic Evaluation also does not find every usability problem and cannot guarantee a perfect design.

Ben Dong - 2/13/2013 4:18:27

If I want to walk over to a friend's place, my intention is to look up directions on how to get there. In order to do so, I take the action clicking on my maps icon, and search for the address. Once I execute the search, I can perceive the returned results. I interpret the location and path provided, and evaluate how to best get there. If I decide to go somewhere else or stop somewhere along the way, I repeat this cycle again.

Heuristic evaluation can be used early on in the development cycle before the finished product is ready, since they don't actually require the evaluators to use the product as would be necessary in user testing. They can also expose flaws more related to general design principles, since the evaluators should be experts instead of regular users. However, regular user testing is likely to catch more potential problems and is better for determining what issues most users will run into, especially if performed on the target group in question.

Weishu Xu - 2/13/2013 7:42:55

1) When I use Google Maps on my phone, the goal I have in mind is to reach an end destination. First I have to translate that goal into an address that can be inputted into the application, and the machine executes and returns a number of outputs: an estimated time, distance, several possible options with regard to transportation, and a plotted out line on a map describing the path from my current location to the destination. At this point, possible inter-referential expressions include the estimated time and distance for the various options of transportation that I can then re-input in order to find the path either by walking, driving, or using public transportation. When I perceive the map and path, I have to interpret that as the course that I would take in real life, and the application helps me by also offering a step-by-step instruction guide on how to reach the destination. In order to evaluate how I would like to get to that place, I would have to look at the different steps and decide if they are convenient or inconvenient (i.e. short drive, toll roads, highways, etc).

2) Heuristic Evaluation is focused on identifying certain intuitions or habits that a user expects when using an application. It also emphasizes having the user analyze and evaluate what is good or bad about an interface. On the other hand, traditional user testing is focused more on identifying certain problems that exist in the current user interface based on the users responses. It is up to the tester to analyze the user's experience and to identify ways to improve a certain design. Heuristic evaluation is focused on getting the users perception of what would be an intuitive model, whereas user testing is focused on understanding what is not appropriate in the current model.

Eun Sun (Annie) Shin - 2/13/2013 8:28:56

1) I will use the iPhone's mail application as an example. One of my usual main goals is to check and look through recent mail. If I have the intention to scroll down, then I will put my finger near the bottom of my screen and move up to move up the email I already saw. Then the app will perceive my executed action and interpret it to determine the meaning of the input expression to get the output expression. Then the application will realize my intended action and will thus scroll down my inbox, allowing me to see more mail. My new goal now may be to open or delete one of the new mails that I ran into upon scrolling down. Then the cycle repeats but with a different goal (of opening or deleting) and motion.

2) Heuristic Evaluation will answer specific, detailed questions that traditional user testing may never answer because the testers have freedom to explore and do whatever they want. The freedom that users have in regular user testing make possible for them to go on tangents or focus too specifically on one aspect of the design. The heuristic evaluation, however, will not be able to answer what the user struggles with the most because users are able to communicate with the observers and ask questions for immediate answers. In a real setting, users often have to look for tutorials or answers online, via phone call to customer service, and so on. Heuristic evaluation will not make clear what features are not naturally intuitive.

Soo Hyoung Cheong - 2/13/2013 8:30:19

1)A mobile application I chose to interact with will be Facebook app. When I arrive at a special location, I want to use the Facebook app with the intention of doing a check-in for me and my friends. I see the “Check-In” button on the screen at the top, which I know will take me to check-in screen and executes the action by clicking that button. Then it knows that I want to get to check-in screen and after it senses my click, it interprets my action and evaluates it into request to go to the check-in screen to reach the first goal. Then with another intention of posting this check-in on Facebook, I click “Post” to execute the action. Then the app immediate takes that request and interprets that I want to post the check-in on Facebook, therefore it evaluates it by doing the posting and showing an updated newsfeed with my check-in post at the top to arrive at the final goal.

2)Questions that Heuristic Evaluation can answer that regular user testing or usability studies cannot are studying tasks that are not yet implemented. Since the Heuristic evaluation can study the process that this simply hypothetical and not physically available, you can go ahead of the functionality of the system to prevent problems, rather than build a system and then trying fix the problems. Also Heuristic Evaluation addresses more of the general design problem, rather than just individual errors/problems .

Questions that cannot be answered by Heuristic Evaluation and can only be answered by regular user testing are whether the design actually cause problem for the user. There can be certain issues that you would hypothetically think a person will encounter, according to Heuristic Evaluation, but may actually not be a problem. Also there are problem that you may have missed, but can be found through the actual performing of tasks, which allows quick discovery of existence of errors.

Aarthi Ravi - 2/13/2013 8:37:12

Watsapp is the mobile application I have chosen for describing the activities of the cycle. It is a chat application that helps you chat with friends on your contact list through the web. My goal is to chat with XYZ. Intention is to first find XYZ. The action required is to search for the person. The interface provides a search box to perform the search and thus I type a person's name and execute the search. Search box has become a habitual input thus giving it a feel of directness with less semantic distance . The system checks for the person in a database. In this case the articulatory distance is minimized as the input can be directly used as a parameter to search and there is no need to manipulate the input. The system perceives that the contact is Not Found. The application interprets this message and concludes that the user might want to add this contact as the contact is not found. Here the semantic distance is large as the app has to conclude what the user wants to actually see. The application displays that contact not found and gives you an option to add the person by entering XYZ's phone number. I evaluate the app to have given me the right output and reduced my cognitive action for the next step as I would have wasted time finding out how to add a new contact.

Heuristic evaluation tells you if the elements of your interface adhere to some usability principles which user testing and usability testing can't reveal. It explicitly states the exact problem with the element,like, the visibility of the feature is poor etc. But through user testing one may not be able determine the exact reason why a user is making a mistake. Heuristic evaluation does not tell you the mistakes the user could make while using the interface as the evaluator is normally told how to navigate or is given a use case. Heuristic evaluation does not provide a way to fix an issue but sometimes through observing users one may be able to come up with fixes.

Aarthi Ravi - 2/13/2013 8:37:25

Watsapp is the mobile application I have chosen for describing the activities of the cycle. It is a chat application that helps you chat with friends on your contact list through the web. My goal is to chat with XYZ. Intention is to first find XYZ. The action required is to search for the person. The interface provides a search box to perform the search and thus I type a person's name and execute the search. Search box has become a habitual input thus giving it a feel of directness with less semantic distance . The system checks for the person in a database. In this case the articulatory distance is minimized as the input can be directly used as a parameter to search and there is no need to manipulate the input. The system perceives that the contact is Not Found. The application interprets this message and concludes that the user might want to add this contact as the contact is not found. Here the semantic distance is large as the app has to conclude what the user wants to actually see. The application displays that contact not found and gives you an option to add the person by entering XYZ's phone number. I evaluate the app to have given me the right output and reduced my cognitive action for the next step as I would have wasted time finding out how to add a new contact.

Heuristic evaluation tells you if the elements of your interface adhere to some usability principles which user testing and usability testing can't reveal. It explicitly states the exact problem with the element,like, the visibility of the feature is poor etc. But through user testing one may not be able determine the exact reason why a user is making a mistake. Heuristic evaluation does not tell you the mistakes the user could make while using the interface as the evaluator is normally told how to navigate or is given a use case. Heuristic evaluation does not provide a way to fix an issue but sometimes through observing users one may be able to come up with fixes.

Glenn Sugden - 2/13/2013 8:38:49

1 - Goal: To ask Siri a question on the iPhone. Intention: To activate a mode on the iPhone that will respond to a voice command. Action Specification: I need to (quickly) lift the iPhone to my ear, changing its orientation from horizontal to vertical (a natural movement that is used when initiating or answering a call). Execution: Raise iPhone to my ear when there is no call active. Perception: I hear a unique, specific tone through the speaker. If I lower the iPhone, I can see that a microphone icon has now appeared as well. Interpretation: The tone is a familiar indication that Siri's voice input is now active. I have seen and heard this tone while activating Siri manually (by holding down the [single] button at any time). The microphone indicates that Siri is "listening" and provides direct feedback about the input levels that the microphone is picking up. Evaluation: I know that Siri is ready to accept commands, and by watching the input levels, I can tell that the microphone is picking up my voice as I speak.

2 - Can answer: problems that certain personas may miss, as (good) heuristic evaluation requires different personas to provide more test coverage. One benefit is that wide, overlapping coverage means that each person doesn't have to be an expert, as problems are likely to be caught by at least one of the other evaluators. Heuristics may also be better at uncovering problems that user testing may miss, as users may take only a few specific paths through an interface, where a heuristic evaluator is able to see the interface "from 10,000 feet" and can see more meta-problems with the overall interface's design. Cannot answer: Experts may have a "professional blindspot" with which they might skip over interface problems that non-experts might be trapped in. For instance: selecting a chunk of text in this document is assumed to be: move mouse pointer to start of selection, click and hold the mouse button, move the mouse to the end of the selection, and then release the mouse button. Experts take this for granted, while non-computer-mouse users might be perplexed the first time they encounter the need for this task to be accomplished.

Elise McCallum - 2/13/2013 9:41:43

1) One mobile application I often interact with is Google Maps. When I begin using the application, my goal is to be able to enter a start address (or use my current location), an end address, and see both the step by step directions and a map highlighting my route from the start address to the end address. I start using the application with the intention to enter a start and end address. If I use my current location, then my input is "my current location" which is evaluated and becomes the GPS address of my current location. Then I enter the end address in the end text box and press the button that says "Calculate". I then expect the application to calculate the route and change the map to reflect that route.

2) Heuristic Evaluation can answer questions of usability, as that is what heuristic evaluations are looking for. They cannot, however, answer questions of how much the users actually like the application. Similarly, they cannot determine the severity of errors of usability, as they only have limited interactions with the interface and don't know how one small error will impact the rest of the usability of the application.

Jin Ryu - 2/13/2013 9:47:23

1. The mobile application I chose to use was the alarm setting on my phone. Goals: get phone to wake me up by 7:00 AM every day. Intention (meaning of input expression): to set up a reliable alarm on my phone to wake me up at that time in the morning daily. Action specification (form of input expression): Press appropriate buttons while viewing the visible items on screen on phone to find the alarm application and make changes. Buttons range from 'OK' center button, left menu button, and up-down-right-left arrow keys. Execution: 1) Go to the menu by pressing the center button 'OK'. 2) Use arrow keys (up, down, left, right) to select the icon 'Tools' and press 'OK' 3) Select the first item on the list 'Alarm Clock' and press 'OK' 4) Select an alarm out of 5 possible to edit and chose the first one by using arrow keys and pressing 'OK' 5) Enter description in 'Description' textbox using keyboard on phone 6) Press 'down' arrow key to edit textbox 'Time' and press 'OK' to edit 7) Use arrow keys to set time (7:00 AM) and press 'OK' to finish editing 8) Press 'down' arrow key to edit textbox 'Repeat' which defines how often the alarm can ring and press 'OK' to edit 9) Select 'Daily' from the list by using arrow keys and press 'OK' to confirm change 10) Press 'down' arrow key to edit 'On/Off' status of the alarm and press 'OK' to edit 11) Select 'On' from the 2-option list and press 'OK' to confirm change 12) Press left menu button (different button from arrow keys and the 'OK' button) to save all changes for the alarm inter-referntial I/O: Phone saves the information and acts accordingly. Perception (form of output expression): There is a picture of an alarm clock that wasn't there before next to the alarm that I switched on out of the five in the list. Also at 7:00 AM, the alarm sounds using phone's pre-set ringtone and volume. It displays two options ('Snooze' which will quiet the alarm but allow it to sound again in 10 minutes or 'Exit' which kills it completely for the day) and keep ringing until the person responds with a button action, or after a few minutes have passed. If no action is taken, it shows up in the 'Missed Alerts' section and by default act as 'Exit' was selected. Interpretation (meaning of output expression): The new icon indicates the alarm is turned on. The time is 7:00 AM in the morning and the person was alerted to wake up if they are sleeping. Also, it is a confirmation that the alarm was on and functioned properly. Evaluation: If the person responded to the sound and turned it off (or snoozed), it means that the alarm worked and the person has woken or at least their sleep has been disturbed so that they are more likely to wake later. Interface was clear and not hard to use.

2. Questions that can be answered by Heuristic Evaluation (not UT/US): a) assessing how intuitive and recognizable an interface would be to use without being complicated by other issues such as implementation problems b) finding and analyzing theoretical design flaws on paper c) see if interface meets many of the broad heuristic principles for good design d) focused opinions and analysis by evaluator of design interface rather than which of his actions cause what e) first impression by evaluator of interface f) questions that user may ask on how to use an interface g) initial usability characteristics of the interface domain

Questions that can be answered by User Testing/Usability Studies (not HE): a) to discover unexpected, actual design problems through real use, such as hidden implementation bugs b) mistakes that users will make when using the interface c) problems in user actions (user doing something unexpected, or program doesn't respond the way it is supposed to) d) how users solve their own questions in using the interface and arrive to a solution e) if design works when used practically by many people and realistically

Derek Lau - 2/13/2013 10:07:18

One mobile application that I've interacted with, that I believe has a great interface represented by the action cycle, is Flipboard. My goal starts as wanting to read through the news that I've selected as interesting to me. To achieve this goal, I need to be able to discover new stories apart from the current story that is displayed. My intention is provided for me at a very high level, decreasing semantic distance, where my intention is to "find the next new story," as opposed to a lower-level representation, which would be something like "increment the story counter by one." As for input, using the touch interface of a mobile device already decreases articulatory distance by mapping of the finger inputs to the application. In addition, in Flipboard, to access a new story, I need to only swipe my finger up on the screen to "uncover" a new story. The ease of use in this application comes from the tight coupling of the meaning and form of the input expression. As for the output, Flipboard reacts by animating the bottom half of the board to flip upward in a pseudo-3D fashion, revealing additional text below the board. What this output means for me as a user is that with the revealing of additional text that there is a new story underneath the current one, which builds a data structure in my head of story stacked on top of one another in layers that I can flip through. My new goal is then to read this new story or to flip through again to the next story. Flipboard's success comes from its incredibly small gulfs of execution and evaluation, as it just seems "intuitive."

In heuristic evaluation, evaluators are able to evaluate against a set of given heuristics (such as the list of 10 usability heuristics provided in the reading) and can pinpoint certain features or problems very specific to these usability heuristics. On the flip side, problems that are not related to these set heuristics may go unnoticed, even if they are common tasks that are used often, only because they might not be outlined in the set of heuristics. Another question that heuristic evaluation answers is how well can the system be used in the context of feedback, since the evaluators are able to ask questions about the system. Again, on the contrary, the observers won't be able to see how well the system is "learned" by the evaluators through trial and error, since they will be more likely to ask a question rather than stumble through the system on their own and spend unnecessary time.

Ben Goldberg - 2/13/2013 10:54:43

1) The app I'll use as an example is my texting app. If my goal is to write "hi" to a friend, I bring up the virtual keyboard and input the characters "h" and "i". These characters then get converted to binary representations. Pretty much as soon as that happens, the screen will represent those bits as pixels on the screen which look like the characters "h" and "i". This signifies to me that the program has received my input and then I can go on to the goal of actually sending that message.

2) I'm not sure that there's any questions that Heuristic Evaluation can answer that regular user testing or usability studies cannot. Heuristic Evaluation seems to be a simple, cost-effective way of testing that will reveal most of the flaws in a system. Other testing can be done to determine these same flaws and more, it'll just cost more to test.

Questions that can not be answered by Heuristic Evaluation are subtle problems that aren't on the list of heuristics. The fact that these problems are harder to point out shows you why Heuristic Evaluation works so well, because it cost-effectively determines the most important problems.

Marco Grigolo - 2/13/2013 11:45:23

1) Seeing more posts in Facebook mobile App. Intention: move the list of posts up, so that the posts that are "hidden" behind the lowest posts might get dragged into the screen Action Specification: Touch the screen and drag the screen up, so that the posts shift up, newer posts that I already saw going away, and older posts that I want to check coming out from under by the dragging action Interpretation: As I drag my finger, is seems that the I am dragging a sheet of paper placed under the screen, so that I can choose which posts I am going to visualize in the screen. Evaluation: To see older posts, I just need to drag them on screen. New Goal: See more older posts: Just drag the screen some more

2) Since they are using heuristic rules, a Heuristic Evaluation can answer why a certain function do not work, and not only if a function work or not. On the other hand, since they follow some rules to evaluate the functionality of an interface, they are more directed and might not catch some functionality problems that a wide user testing or usability study might catch.

Kevin Liang - 2/13/2013 12:03:22

1) Let's take something simple like shazam for example. Input requires me tapping the screen while the music is playing. The goal is to find the name of the song. The new goal is achieved about 10 seconds later finally outputting the name of the song. The next goal is to verify whether this is the actual song. Shazam provides links to iTunes and YouTube that can indeed help verify it.

2) Well Heuristic Evaluation generally involves UI experts evaluating the UI through a set of rules. These rules are basically "proven" to work with UI so no user testing or usability studies cannot answer. Vice versa, user testing and usability studies can answer new questions. If your UI has not been researched enough to have an "answer" or "solution" to how to design it, then user testing and usability studies can help!

Sumer Joshi - 2/13/2013 12:04:11

1) The mobile application that I picked is the ESPN ScoreCenter application. As a user, my goal is to see the scores of sports games that are happening real-time so I know how my team is doing. These sports could range from football, soccer, or Formula One Racing. The first intention is to open up the application. Semantically, I know what team to search for by typing in their name or going through the scroll down menu to see what league they are in. The articulation of this task is then initiating a mouse click on the league. This will then execute, and give me an I/O response of what past or current events is occurring with that league. Suppose now the Golden State Warriors are currently beating the Bulls, with a minute left in the 4th quarter. I perceive this outcome by looking at it, in my mind I am now interpreting that the Warriors might win. Finally, evaluating the outcome would be figuring out whether or not I should turn on the TV and actually watch the game.

2) Heuristic evaluation is very rigid. Unlike user testing, the heuristic evaluation allows evaluators to ask questions while these questions can be asked in a prompt and timely manner. The questions that can be asked are directly related to the mechanics of the interface. For example, the evaluator can have a lot more user control that they can be helped, while the user tester actually needs to go through it. Vice versa, by being helped out so much, this heuristic evaluation is a minimal scope test of an actual usability test that only shows minor issues and not major ones.

Arvind Ramesh - 2/13/2013 12:22:32

1. A good example of this process in a mobile application is Yelp.

When using Yelp, my initial goals are to find the best place to eat/drink/watch a movie etc. It is the most useful application for these kind of goals because it so many user-reviews, which I think are the best way to measure the quality of something.

Intention: My intention is to find a good, cheap restaurant that is within walking distance.

Action Specification: This usually consists of me using various filters, such as price, rating, and distance, as my search parameters for nearby restaurants.

Execution: For Yelp, this is just pressing the "search" button after I have entered in all my filters.

Perception: The first thing that comes up is a list of various restaurants along with their rating, price, and distance.

Interpretation: I will go through the restaurants that came up with my search and see if there is anything I like. If not, I will change the search parameters and redo the search (thus going back to step 1). If I find something I like, I will click on it to read some user reviews.

Evaluation: After I feel I have enough information, I will decide whether or not I want to go to the restaurant I found. If I decide I want to go there, I have to find the directions, which become my new set of goals. I click on the map feature of yelp, and go through the process again.

2. Heuristic Evaluation can answer a lot of questions that are easily missed by regular user testing because testing with a larger amount of people decreases the chances that a usability flaw will be missed. For example, if your new mail client cannot sync to the users' calendar, the heuristic evaluation is sure to uncover this, as at least one of the users will try to do it. Perhaps "Syncing events to the calendar" was one of the heuristics. On the other hand, a user without a set of heuristics might not even think of such an issue.

However, heuristic evaluation cannot answer questions that didn't exactly fit in one of the heuristics the evaluator was given. In this case, maybe some rare or unforseen circumstances will causes problems in your UI.

Avneesh Kohli - 2/13/2013 12:26:34

The example I’ll use is interacting with the Spotify app on my iPhone. My goal is that I want to play a particular song that I have in my head. My intention is to be able to find this song and start playing it. The interface of the application is structured to allow me to express the song that I want to play in a couple of ways, including searching for it explicitly, or browsing through playlists. When I find the song I’m looking for, in order to satisfy my goal of playing the song, I can simply tap the song to start playing it. I perceive that the song is about to start playing because on tapping the song, it highlights temporarily to indicate a selection has been made. At this point, I can reasonably expect to be successful, and can evaluate my success based on whether or not music starts playing. If music does start playing through the headphones/speaker, and I can immediately interpret that to mean by action was successful. At this point, my goals might change to something else, such as adjusting the volume of the audio, or browsing through my library to find another song to play.

Since heuristic evaluation asks the user to evaluation specific things about the usability of an application, you’re much more likely to get feedback that points to particular things that should or shouldn’t be changed. In standard usability studies, administrator is more likely to get very open ended feedback, that might be influenced by things like cosmetics and other miscellaneous factors. Since you’re unable to control what other various factors are playing into the feedback of a usability study. On the other hand, open-ended feedback through user testing can be very valuable, as it can be an indication of whether your app is even going in the right direction, and just the overall experience that the user is getting out of your application. Heuristic evaluation is strictly limited to a user’s ability to perform certain actions in an application, and that won’t give you the user’s overall impression of the application.

Mukul Murthy - 2/13/2013 12:35:18

1. My goal is to use the ConnectBot Android app to access my cs161 account so I can submit my homework. I first specify my intention; I want to connect to my account, cs161-dk, using SSH on the default port, and choose as the server I will use. Now that I know the meaning of my input, I need to figure out how to communicate this to the app - in other words, how to bridge the articulatory distance. When I start the app, I see a dropdown menu defaulted at SSH, so I stick with that. The other part of the action specification is to enter my username and server. The app has a text field which says "username@hostname:port", so I know the exact form of my input expression. I type in "" in this field and hit Enter. The app prints some text in a console, and I perceive that it is saying I am connected but not authenticated. I interpret the last line, waiting for "keyboard-authentication", as requesting a password. Thus, I evaluate that I need to enter my password to authenticate myself. That distance - the app printing some text to me understanding that I have to authenticate, is the gulf of evaluation.

My new goal is to authenticate myself, so I must go through the cycle once more. I intend to authenticate myself with my given password. To do this, I enter it in the console where I had been prompted and again hit Enter. This time, I see that the messages that fill the console are different. I interpret these messages as saying that I have been authenticated, and I evaluate these messages as success. I have connected to my class account and can now do whatever I want on it.

2. Heuristic evaluation and regular user testing both have their benefits. Heuristic evaluation can answer early questions about the apps simply using prototypes much better than user testing can. In heuristic evaluation, someone is evaluating the interface, not the way the app is used to help them out, so a flashcard prototype of a mobile app would work fine. Heuristic evaluation can also answer fundamental questions about the interface; the people doing these evaluations generally know something about user interfaces, whereas user testers may not. Heuristic evaluators can also find flaws in the interface that may not come up through general user use, but are still problems with the way the interface is set up. On the other hand, user testing is the best for answering the question "does this app help its target users?" because the testing is done by target users. This is better for finding critiques of the functionality of the interface for how it will be used, in a more concrete way than heuristic evaluation does.

Eric Xiao - 2/13/2013 12:39:06

1) For the purpose of this exercise, I'm going to choose Wunderlist as my app. My goal here changes ever so slightly each time, but the majority is to update my immediate to-do list and write down what I need to do within the next few days. This is the meaning of my input expression. This translates to a formulation of an action, which is that I want to add another thing to my to do list.

I click the to-do list of choice "Immediate" and click the "Add an item" box, type in what I need, and hit done. The application responds by placing what I just entered onto the top of the list, and this is my interpretation of what's happening. Then I formulate that I have completed my to-do list, and realize my goal is completed. Perhaps I want to create a new list or add things to the classwork list. Then the cycle repeats.

2) Heuristic Evaluation can help answer questions about user interface design before the actual interface has been built. It can be conducted on paper and utilizes design principles over analyzing user behavior, which is softer but also more detailed and scrutinizing. It's not backed by data, but it makes up for it by offering clear connections to design principles that help bring together a solution. It's a type of evaluation that puts high value on the opinions and skillsets of the experts, which are the evaluators.

Heuristic evaluation do not test whether or not an interface will actually make sense to the masses. They can be done with a target audience in mind, but products change and the audience that actually benefits from the app may also change. These expert evaluators also may not have the background that the users would have, and might fail to spot what's important to the target user, even if by design principles the user interface is clear. An example of this would be evaluating the effectiveness of a website to help manage online ad content. The information might all be presentable, but the experts might not know the importance of the data describing the visibility of the ad vs. the importance of tracking spending. These are both important, but in different contexts, one may be better suited than the other.

Kayvan Najafzadeh - 2/13/2013 12:39:17

For this assignment I looked at the play store and find an app that is new to me to start the action cycle with it. App name: HowAboutWe Dating Goal: arranging a date Intention to act: find a date! my expectations were not good regarding this app, like it will ask many questions before giving any data, requiring many confirmations and etc. after installing and opening the app i had to sign up (which took me a second because there was only 3 fields: email, username, password) then couple of questions which each were presented in a different screen so it didn't seem bad (questions like gender, interested gender, date of birth, age range of interest, zip code) and at the end it asked for an idea for a date. and done. the application listed nearby users which are match to my interests with their date plan. I could have messaged each of the listed people and "Ask out" or open their profile to see details regarding them. new goal: this application is not only for serious dates and can be used for finding new friends and hanging out with them.

Zeeshan Javed - 2/13/2013 12:44:08

1.Look at Figure 6 on page 331 in the Direct Manipulation reading - this is a very important diagram. Give an example of you interacting with a mobile application of your choice and describe at least one full roundtrip through this action cycle when you work with that application. Start with your goals, and describe all steps until you end up at new goals.

The application that I choose to interact with is the Pandora Radio music application. In describing the cycle of using this application I would start with my goals and intent for using the application, that being to listen to music of my desired genre. My intent would be to enjoy and discover music of a specific genre that I may not be familiar with and may enjoy. My action specification that the application allows me to do, is typing in a song that I am familiar with, and one that I would like to hear music of a similar kind to. The application allows me to do this quite easily through the form of input expression it has. The application then begins playing music according to its algorithm and my perception dictates that it is doing what I expected it to. By playing a song that sounds somewhat similar to the one that I input my interpretation of the application’s output is very clear. My evaluation assesses the relationship between the meaning of the output from the form of the output goals and because it does what my goals intended, I can be satisfied with the application for satisfying it.

2) What kind of questions do you think Heuristic Evaluation can answer that regular user testing or usability studies cannot? Vice versa, what questions cannot be answered by Heuristic Evaluation?

The principle of heuristics greatly deals in how humans approach and solve problems that are presented to them. Furthermore Heuruistic Evalauation specifically involves evaluators examining the interface and judging its compliance with recognized usability principles. Heuristic evaluation can answer questions of how usable, flexible, error prone, aesthetic, and consistent the application system is to humans who are using it. It can also answer how visible the intent of the application is and how well the feedback provided by the system aides the user in guiding them to a successful experience. Questions that a Heuristic Evaluation can probably not solve is how relevant the application is to a targeted market and how popular it will be. For example, if the application is for people who surf and the application succeeds Heuristically it does not mean that it will neccirily be used or useful to surfer, and Heuristic evaluation will not detect that like other methods would.

Yong Hoon Lee - 2/13/2013 12:46:16

1. I will go through my daily use of the mobile app "BeyondPod", a podcast manger and player. Every day, in the morning, I start with the goal of downloading all of the new podcasts from the feeds to which I am subscribed. Hence, my first intention is to refresh all of the feeds, making the semantic distance rather small. In order to do this, the action specification is to press the small button with a "refresh" logo, namely two arrows arranged in a circle. I then execute the action by tapping the button (very minimal articulatory distance), at which point a dialog box pops up which confirms this intention and tells me that this process may use a large amount of data. I perceive this output, and interpret it as a warning that I should not be using mobile data (again, minimal articulatory distance), and evaluate that I should check that I am on Wi-Fi, a step which is not as automatic as the others, so that the semantic distance is greater than we have seen thus far. Hence, my new goal is to verify that I am using Wi-Fi instead of mobile data. My intention is to do so, and the action specification is to look at the top bar and see that the Wi-Fi icon is present, namely four concentric circle arcs. This is a very automatic process, so the gulf of execution in this case is very small. Likewise, in most cases, I am on Wi-Fi, so I perceive the Wi-Fi symbol and interpret it as such (an automatic interpretation due to the ubiquity of that symbol) and evaluate it as having Wi-Fi. Thus, I go back to my original goal, and form the new intention of continuing with the process. In order to do so, the action specification is to click "OK" on the dialog box. I execute it, and at this point, a notification appears in the notification bar which contains an arrow pointing down. I interpret this as the podcasts downloading (as it is the standard symbol for downloading on an Android device), and my evaluation is that the task is in progress. My new goal, having completed the previous one, is to add the new episodes to my playlist. Through this process, BeyondPod narrows the gulfs of execution and evaluation as much as possible by using standard symbols ("refresh", "download") to indicate confirmation of my actions and making the actions easy to access.

2. Because heuristic evaluation is focused mostly on pointing out problems, it is well-suited to answer the quantitative question of how many things are wrong with an interface. Of course, this question can be answered through traditional user testing, but in traditional user testing, the user has a tendency to focus on broad critiques instead of looking for problems. Furthermore, as mentioned in the article, because heuristic evaluation presents the tester with a clear set of heuristics to focus on, one can add domain-specific heuristics that the user can examine more closely, something that is possible only by advising the tester in user testing. Conversely, heuristic evaluation prevents the user from making design critiques, so that the question of how the interface performs overall cannot be answered unless asked after the heuristic evaluation. Hence, it is not useful for getting a broad sense of the quality of the interface. Furthermore, heuristic evaluation provides no answer to the question of what things were especially positive about the interface, as it is focused with finding problems. Finally, heuristic evaluation does not give an answer to how the problems found should be fixed, as mentioned in the article, though often the solution is self-evident.

Zach Burggraf - 2/13/2013 12:47:51

1) Very simply, the goal of checking if you have new email on your iPhone. This intention leads to the action specification of pulling down on the list of previously loaded emails (perhaps not very intuitive) which triggers the execution of whatever code gets new emails. Then the page is loaded and perception leads to the interpretation (that each item in the list on the page is an email) and evaluation determines whether or not one of the emails is new or not. If there is a new email your new goal is probably to read it; if not your new goal might be to go back to playing temple run.

2) Heuristic Evaluation can determine the severity of the problem and provide more insight into how a problem can be fixed. It is also faster and cheaper at exposing major, common problems. However, User Testing and studies are more suited to uncovering the rarer problems than Heuristic Evals.

Yuliang Guan - 2/13/2013 12:48:08

Heuristic evaluation if a way to let designers know whether their interface fulfill users' guidelines. Heuristic evaluation can save more time than regular user testing. Generally, heuristic evaluation needs several evaluators and each of them only needs one or two hours, but regular user testing may take several weeks or even longer. In heuristic evaluation, the problems cannot be shown as code, instead, they are only described by users. In another word, it's not necessary to interpret users' actions. Meanwhile, there are some questions that heuristic evaluation cannot answer. It is hard to get good severity estimates from the evaluators . A single evaluator can only find a small number of problems, so the severity rating is incomplete. User testing is more accurate than heuristic evaluation. On the contrary, heuristic evaluation may miss some problems. Heuristic evaluation does not provide a systematic way to generate fixes to the usability problems or a way to assess the probable quality of any redesigns. We can not only do heuristics evaluation or regular user testing. Instead, we'd better do heuristics evaluations prior to regular user testing.

Juntao Mao - 2/13/2013 12:48:51

1) The mobile application that I will use to show the cycle is Google Map. So the goal is to go to Union Square. The Intention is to leave from home, and arrive at Union Square in San Francisco. The Action Specification is Home location in the “from” field, and Union Square in the “to” field. And then Google Map does its magic (execution) and output a route. Interpretation is me looking at the map of route, and trying to understand it. Evaluation is when I look at the map to see if its the right locations, and if the bus route is efficient. And thus I arrive at my original goal (or potentially needs to redo because the output is wrong).

2) Compared to user testing experimenters, Heuristic Evaluators would be more abled to answer questions about whether the UI fulfills the general rules that describes common properties of usable interfaces. The evaluators would also be better at imitating a specific group of people because they are given a list of heuristics that the target group should need, and not have to infer themselves should they not be the target group. Questions about how non-domain-experts would react to the UI is not very well answered by Heuristic Evaluation.

Ryan Rho - 2/13/2013 12:49:01

1) Look at Figure 6 on page 331 in the Direct Manipulation reading - this is a very important diagram. Give an example of you interacting with a mobile application of your choice and describe at least one full roundtrip through this action cycle when you work with that application. Start with your goals, and describe all steps until you end up at new goals.

I would like try Wolfram Alpha application for iPhone, a multi-purpose calculator that understands your human-friendly input returns appropriate data. The goal is to calculate 15% of $53 in a restaurant. You pull up your iPhone and tap Wolfram Alpha app. Then there is a text box on top. You would like to type '15% of $53' followed by pressing 'Done' button. The intention is to get the single value of 15% of $53 with at most two decimal numbers after the dot. After that you check whether you can type it using the keyboard. Once you see the keyboard, you type the string and press 'Done'. After that, the app returns several data where the first data is what I would like to know. Once I perceive the value, I check and understand that the value is the value what I want.

2) What kind of questions do you think Heuristic Evaluation can answer that regular user testing or usability studies cannot? Vice versa, what questions cannot be answered by Heuristic Evaluation?

Heuristic Evaluation mainly focus on understanding and identifying usability with a set of heuristics. So it's easy to detect common and widely-known usability problems. There is no guarantee that user testing would find out all the usability problems because the users do not stand for the whole group of users. On the other hand, heuristic evaluation can effectively point out commonly know problems. However, there are many types of user interface designs, especially detailed ones, that cannot be evaluated until they are user tested. Especially if there is a new user interface design, it is hard to evaluate it with Heuristic Evaluation because some aspects may not be studied so that some usability problems are not in the evaluation.

Thomas Yun - 2/13/2013 12:50:27

The application that I will use is a note taking application of some sort. My goal is of course to copy down maybe a word or a sentence. So my intention is to copy down a letter and the action would be to type or hit that specific key on the screen. Of course I would have to bring up the keyboard as well. After the machine performs the necessary action, the output and evaluation should be similar as the output would be the letter on the screen. There is no further observation required to evaluate as what we want is displayed. The new goal would then be the next key to hit.

Heuristic Evaluation can obviously answer questions regarding the 10 usability heuristics. In addition to the guidelines, with Heuristic Evaluation, the evaluator can possibly answer questions regarding other usability problems that they may find along the way. Of course as the reading suggests, with more evaluators, the possibility of more usability questions getting answered increases because some may find problems while others may not. Because each evaluator performs the evaluation alone, there is less likely to be conflict between different evaluators which may cause bias with each of their judgements. It seems that it would be easier to discover problems as well with Heuristic Evaluation because the observers provide hints on how to use the interface where with user testing, the evaluators generally are encouraged to discover problems themselves. But this also can be a problem as providing hints may make evaluators overlook certain problems with the interface.

Edward Shi - 2/13/2013 12:53:31

1) My goal when I first open my photo app on my iphone is to access my photos. To do so, I have to tap on the album for which I want to see. The tap is the form of input expression and I execute that. I perceive that the screen slides into a new screen with many photos. The output is a screen containing all the photos in the album and is where I wanted to go. Now I need to open the photo I want to see.

2) I believe the heuristic evaluation can show whether or not your instructions are clear, or whether or not you understand you interface yourself. Regular user testing, you are not supposed to provide any instruction and such you don't even know if you're clear on it. I think that the are very few interfaces that can be navigated without any directions or prior knowledge. As such, heuristic evaluation allows you to target that. Heuristic evaluation also allows you hone in on different heuristics and aspects. Say that there is one issue that people find unclear but everything else may be fine. In regular user testing, you may not be able to move on from the mistake but in heuristic evaluation, you can provide instructions so the evaluator can move on from the roadblock and continue testing the other aspects of your program.

Raymond Lin - 2/13/2013 13:02:49

1. The mobile application that I will be describing is the Facebook App. My main goals in using the facebook app is to see what's going on in the lives of my friends as well as keeping them updated on what I'm up to. My intention of doing so leads to my input action of posting messages on other people's walls or more commonly posting update status's. The action specification is essentially a block of text or in some cases pictures which makes the gulf of execution fairly thin. This output is simply posted on a news feed that all your friends are able to see. The way the information of my friends is presented to me is very clear and I'm able to interpret it very easily and respond even.

2. In Heuristic Evaluation, you are able to inspect the interface in a very methodical format, in a way regular user testing doesn't really provide as those are mostly opinionated. However, in that same regards people who know well what they want/expect are able to provide a different perspective about the interface.

Linda Cai - 2/13/2013 13:04:01

Say I wish to delete an app from a page on the icon shelf on the iPhone. This is my goal. To go from what the user wants to say to the meaning of an expression in the interface language, the user will bridge the semantic distance in the gulf of execution. Here, most of the structure is provided by the system. The interface is a very high-level language, and objects can be manipulated by touching the screen and ‘moving’ objects in the task domain directly. The output of the interface also shows changes as they happen. Thus, in this case the user can touch the object they wish to manipulate. Next, to go from the meaning of the input expression and the form of the action specification, the user bridges the articulary distance in the gulf of execution; here they do so by pressing and holding the app until an ‘X’ appears in the upper right corner of the app and then clicking the ‘X’. While pressing and holding is an arbitrary action, it gives a feeling of directly manipulating the object, and pressing ‘X’ is a common deletion or closure in many user interfaces, so it will provide ease of use due to consistency. Then the action specification’s form is executed through the I/O to the system, the system outputs an expression for the users to perceive (perception stage) that a change has been made. Here the form of the output expression is the icon disappears and the icons to the right/bottom of the app shift left/up. In bridging the articulary distance, we interpret that the app has been deleted. Once it has been deleted, we evaluate the meaning of the output and form new goals (semantic distance in gulf of evaluation). For example, my new goals could be, to rearrange the apps since they have been shifted over or move an app from another page onto this page.

Heuristic evaluations can find user interface issues that are not commonly faced by users on a regular basis. Evaluators often go through the interface multiple times using a list of heuristics, so they will likely find hard to encounter problems that regular user testing may miss. Moreover, users may waste time struggling with the mechanics of the interface instead of giving information about other parts of the design. Heuristic Evaluation can better give the severity level of the problem at hand, since they will be able to assess the frequency, impact and persistence of a problem once they notice it. Users on the other hand, will be given little guidance, so they will not likely keep exploring a particular problem with an interface. Users are likely to only notice significant issues, while evaluators will be able to identify a large amount of potential issues since they will scrutinize against the heuristics. Evaluation can also identify issues when the design is still in the very early phases, while user testing will be in the later stages. And also, specific reasons for the issue the user is having may not be conveyed to the observer, due to difficulties for a new user to explain why they feel an issue is problematic. On the other hand, heuristic evaluation is less representative of the target user population, since these evaluators are chosen because they are knowledgeable of user interfaces, are given hints when they get stuck, and follow heuristics carefully, knowing what to look for. User testing can help with iterative design, since new users can be tested throughout the design.

Linda Cai - 2/13/2013 13:05:28

Say I wish to delete an app from a page on the icon shelf on the iPhone. This is my goal. To go from what the user wants to say to the meaning of an expression in the interface language, the user will bridge the semantic distance in the gulf of execution. Here, most of the structure is provided by the system. The interface is a very high-level language, and objects can be manipulated by touching the screen and ‘moving’ objects in the task domain directly. The output of the interface also shows changes as they happen. Thus, in this case the user can touch the object they wish to manipulate. Next, to go from the meaning of the input expression and the form of the action specification, the user bridges the articulary distance in the gulf of execution; here they do so by pressing and holding the app until an ‘X’ appears in the upper right corner of the app and then clicking the ‘X’. While pressing and holding is an arbitrary action, it gives a feeling of directly manipulating the object, and pressing ‘X’ is a common deletion or closure in many user interfaces, so it will provide ease of use due to consistency. Then the action specification’s form is executed through the I/O to the system, the system outputs an expression for the users to perceive (perception stage) that a change has been made. Here the form of the output expression is the icon disappears and the icons to the right/bottom of the app shift left/up. In bridging the articulary distance, we interpret that the app has been deleted. Once it has been deleted, we evaluate the meaning of the output and form new goals (semantic distance in gulf of evaluation). For example, my new goals could be, to rearrange the apps since they have been shifted over or move an app from another page onto this page.

Heuristic evaluations can find user interface issues that are not commonly faced by users on a regular basis. Evaluators often go through the interface multiple times using a list of heuristics, so they will likely find hard to encounter problems that regular user testing may miss. Moreover, users may waste time struggling with the mechanics of the interface instead of giving information about other parts of the design. Heuristic Evaluation can better give the severity level of the problem at hand, since they will be able to assess the frequency, impact and persistence of a problem once they notice it. Users on the other hand, will be given little guidance, so they will not likely keep exploring a particular problem with an interface. Users are likely to only notice significant issues, while evaluators will be able to identify a large amount of potential issues since they will scrutinize against the heuristics. Evaluation can also identify issues when the design is still in the very early phases, while user testing will be in the later stages. And also, specific reasons for the issue the user is having may not be conveyed to the observer, due to difficulties for a new user to explain why they feel an issue is problematic. On the other hand, heuristic evaluation is less representative of the target user population, since these evaluators are chosen because they are knowledgeable of user interfaces, are given hints when they get stuck, and follow heuristics carefully, knowing what to look for. User testing can help with iterative design, since new users can be tested throughout the design.

Moshe Leon - 2/13/2013 13:08:16

1) I chose the app TalkBox for my evaluation, since it is an app which I use much to connect with my family and friends. I will start with the app right at the beginning- it is supposed to simulate a conversation, so as I want to speak to someone I click the person, and get into a chat room. What is unique in the app is the fact that the chats are short voice recordings, which simulate a walky-talky. Within the rooms, there are bubbles which represent parts of the conversations, which are being repeated, as recordings, when the user presses them. The cycle begins when I want to engage in a conversation with someone. First, I need to look for that person. The app is pretty standard insofar as helping you in finding the person, with a list of people whom you have as contacts. My goals as a user is to engage in a conversation with a real person, in real time, however, that is not possible. The app is trying to simulate a model-world, by creating chat rooms, with the people whom you wish to talk to. My goals change by necessity, and I am willing to take in and adopt this world. After I pick the person of choice and create the chat room, I can get inside it and engulf myself in this unique environment. I already get an output, by looking at a visual picture of the person in which I am about to converse with. Rather than typing in the chat, like in other chat-apps. There is a big button which reads ‘HOLD TO TALK’, really impossible to miss that. Once you press the button and hold it down, you are actually recording what you want to say to the person you will engage in a conversation with. My intention is to speak a sentence which will be delivered to the right place, and as soon as you release the button the message is being sent, no more work needed. The interface made sure to take care of all of the extraneous details that would have made it faster to go, find, and meet the person to talk to him in person. It reduced it all into a simple button to press and hold. I really think that the creators did a good job in reducing the Articulary Distance to minimal, and the Gulf of Execution is as small as it can be. WYSIWYG place a big role in here as well, there are no hidden features or things that are hard to comprehend. Semantically, the distance is also reduced since you literally speak up the same way you would if the person was next to you. The System takes in your input, and immediately (if you have internet connection) shows a chat bubble which contains your recorded sentence. As far as the Gulf of Evaluation, it is also minimalistic. The neat feature which I really like is to re-listen to my own words, in case I forgot what I said. It is a little out of connection with the real world, but in this virtual world I am now fully immersed in, it’s okay and fully acceptable. There is also a sound of uploading, which sounds as soon as you upload a recording. I don’t see the reaction on the other person’s face, as confirmation to have received the message, but it shows a little checkmark next to it as confirmation. Once the person actually listens to it, it will put a ‘double-checkmark’ on it, so I know I should expect an answer soon. The problem with the app, just like any other texting app, is that I don’t know when I will receive an answer. The other person might be too busy, and respond a day later. This Gulf, unfortunately, remains impossible to bridge. It is not the Gulf of Execution nor is it the Gulf of Evaluation, but a flaw in this model world which prevents the conversation from ever being realistic. An upside is that you cannot cut someone while he is talking- might be extremely useful when you really have to say something uninterrupted. The second cycle is when I receive a message, and the input, once more is as simple as pressing down a bubble which represents an incoming sentence. It is on the other side of the screen, separating my sentences from the other person’s sentences. There is also a sound which tells me of an incoming message. Execution, once more, is reduced to pressing the bubble button which you want to hear. The Gulf of Execution is fully bridged as far as I am concerned. The Gulf of Evaluation is also fully bridged, since the feedback is RAPID, and the recorded message from the other person is heard immediately, in the form of a regular voice. My goals, assuming the other person is responsive, are fully met. A conversation is really smooth if handled properly, and can flow very rapidly, almost like a real conversation, with a minor delay which results from the users having to push the buttons and record their message. But this is a small price to pay, especially when you don’t have the time to call someone, and that person is not next to you. GREAT APP!

2) It seems like the heuristic evaluation is targeting very specific test cases, so it might or might not cause certain tests to fail/pass, due to lack of randomness. Randomness is the key element for which new faults can be found, and just by letting a user use the machine the way he wants to these rare events can occur. Some designated heuristic tests might not even trigger a problem, which might need some outside element to fail, which is not included in the test case, or the scenario being run. It is not the way that a user would use the machine, without these areas of focus. Also, Severity testing cannot be properly engaged, for the same reasons mentioned above. Because the heuristics are so fixed at some particular thing they are about to resolve, it is sometimes better to just let a user do whatever he wants with the machine, and see what happens. An error, as bad as it is, might not be ever reached unless a user is directed towards that area of the interface. An important question that cannot be answered by heuristic evaluation is how to fix the bug/error, however, it will, sometimes in detail, express the value it might have in profit- (or loss) if it was fixed (or left as is).

Dennis Li - 2/13/2013 13:10:31

1) My goal is to update my facebook status using the facebook app. Open application, press status button, type in status, press post button. I can see that my new status is now displayed on the "newsfeed" for all my friends to see. This is the desired outcome and now my status update is complete. I can now work to achieve other goals such as commenting on posts.

2)Heuristic Evaluation is focused on recognized usability principles. This may be advantageous over user testing because the evaluators of Heuristic Evaluation are focused on specific usability problems that the product developers are interested in. By using Heuristic Evaluation, the developers can be sure that features they want to make available are indeed functional. On the other hand, Heuristic Evaluation does not focus on product usability. A feature that many normal users would want may be unknown to the few evaluators that are chose to test the product. Additionally, Heuristic Evaluation is not focused on how intuitive and easily usable a product is. The lack of focus on these areas could result in a product that has all desired features, but hard to use.

Lauren Fratamico - 2/13/2013 13:12:28

1) I will discuss using the talk app. This app is google chat for the mobile phone. Let's say that I want to start a conversation with someone (I have already clicked on their name). My goal is to send them the message "hi". I would like to be able to type that to them and see that the message gets sent. My intention is to type "hi". The form of input expression is a text box. I must click it, wait for the keyboard to come up, type the message (clicking buttons along the way), then hit the send button. This sends the message to the google server and displays it on my phone as a sent message (the form of output expression), and I can view the message I just sent. If it displays "hi" than I have been successful and my goal has been met. This whole series also comprises smaller goals for example pushing buttons leads to text being displayed in the text box.

2) Heuristic evaluation and user testing complement each other nicely. Heuristic evaluation is done earlier in the design process, so many potential problems with the interface can be found early. However, especially if the heuristic evaluators are not part of the end population that will be using the product, many of the issues found may be insignificant. Heuristic evaluations tend to be cheaper and faster so you can perform more of them. User testing tends to identify more significant bugs as it is tested by the end user group. Both evaluations find issues with the product, and are good complementary actions to take if time and money allows.

Harry Zhu - 2/13/2013 13:19:08

1) When I want to find out the title of a song that's playing right now, I'll open up soundhound, press the button to start listening to the music. The app interprets that button press as me wanting to find the information of a song playing. It then evaluates the audio input its receiving and searches through its databases to find the song match. Once it finds a match, it then intends to show me the information it found to me, displaying it on the screen.

2) I think some questions that Hueristic Evaluation can answer that normal user testing cannot is how will users interact with your model without any outside help or previous knowledge of the model. Some questions that cannot be answered by Hueristic Evaluation is the evaluator's thought process when he is actually doing the test. The observer could always ask the evaluator after the test has been done, but it most likely wont be as accurate as when the user is actually interacting with the model.

Dennis Li - 2/13/2013 13:19:40

1) My goal is to update my facebook status using the facebook app. Open application, press status button, type in status, press post button. I can see that my new status is now displayed on the "newsfeed" for all my friends to see. This is the desired outcome and now my status update is complete. I can now work to achieve other goals such as commenting on posts.

2)Heuristic Evaluation is focused on recognized usability principles. This may be advantageous over user testing because the evaluators of Heuristic Evaluation are focused on specific usability problems that the product developers are interested in. By using Heuristic Evaluation, the developers can be sure that their product is easy to use and intuitive to the user. The advantage of using evaluators for this process rather than users is the rigid systematic way that evaluators rate usability. This systematic rating system makes it easy to identify exactly what is wrong with the current product. Additionally, Heuristic Evaluation can be done on products mockups. The heuristics can be applied to paper mockups, an option that would be incredibly difficult with average users, not used to dealing with an unfinished product. On the other hand, the Heuristic evaluation model may not be able to fully capture every issue within a product. The rigid rating system, although useful in organizing and identifying problems, is a limitation on problem detailing. Also, key usability options that normal users are accustomed to in applications, may not specifically be addressed with the heuristic method. Additionally, specific problems that only arise when there are a great number of users cannot be tested or evaluated using the Heuristic Evaluation.

Michael Flater - 2/13/2013 13:31:31

1) GroceryIQ is an app that helps you keep a shopping list.

Goal: Add a specific item to the list. Intention: Get the item added to the list via the apps input method. Action Specification: Find and press the "Add Item" text box and enter the item "Bread" The application interface now has the input and the user, me, waits for some form of output. In this case the app accepts the input and acknowledges this with a pop-up "Bread added" Interpretation: I see the pop-up and read its contents. Evaluation: I decode the message to see that, in fact, bread has been added to the list. My goal has been met and the process continues with the next item.

2) Since heuristic evaluations focus more on reports from the evaluation instead of direct feedback from a observed user it stands that most practical problems will be discovered, probably more than with a traditional user testing model. The thing that will not be fully described is the usefulness of the system or interface. Since typical user testing uses real-world situations to find flaws in interface design or system design, it can be more helpful to determine a users actual needs.

Jian-Yang Liu - 2/13/2013 13:35:18

1) We take Google maps. The goal is to find the fastest route from where I am currently located to some chosen destination, say my home. Intention: I input my destination as well as my current location (using GPS) Action specification: The form of input is both a string and another program, the GPS, which also delivers a string input After execution... Interpretation: Google maps will spit out an image of a map with two points A and B on it, as well as a line conncting the two. Evaluation: From this, I understand that A is my current point, and B is my goal destination. The line connecting the two is the path that I will want to follow if I want to be as fast as possible in heading home. I now can have new goals if I wish, say to visit another destination in the midst of travel. I can simply change the input for destination to that new location, or add a new point in between A and B with that destination.

2) Heuristic evaluation outputs a list of usability problems in the interface with reference to those usability principles that were violated by the design in each case in the opinion of the evaluator. As such, as a whole, heuristic evaluation will be able to discover most of the problems that the user should need to use, answering such questions as whether all the required principles are there, whether these principles are easy to find/easy to access, as well as the simplicity of the overall design. The problem with heuristic evaluation is that the evaluator goes through the interface several times and compares dialogue elements with a list of recognized usability principles. Further, the evaluators may be reluctant to provide more help than necessary, and so if they were able to figure out the mechanics of the interface, they may decline to speak anymore regarding the topic. Not only this, but the evaluators will check only that the recognized usability principles remain, and that they are still within easy reach. Heuristic evaluation then will be unable to answer problems with regard to user habit. Also, users may wish to have additional features due to habit, which evaluators themselves may not care about if these features are indeed there, simply implemented in a different way. Thus, the questions heuristic evaluation cannot answer include how well the user will be able to adapt to this new design, how well it will be accepted by the community, as well as its usefulness to different kinds of people.

Tenzin Nyima - 2/13/2013 13:35:29

1) One of the app that I regularly use is Flixter - an application that I use to check if any new Hindi movie is currently being shown in the nearby UA Emerybay 10 theater. So my goal was to see if any new Hindi movie is being shown in the theater that usually show Hindi movies. The following are the steps involved: Once on the app, with the intention to find the theater, I click the button that says "Theaters". Clicking the button was the form of input expression. And in return I see the list of nearby theaters - the form of output expression. What I could interpret from the list is that these are the possible names of the movie theaters that I was looking for. Now I need to evaluate the names of the theaters and I need to click on the theater name that I was looking for. My goal hasn't reached yet. So I have to restart the cycle with my new intention to click on the name of the theater that I found in order to find a Hindi movie. My click on the button that says “UA Emerybay 10” was the input expression for this cycle and the list of movies that is currently being shown was the output expression. From the list of movies that the app provided, I could finally evaluate the name and the timing of the Hindi movie that is currently being shown in that theater. That's the end of my goal and now I can start with my new goals. For example, to check the reviews of that movie.

2) As read in the readings, in a user test situation, the experimenters simply interpret what his/her actions are and that doesn’t require any knowledge about the interface. But in Heuristic Evaluation, inspectors’ job is not simply to interpret the actions but usually suggest solutions to the problems they found. In user test situation, the experimenters cannot be helpful in reducing the errors while in Heuristic Evaluation, inspectors can help reduce the errors. So questions about the cause of the problem, questions about the further improvement of the interface and questions on how to eliminate the errors can be answered in Heuristic Evaluation but not in user testing situation.

On the other hand, depending on the expertise of the inspectors, in Heuristic Evaluation, the suggestion from the inspectors may not necessarily reflect what the actual users might think. So, in Heuristic Evaluation, questions that an ordinary users of the app might have could be missing and the final result based only on Heuristic Evaluation may not serve the best purpose for the real users of the app.

Timothy Ko - 2/13/2013 13:45:39

1) I want see my clock alarms on my phone. First cycle: My intention is to go to the alarm section on my clock mobile application. The specified action is to touch the alarm tab at the bottom of the screen. The machine interface then shows a screen of all my current alarms, which is what I perceive. I interpret this as a list of all my current alarms. I evaluate this as meeting my goals. I want to turn on an alarm that is off. Second cycle: My intention is to turn on an alarm that is inactive. The specified action is to move the switch of the target alarm from off to on, using the mini scrollbar on the screen from left to right. The machine interface moves the bar from left to right, and also changes the unoccupied space of the scrollbar from a faded “OFF” to a bright blue “ON.” I interpret this as having changed my alarm from off to on. I evaluate this as meeting my goals.

2) Heuristic Evaluation is a powerful tool for answering questions of the usability of an interface in the hands of an intermediate to expert user. By giving evaluators help when they need it, experimenters can quickly examine if an interface is truly valuable by reducing the time it takes for an evaluator to learn an interface. In user testing, the experimenter is not allowed to assist or interfere with the evaluator at any time, so it is possible for the evaluator to get stuck on one confusing feature for a long time, which causes the testing to yield few helpful insights.

However, Heuristic Evaluation is limited by the fact that it cannot address questions about problems that don’t arise from the standard usability principles. The scope of Heuristic Evaluation is narrowed down to a specific set of standards that the experimenter is looking for, leaving out unexpected problems that aren’t covered by the set of heuristics. Additionally, since the experimenter is allowed to assist evaluators using the interface, this further narrows the number of problems that could arise during testing.

Cong Chen - 2/13/2013 13:45:49

1) An example mobile application that I use is Instagram. Usually when I open Instagram, my goals are to take a picture of something nifty, make it look cool, and then share it to my Instagram and Facebook newsfeed. Starting from my initial goal, my first intention is to take a picture of a "nifty thing". I accomplish this by clicking on the center camera button. After that, a camera pops up and then, I take a picture. After taking a picture, the output expression is the picture I just took. The meaning of this output expression is that this is the picture of the "nifty thing" I want to share. Now, my next new goal is to apply a filter to make it look cool. My input expression is to apply a filter and I complete this input expression by scrolling along the possible filters and pressing one. Once I press one, the output expression is the picture with the filter applied; this output expression means this is what the picture of your "nifty thing" looks like. Now, my new goal is to share it. My input expression is to share it and my input form is to press the share button and select Facebook and Instagram. The output for expression is giving a notification that the picture was successfully shared and the meaning of the expression is that my picture was shared.

2) I think that Heuristic Evaluation can answer questions related specifically to how a interface can be improved and what about them is "okay" and can be improved. When conducting Heuristic Evaluations, you specifically tell the evaluator to watch out for certain heuristics and to pay attention to certain features. Thus, they will probably be more nit-picky about features, such as ones that are okay, but could be improved. This is unlikely in simple usability because a user will probably accept the "okay" feature.

I think that usability studies though can answer questions that are more along the lines of personal preference of the testing users. By doing simple usability studies, you will get the honest opinion of the users on the interface and features. When you give them of list of heuristics and things to apply to, it is more likely that they will generalize their response to what "most" people would say versus just saying their own opinion on the interface.

Achal Dave - 2/13/2013 13:47:05

1) The application of my choice is the calendar app on my phone. When I open it, my intention is generally to view my events for the rest of the day, which is a rather simple task. I open the app, and view it in the default "day" view, or the "agenda" view. If I want a list of events for the day, I swipe to the agenda view (the form of input expression). The phone displays a list or a schedule view of these events with mostly text and a few lines (form of output expression), which I then use to figure out when I can best nap in the day.

2) Heuristic Evaluation can give better explanations for why things are wrong, since evaluators are specifically looking for issues with the usability heuristics. With regular user testing, you are more likely to get responses along the lines of "this is annoying", while an evaluator can more specifically state that "the interface is too cluttered here". On the same lines, however, sometimes certain interfaces simply "feel" wrong, in an impalpable sense, to users. User testing could allow you to understand the user to see where they feel this way, while evaluators may not be able to place such notions in terms of heuristics. Furthermore, evaluators may be experienced and miss simple issues that average users may encounter often.

Monica To - 2/13/2013 13:57:33

1). The mobile application that I will use for this exercise is Instagram, the photo filter and photo sharing application. At the top of the diagram, I start with goals. My goals when interacting with this app is to take a photo, use a filter to increase its aesthetics, post it to the feed to share it, and lastly, to receive feedback from my friends. My intention is expressed by using my hands and fingers to navigate to the application on my iphone. The action specification is locating the camera button and understanding that tapping my finger on this button will bring up the phone's camera. Execution is pressing this button on the user interface. Inter-Referential is the stage between the form of input expression and form of output expression. Interpretation is determining the mean of the output expression; the camera has appears and I understand that now I move my phone around until I find the perfect angle to capture my photo. And evaluation is looking at the meaning of the output expression and comparing it to my goal. Did I capture and image with this app? And after going through this diagram several more times as I go through the process of taking a photo, applying a photo filter, and posting onto the feed, and getting feedback. I would have achieved my intended goals for using this app. The process breaks down every intention and action of a user when using a product with a user interface. It can definitely be helpful and applicable when we start to examine the usability of the UI for our mobile applications.

2). Regular and traditional user testing has a different goal from heuristic evaluations. The goal of traditional user testing is to find mistake, errors, and problems in the product or user interface. An example provided in the reading was that in user testing, the experimenter could not inject and help the tester out if they were stuck on some part of the product or didn't know how to navigate through the interface. The main difference is that experimenters aren't allowed to intervene if a tester comes across a trivial problem. For heuristic evaluations, where the focus is a domain-specific application, it would be a waste of time to allow the tester to struggle on a trivial problem with the interface mechanics for any extended amount of time. Heuristic evaluation involves more of an involved process where the experimenters and evaluates are discussing throughout the process; evaluators are allowed to ask questions and experiementers are there to explain. With this method, evaluators understand the goal and both parties could work towards the same goal. Practicing methods of the heuristic evaluation, answering an evaluator's questions or addressing their struggles when necessary will give the experimenter the opportunity to evaluate the usability of the user interface of a certain product with respect to the qualities and focus of the domain. I think it means that an experimenter should ask and allow relevant questions and that as long as answering certain questions will further the depth of observance of an evaluator during a testing session, then it should be permissible. The types of questions that cannot be answered by Heuristic Evaluation are problems that revolve around the mechanics of the user interface and it could not provide solutions to usability problems. Instead, they provide explanations of the usability problem relative to design principles and model system designs.

Tiffany Jianto - 2/13/2013 13:58:32

1) One mobile application I use is “iNextBus.” When I use this application, my goals are to find the real-time estimations of buses to my closest bus stop. My intention is to find the time I need to wait for a bus, and the action includes the calculations of the bus locations to my current location. The execution results in the display of the times of the different buses which stop at the stop I am closest to; I then perceive these different buses and their respective times. Interpretation is done when I look at the times and understand what they mean under each bus line. Finally, I evaluate the meaning of those output numbers with my goal; in this case, my goal has been reached and I have found out the bus arrival times at my closest stop.

2) Some questions that Heuristic Evaluation can answer that regular user testing or usability studies cannot are the many problems that are wrong with the application, because different people find different usability problems with applications; thus, it is more effective. Also, in heuristic evaluation, answering the questions of evaluators is more encouraged because it allows them to assess the usability of an application, whereas for normal user testing, the experimenters are more reluctant to provide help since they want to see the mistakes that users make. This allows heuristic evaluation to see what is more “usable.” Furthermore, in heuristic evaluation, people learn the explanation and reasoning behind why features are disliked, as opposed to just watching and seeing what happens; thus, this gives a problem and solution, as opposed to just a vague problem. Also, the severity of a problem can be determined. However, heuristic evaluation cannot answer the most common or best usability problem and can be hard to generalize because there will be a wide range of results. Also, it does not provide a systematic way for the fixes since everyone has different opinions and ways of doing things.

Brent Batas - 2/13/2013 13:59:12


One mobile application I use is ESPN Score Center. My goals are to keep up to date with my favorite sports teams—real time scores if they are currently playing, and info about when the next game is if they aren’t currently playing. I also want to be able to read recaps of games I might have missed. I also want to see scores of current games of other teams (that aren’t my favorite).

After installing the application, I am asked to add favorite teams and sports. This is expected; after all, if my intention is to see scores of my favorite teams, I need to tell it what my favorite teams are. I add the Tampa Bay Lightning, Tampa Bay Buccaneers, and Tampa Bay Rays. I also add NHL, NFL, MLB, and NBA since I like all four of the sports. The teams and sports I provide presumably, through the action specification, will be used to determine which sports scores, headlines, and articles the app delivers to me.

I also see the option to add alerts, but I don’t really care about that right now, so I ignore it.

After adding my favorite teams and sports, I am taken to the My Teams page. This screen shows various upcoming games of some teams, but not all. My interpretation is that this screen shows only upcoming games that my favorite teams are a part of—it does not show any other games.

To see info/scores for other games, I must go elsewhere. I click the menu button in the top left. A menu drops down with a list of sports: NHL, NFL, MLB, and NBA. I select NBA from the dropdown, and I am presented with a new screen that shows all basketball upcoming games and current games. So my interpretation is that this menu showed the sports I added earlier, and I need to select a sport if I want to see scores for that sport.

The application more or less accomplishes my goal, even though I had confusion initially about seeing only scores for my favorite teams, rather than all scores. Also, I had some confusion that I had to choose a sport before I saw scores for games. However, once I got into a habit of adding lots of teams to My Favorite Teams (aka any team that I am remotely interested in, even if I would hardly call them my favorite), then the app does what I want. I just open it up now, and go to My Teams, and I see all the scores I want.

My new goals are basically just checking the application whenever I want to see scores, as well as adding/removing various teams when I start/stop caring about them.


Heuristic evaluation can find usability problems that are only found by few evaluators, since it typically involves using several evaluators (~5). It can also be good for answering which types of usability problems are more severe than others, and more commonly encountered. Heuristic evaluation is also good because it places the person trying to use the system is also the person who makes comments about it, rather than typical user testing where the two responsibilities are carried out by different people, which is a possible source of misunderstanding.

Heuristic evaluation, however, does not afford a systematic way to generate fixes or determine exactly how the design can be improved in any sort of quantitative way. However, given that usability problems very often fall into general principles (like the 10 discussed in the article), often a fix isn’t that hard to come up with, even if there is no systematic way of doing so. Another question heuristic evaluation can’t answer is precisely what is wrong with an interface. If the problem isn’t completely identified, you might end up producing a new interface that has a similar problem. Finally, heuristic evaluation in general is pretty unreliable if you only have one or two evaluators—the article suggests using at least three evaluators to have satisfactory ratings.

Soyeon Kim (Summer) - 2/13/2013 14:00:31

1. <An alarm clock app> I would open the app with a goal of setting an alarm for a certain time. This intention to set up an alarm specifies the meaning of the input expression which is to satisfy the user’s goal. Action specification, such as the activity of me clicking on the alarm clock and setting up an alarm by scrolling up/down the time I want, spans the articulatory instance in the gulf of execution. The form of the input expression, the numbers (indicating minute and hour) is executed by the user (me) on the machine interface and the form of the output expression (i.e. also the numbers) also appears on the interface, to be perceived by me. I stop the scrolling activity of numbers once I found a desired alarm time matching to the time appears on the alarm clock interface. I perceive that time to be the time to b e alarmed from the machine (i.e. my iPhone). I finally click on the “on” button in order to enable the alarm at designated time. At the end, I evaluate the outputs (the set alarm time) on the interface and my original goal, which was to set up an alarm for a designated time. This process is initiated again once I have a new goal, to set up an alarm for another designated time.

2. Since the evaluators are provided with the Heuristics principles that are category-specific, it becomes much more specific and easier for evaluators to point out violations than user testing. Also by Severity Rating, heuristic evaluation can provide a better planning in problem fixing because one can allocate resources to fix problems acridly to the severity of an issue and this can also make it easier for the design team to prioritize their follow up tasks.

Heuristic evaluation may miss a problem that is not a part of given heuristic. Also, user testing can provide more accurate (by definition) and natural because their feedbacks are derived from user’s intuitive approach to the application, rather than evaluation of application with given heuristic principles.

Oulun Zhao - 2/13/2013 14:01:46

1) look at diagram. Example(a full roundtrip through the action cycle): Whatsapp (chatting app). I see a tab named chats (evaluation) and click on that tab (action specification), the app executes (execution). After that I expected to see a chat room (perception), then I see an empty chat room with clean interface (interpretation), then I notice an new message button on the right top (evaluation). Then I go for new goals, create a new message to my friend. 2) What Heuristic Evaluation can answer (other methods cannot): Some very detailed interface problems. Because Heuristic Evaluation is very systematic, it will evaluate the interface comprehensively. However it could also be a disadvantage of Heuristic Evaluation, because it tends to discover too many low severity problems (false positives)

What Heuristic Evaluation cannot answer: Since Heuristic Evaluation is not really user testing. The evaluator either have expertises in related area of have some level of knowledge about the product. Therefore it will not fully simulate the real user situation. Heuristic Evaluation might uncover many minor problems but might still overlook some major problems, which might only happen to users who do not have any knowledge about the product.

Erika Delk - 2/13/2013 14:03:58

1. When using the Google Maps app on my phone, I at first percieve the background and the options menu. Because I can read English, which the menus are written in, I can interpret the interface. I can then evaluate what steps to go through to find the directions to my destination. In Google Maps, this involves selecting the drop down menu and entering the name of the destination. My goal at this point is to enter my information and find my destination. My intention is to use the app to accomplish my goal. I user my fingers on the screen as the action specification to enter the name of my destination. I do this successfully. I then get a map to my destination.

2. Heuristic Evaluation would give more information about the parts of the interface that users were likely to find confusing or unintuitive, even if they were still able to accomplish their goal. This is because HE encourages users to ask questions and clarifications and can show evaluators what parts the testers are struggling with.

Heuristic Evaluation would not be able to provide feedback about the aesthetic design of your interface because it focuses on usability.

Scott Stewart - 2/13/2013 14:07:27

When using the flashlight application on my phone, my goal is to turn on a light. My intention within this app specifically is to turn on my phone's light. The action that is required is to press the large button in the center of the screen that says "Power". The output that I receive is all visual. I can see the light come on, and the power button turns from blue to green. I can interpret the green power button to mean that the light is on, but the immediate result of having the light turn on is enough to determine that my goal has been accomplished. At this point, my goal will shift to turning the light off.

Heuristic evaluation can answer questions that are more related to the design elements of a project. This would include the typography example from the reading. User testing may not address these issues because users may not specifically point out that the font is distracting, even though they would notice the effects. An expert during a heuristic evaluation, however, would know to look for these details that may affect the user. Heuristic evaluation may not answer certain questions about usability within a specific domain. The evaluators may not be experts within the field that the application is designed for, so they would not be able to address how effective the application is at solving certain problems.

Matthew Chang - 2/13/2013 14:08:03

1) My goal is to look at today's news from my twitter feed. Say I open up Tweetdeck on my phone. The app itself loads and displays my main twitter feed. It shows the last tweets that I was looking at the last time I had the app open. It doesn't provide as to when the tweets were sent, but it does give a qualitative indicator of how many new tweets have been made since last opening the app. Looking at this indicator, I see that the yellow bar reaches from the top of the screen all of the way down to the bottom, which I interpret and evaluate to mean that there have been a lot of tweets. I re-evaluate my goal to determine when were these tweets made. To convey my intention to the app, I swipe downward in the middle of the screen. The top bar now indicates roughly what time these tweets were made and I can now re-evaluate on whether I want to continue looking at the newer tweets.

2) Heuristic evaluation can surface arguably more subtle things regarding an interface. Some of this can be in documentation, where the general practice of an observer to help the evaluator when stuck is an advantage for heuristic evaluation. Another is that since there is dialogue with the observer, the thought process of the evaluator can be surfaced and answer questions such as what should be visible for a given view or expectations for various steps. A strong downside for Heuristic Evaluation is that it does not provide the designer a clear means of solving the the problem.

kate gorman - 2/13/2013 14:08:16

1) Using the paper app: goal: draw a funny picture intention: to draw a fat blake line action specification: choose a brush and a color from the tray execution: move finger across screen to use the brush perception: lines appears abross the screen interpretation: this line is the one i drew evaluation: the line is too thin new goal: erase line and redraw.

2) Heuristic Evaluation can answer questions about whether the current interface falls victim to any common usability issues that are known by the evaluators, instead of having a group of users come in who may or may not experience the usability issue. The user here gives the evaluation, as opposed to in user testing where the observer rates the interaction. Heuristic evaluation allows for more insight into the users actions by seeing what flows they prefer and how they intend to interact with the system, rather than only observing the comments and feedback of the users in user testing.

Heuristic evaluation cannot help tell you task analysis or desire information or how likely a user is to desire or crave a certain functionality or interaction. It can only evaluate the interface alone for the standard usability flaws.

Anh Mai - 2/13/2013 14:09:07

1) Look at Figure 6 on page 331 in the Direct Manipulation reading - this is a very important diagram. Give an example of you interacting with a mobile application of your choice and describe at least one full roundtrip through this action cycle when you work with that application. Start with your goals, and describe all steps until you end up at new goals.

When I'm on my phone and using the Mail application for example, my first goal is to check to see if any new mail has arrived in my inbox since the last time I have used the application. My intention here is to prescribe an action that will tell the application to check for new emails. The action specification is to swipe the screen downward so that the application will know that I want to check for new mail and does it accordingly. The execution happens after I accomplish the prescribed gesture, and new mails are displayed onto the screen, marked as such - this is the form of the output expression. My evaluation is that the program has done its job, and I now see new mails in my inbox. This will give rise to new goals which are usually to read and possibly reply to those emails

2) What kind of questions do you think Heuristic Evaluation can answer that regular user testing or usability studies cannot? Vice versa, what questions cannot be answered by Heuristic Evaluation?

Heuristic Evaluation is designed to answer specific questions about usability principles and standards that the interface design is supposed to achieve. This kind of evaluation makes it possible for the experimenter to just record comments made by the evaluator and not have to interpret his actions. I think this kind of evaluation can answer questions like whether the system will be useful to its targeted audience, whether there are any use case by a regular user that has not been covered - basically the interface's usefulness to someone who already knows how to use it and would possibly apply it to every day use. On the other hand, regular user testing is designed to test whether the interface is usable and intuitive at all - since evaluators are asked to find the answers to any questions they have about the system by using the system itself and not ask the experimenter.

Brett Johnson - 2/13/2013 14:09:19

1.) Clear is an iPhone application that I use when my goal is to enter a new to-do or reminder. The first step is to create a new entry. The intention, or meaning of the input expression, of the entry is that it will serve as a space to write down what I need to remember. The action specification in this case is pulling the list down, an action performed on the touchscreen. After executing this touch gesture, the interpretation of this action is that now there is a new entry, and the evaluation is that this is where I can place my text. After this first cycle, a similar one begins in order to actually enter the text. The intention is that this text will remind me later on of what I need to get done. The action specification in this case is a string of characters entered via the onscreen keyboard. After typing my reminder and entering “Done” on the keyboard, the output is my newly created entry with the specified text. After evaluating whether or not this is what I wanted to enter, the task entry is done.

2.) One of the things about heuristic evaluation that interested me was the use of providing the evaluator of an unfamiliar system with a “typical usage scenario.” Compared to other forms of usability testing in which the evaluator is on their own to figure out the system, I think that this may be able to provide the observer with some useful information if the evaluator would otherwise have no idea what to do with the system. However, I think that this could introduce some issues into the test as well. If too much information is given about how people normally use the system, the evaluators might be pushed into one way of using the system. This could mean the evaluators missing out on designing for people new to the system.

Eric Wishart - 2/13/2013 14:11:35

1) For my bank of america mobile app I first perceive the saved account ID's list. There are also several tabs labeled Deals, Alerts, Locations, and More. My goal at this point is to check my account which is already stored as an option on the screen. I intend to touch the box that has my stored account. I am going to do this using my finger. I successfully touch the box, which then loads the password screen, taking us back to the beginning of the cycle.

2) Heuristic Evaluation can answer questions that determine if an interface is usable rather than just if you can accomplish tasks regardless of usability. Regular user testing would not determine annoyances or stylistic issues.

On the other hand it would be difficult to determine things that are exposed in a large user pool if doing heuristic evaluation. This might include issues that specific user groups might have vs the average user.

Sangyoon Park - 2/13/2013 14:16:34

1) Let's say we have a mobile application that is designed to help us find fresh produce in a local farmers market. The user wants to find a best quality apple in the market today (goal). User runs the application and looks at the search produce menu, then the user thinks what menu he wants to select. The user's intention is now to find an apple that he wants. The user forms his input expression by trying to type 'apple' in the search bar. The word 'apple' types in, and it goes through inter referential I/O, and the device outputs a list of apples with each description and ratings(output). The user now looks at the output and thinks he is looking at all the apples being sold in the market. He sets a new goal, looking for the best apple based on the given information. The user continues, a few cycles later, he will find the produce. 2) Evaluators are walking through somekind of checklist when they conduct heuristic evaluation. User testing is about a real usability test among many users. The difference here is that, in heuristic evaluation, evaluators will find well known/studied problems that users make while it can't easily find something that is totally unexpected. Conversely, user testing can find most of real problems but they can be limited since user testing is more about focusing on the goal that users want.

Eric Leung - 2/13/2013 14:16:58

An application I have used is the Yelp app. My goal was to find a restaurant near me, that was open now, and had a $ or $$ for price. After typing in restaurant and using my location, I was given a list of all restaurants near me. Then I had to filter down by open now and price, and got a reshifted list of restaurants that followed my specifications. From here, I sorted by rating (not that fancy Yelp sort, whatever it is), and finally picked a restaurant nearby.

Heuristic Evaluation can answer whether or not a user interface follows standards that are common to most people and whether or not there's extra information on the screen. User testing and usability studies can't answer these because users may not necessarily notice that the wording is different and be thrown off by it, or may not have absorbed and been conflicted by extra information. Heuristic Evaluation is also cheaper and faster than having actual users testing the product. Heuristic Evaluation, however, cannot answer whether a user is comfortable using the interface, even if it is made "perfectly" by Heuristic Evaluation standards. Also, it cannot tell the designer whether they would use the product or not.

Christine Loh - 2/13/2013 14:17:16

1) When I interact with the Facebook mobile app, my goals are to talk to my friends. When I get into the app, I access the newsfeed automatically, which changes my goals to getting to know what is going on in my friends' lives without even talking to them, and even getting the urge to talk to friends because of what I see is going on. I can access the messages, which is right next to the notifications button -- where I can see anything going on that's related/tagged to me. This also changes my goals to figuring out what relates to me in terms of my friends' lives.

2) Heuristic Evaluation can answer more questions with a better average value than regular user testing. It can figure out things with an unbiased and independent sense of view from the subjects. It can answer questions from people who can not communicate with others about their experience. However, it can not answer questions regarding dual experiences, or experiences where subjects can ask friends what they think -- and dual experiences is often what an app is based off of.

mia kissick - 2/13/2013 14:17:55

1)My interaction with a Bus Rider App: Goals: check if one particular bus' s arrival time. steps: open up application --> refresh list of screens to check bus --> could not see my desired bus on the screen --> new goals: find a particular bus in the app 2)Heuristic Evaluation can answer questions like 1) Do users have the freedom to exit the system without going to complex processes? 2)Is the interface flexible for expert users for more accelerated user? 3)Is it necessary to provide documentation for the product? Question that cannot be answered by Heuristic Evaluation includes: 1)In the long run, will the user continue to use the product? 2) Do the user require any additional features?

John Sloan - 2/13/2013 14:19:59

1) I chose to work with FindMyIphone. Say that my laptop was stolen (this hypothetical has actually happened to me) and I want to locate it from my cell phone. My goal is then to locate it, so I open the app and the first I see a splash screen saying 'locating', followed by a list all my devices connected to my account. Looking at this list gives a clear intention of what I need to push to accomplish my goal. The arrows next to each device imply that if I want to locate that device I click on it. Next I interact with this button on the device and push it. What follows is the perception of a map with my device located in the center. I then interpret that it is indeed my device and I can use the detailed map to exactly where it is. Evaluation is then satisfied since the result was exactly my intended goal.

2) Heuristic evaluation is better for ansewring questions related to design principles and since there will be several different evaluators it will often find a lot of problems with the interface based on different peoples background in heuristics. Regular user testing can be better for answering questions about how real life users will use the application and interact with the design without much prior knowledge. This is more dependent on the people watching users use the app and their interpretations of the way it was used. Also heuristic evaluation can be done before the app is actually implemented which makes it more valuable in answering questions about the design very early in the process. This is not possible with user testing.

Nadine Salter - 2/13/2013 14:23:59

Direct Manipulation.

  • Goal: advance through slides in a presentation using Apple's Keynote Remote iPhone app
  • Intention: go to the next slide
  • Action Specification: on your iPhone, swipe right-to-left to shove the current slide off-screen and pull the next slide on-screen (assuming left-to-right ordering, as in the LTR language English)
  • Execution: your iPhone animates the next slide on-screen locally then sends a message via WiFi to your Keynote instance on your computer
  • Perception: your iPhone and computer both advance slides simultaneously
  • Interpretation: you went to the next slide
  • Evaluation: success! ...and 20 GOTO 10

(...this, of course, breaks down very quickly if your iPhone is on AirBears and you lose your network connection mid-lecture...)

Heuristic Evaluation is a more expert-driven approach to interface assessment that is largely theoretical: an HE evaluator can intentionally look at "big-picture" overall interface flow that users in user testing would not necessarily pick up on. Similarly, an expert evaluator will know (or consult a list that tells her) to evaluate usability in a number of different areas, while interface coverage in user testing might be spottier. HE, however, is executed by experts in user-interface design, not by domain experts; user testing might reveal that even a very well-built interface is at odds with the standard way things are done in the target domain, or highlight inconsistencies between the interface's assumptions and the end-users'.

Brian Wong - 2/13/2013 14:27:50

1) I use an Android application called Fitocracy which keeps track of physical activity in a gamified manner. I want to enter a workout after playing basketball (goal). I think of the workout in terms of how long I played, and there is a simple scroll wheel to enter time, giving a small semantic distance. And because the workout button is prominent and has a Barbell icon next to it, it is fairly well articulated (there is no specific basketball icon, only the word basketball, though). After pressing the "End" button (pictured with a stop sign to represent the end of the workout), it always outputs "congratulations on your workout" or something of that nature to articulate that your input went through. And it also displays the hours you played and the points earned (in this gamified system) so you know your goal of "tracking a basketball workout" has been reached.

2) Heuristic Evaluation seems to be able to break down issues much more systematically than regular user testing. At every point a user faces an issue, one or more of the usability heuristics is noted, and help is given to the user if necessary. After a number of users have gone through this evaluation, questions about which heuristics, specifically, and at which point they occur can be answered with precision, compared to regular user testing. Heuristic Evaluation can also progress farther, and thus answer questions about the use of the product through it's full cycle, because the evaluator will often help the user along when necessary. Heuristic Evaluation does not answer the question, however, about if users can learn about the product, on their own, without outside help. It does not also answer the question about how users like, or dislike, features, for it is more focused on what is happening rather than how that affects the user.

Timothy Wu - 2/13/2013 14:28:31

1) A great example of a mobile application that can be broken down into this action cycle is the Google Maps app. The high level interaction with this application is that you think of a destination and input it in the search text box. Once the app has found the destination, you can select from different options of the output, like a map, public transit directions, or car directions.

For a full roundtrip of this action cycle broken down in the steps of Figure 6 from the reading, the starting point would be the user's goal of trying to find how to reach a specific destination. For the sake of this response, we will assume the user is trying to go from his current location to some goal destination. This goal of trying to travel from where they currently are to their destination is the Intention, the meaning of the input expression. The form of the input expression or the Action Specification, is in this case the text that comprises the address or the name of the location. The user has to type this text into a text box, and this text is the form that their goal takes on as an input into the application.

Once the user presses the confirmation button, the application executes and after a short time the user receives an output that represents the information that the user sought, the way to travel from their current location to their destination. Google Maps gives the user a choice of different output representations, but I will focus on the list of directions format. In the list of directions format, the output contains various elements like the numerical distance in miles, the estimated time arrival, and of course, the numbered list of turns, streets, and routes to take to reach the destination. The form of the output expression is thus in the form of text that describes what actions the user has to take in order to achieve the goal of reaching the destination. The meaning of the output expression is perceived by the user through their understanding of written English or another human language. The form of the output is Interpreted as some kind of human language text, and then the user Evaluates this text to extract the information of how to reach their destination.

This concludes one cycle of the loop. Once the user gets information about how to reach their destination, they simply use the information by Evaluating and Interpreting the output. The next time the user uses the application would be if they have the Goal of traveling to a different destination. In that case, the loop would begin anew.

2) A kind of question that Heuristic Evaluation could answer that regular user testing or usability studies cannot are the types of questions that relate to objective measures of usability. These objective measures are defined by the predetermined "heuristics" and should provide a more dispassionate and measurable way of gauging the effectiveness of an interface. Thus, objective questions about how usable the application is are easier to answer because there are criteria through which each evaluator individually measures the usability against. In contrast, traditional user testing is very subject to the biases of the observer. However useful the information gained from the traditional user test, it would need to be filtered for the bias of the observer, who is only one person. The tradeoff here is between putting more bias in the hands of Observers with traditional testing, or putting more bias in the hands of the Evaluators with Heuristic Evaluation. With Evaluators in Heuristic Evaluation, there are more people using the same objective measurement criteria, which would result in more better answers for questions about objective usability.

I think that questions about how the user first perceives the application and how the user first tries to interact with the application are more difficult to answer using Heuristic Evaluation. The reason for this is because the user has to describe their feelings or perceptions of the application on paper after the reaction has taken place, which means the user has to translate the reaction into words. These words may omit detail that otherwise would be present if the Observer was watching and trying to interpret the evaluator's reactions directly. By watching directly, the Observer can look for body language like facial expressions and for ways the user is using the application that might betray what the user is subconsciously thinking, like if the user was trying to use the application fast, it might indicate some kind of frustration.

Tananun Songdechakraiwut - 2/13/2013 14:29:19

1. My example is me interacting with "SoundHound" app. My goal is to identify an unknown song. When the app is launched, there is an icon explicitly labeled "What's that song? Tap here!"(intention). It's pretty obvious here to tap it(action specification) so I do so(form of input expression is executed by me). During the execution, it displays "Listening" which implies it's determining the song. After a few seconds, the app shows the resulting screen with the name of the song on(perception). So I interpret it(Interpretation) and know immediately that it's the song name I wanted to know(Evaluation&ending up at a new goal).

2. One of the advantages of heuristic evaluation is that it uncovers issues that are harder to find in user testing since users barely visit parts of application outside the assigned tasks. Heuristic evaluation can be implemented during early design phase of the interface and thus with helps of experts, identifies problems early. Importantly, it is relatively less time consuming and possibly inexpensive. However, it is less representative of target users and can be biased. On the other hand, user testing can identifies mostly significant problems. Also, more users mean more problems found with the interface. But, again, it can be very time consuming and you could only test a few participants if time-sensitive.

Alysha Jivani - 2/13/2013 14:30:25

(1) Google Maps Goal: Find directions to a Barnes & Noble Bookstore nearby Intention: The meanings of the input expressions are locations in space (i.e. my location and the location of the bookstore) Action Specification: Actual addresses (Street Address, City, Zip Code) to be entered into the application interface Execution: (information entered) Interpretation: the output is in the form of a list of directions, turns, and distances Evaluation: The meaning is a set of directions to take me from one location to another.

(2) Heuristic Evaluation can find lots of the individual usability issues that might be glossed over during user testing since each evaluator investigates a different aspect. However, it cannot really address the overall big picture of the application (while you can get an aggregate of all the individual data points, it is possible that the Gestalt of the entire interface experience could get lost) and it cannot really provide suggestions for re-design like a user in user-testing can.

Samir Makhani - 2/13/2013 14:31:53

The mobile application I choose is Yelp. When I use the Yelp application, my goal is usually to find a restaurant within 2-3 blocks of my location that is open, at least four stars, and has food that I am currently in the mood for (which varies depending on the day). My intention of using this mobile application is to quickly find a food place so I don’t spend too much time making indecisive decisions. My first action specification is to fill out the form to search on Yelp so I can receive results. I do this through the home interface, which presents me a menu with options such as “Nearby,” “About Me,” “Bookmarks” etc. Thus, I click on “Nearby,” and the form of output expression is a list of things that are nearby. I perceive a list of objects, and now I must interpret and evaluate the meaning of this output expression, in which I realize my next goal is to click the button that will take me to the list of restaurants. Some example options after I click on “Nearby” are Restaurants, Bars, Nightlife, Coffee & Tea, etc. My goal is to click restaurant, in which I repeat the cycle again.

I think Heuristic Evaluation can answer interface questions on a higher level, and from a more experienced perspective, because heuristic evaluation allows for the set of evaluators examine the interface and judge compliance with respect to recognized usability principles that are commonly known and have been iterated upon from the past. They can answer questions that do not have a simple, obvious, solution from a higher approach that suggests design general principles that have worked in the past for other interfaces. Because Heuristic Evaluation compares the interface with recognized usability principles, there may be some cases where we want a fresh pair of eyes to look at the interface and provide feedback, with no priori knowledge of these recognized usability principles, just to get another perspective from an individual who is not very family with design principles already.

Eric Ren - 2/13/2013 14:38:35

1) When I use my gmail app on my phone, I see a list of email headers. I interpreted the list as an inbox of emails, and expect to see more as I go scroll down. When I press one of the boxes, I expect to see the full contents of the email, and the app will render out the text as a whole page.

Lemuel Daniel Wu - 2/13/2013 14:38:55

1. One mobile app that I use often is my Paypal application. When I turn it on, my goal is to pay someone money, usually one of my friends to help split the bill. The semantic distance involved is greatly reduced when I use this app, however, as it takes in just 3 pieces of information, and does everything that I want it to do (my friend's name, how much money I want to send, and whether this is a personal reason for sending money or a commercial reason). Thus, as I start to use my app, I first think of who I want to send money to, and how much - this is the formation of my intention. Then, in my action specification, I take my friend's name and put it into the text box for his name and put the amount I want to pay into the next input box. Then, during program execution (which takes at most, a few seconds), the data is sent to Paypal, and they do the actual monetary transaction. At this point, a response page is displayed on my phone. I then interpret the meaning of the page to understand that either the payment did not go through, or that my money was successfully transferred to my friend. My evaluation of the interpreted output then determines my new set of goals: if the transaction was successful, I go on happily with life. However, if the transaction was not successful, then I either try again, or try to find some other way to pay my friend back.

2. Heuristic Evaluation seems to be very good at answering whether one user's problems or complaints are consistent with how most people will react to the application, and seems to also be good at answering how well the application can reduce the burden of memorizing things for the user. It also seems to be a good test for how well the application can stand up to malicious users, because Heuristic Evaluation seems to be focused on recovering from bad input and errors far more than user testing.

Elizabeth Hartoog - 2/13/2013 17:47:09

While interacting with the reddit is fun application I have a goal of choosing a new subreddit (sub forum) to browse. I intend to change to the subreddit r/fun. Therefore, I input into the mobile application when it prompts me for a subreddit with "fun" and submit. What the application returns is a new interface with new threads and information from this subreddit fun where I can now choose to look at the threads presented to me by the screen or choose to browse a new subforum.

By using heuristics to evaluate an application, you can more precisely target and diagnose problems with an application that a user might not even notice. As we read before, users a prone to developing habits. Perhaps a user always goes for the menu dropdown located next to the <- and home buttons and therefore never examines the methods of reaching/handling the menu. This is not the fault of user testing. In heuristic evaluation, the evaluator would take the time to examine every method of accessing the menu and evaluate them each individually against a set of standards.

Heuristic Evaluation by its nature is taking every element of the interface and questioning its purpose and its design. This in depth analysis lets the evaluator have a very keen understanding of the application. This, however, prevents the evaluator from having an "untainted" view of the application. User testers when presented with an application will have no idea how to use the interface and will be left to their devices to figure out how to use the interface. They do not do keen examination and therefore the observer is presented with someone who is prone to mistakes and interface misuse. These are things that are very important to discover. How difficult is the interface to quickly interpret for a new user? What mistakes do they usually make on first encounter? These questions are more easily answered with user testing.