Evaluation methods for interaction design

In this article, I am going to describe 5 Evaluation methods.

1. Heuristic evaluation
2. Walk-throughs
3. Web analytics
4. A/B Testing
5. Predictive Models

Heuristic evaluation

Heuristic evaluation is a process where experts use rules of thumb to measure the usability of user interfaces in independent walkthroughs and report issues. Evaluators use established heuristics and reveal insights that can help design teams enhance product usability from early in development.

How to conduct a Heuristic evaluation

1. Know what to test and how — Whether it’s the entire product or one procedure, clearly define the parameters of what to test and the objective.
2. Know your users and have clear definitions of the target audience’s goals, contexts, etc. User personas can help evaluators see things from the users’ perspectives.
3. Select 3–5 evaluators, ensuring their expertise in usability and the relevant industry.
4. Define the heuristics (around 5–10) — This will depend on the nature of the system/product/design. Consider adopting/adapting the Nielsen-Molich heuristics and/or using/defining others.
5. Brief evaluators on what to cover in a selection of tasks, suggesting a scale of severity codes (e.g., critical) to flag issues.
6. 1st Walkthrough — Have evaluators use the product freely so they can identify elements to analyze.
7. 2nd Walkthrough — Evaluators scrutinize individual elements according to the heuristics. They also examine how these fit into the overall design, clearly recording all issues encountered.
8. Debrief evaluators in a session so they can collate results for analysis and suggest fixes.

Pros

• It is a detailed, technically sound process that assesses the product against very clear criteria.
• Because it is done by several people there is a better chance of getting a range of views and picking up more potential problem areas.
• The very act of setting up the heuristic evaluation is a useful exercise as it forces you to identify the root elements of the product and focuses development on the main issues.
• There are fewer practical and ethical issues attached to heuristic evaluation as testers are testing in a virtual space.
• Heuristic evaluation tends to focus on fewer, more relevant areas so the problems it identifies tend to be important ones.

Cons

• The evaluation is only as good as the people you get to do it. This means you have to spend a lot of time analyzing and reviewing experts to make sure they are relevant and experienced in the issues you are concerned with.
• A number of experts are required and this can be time-consuming and expensive to research and set up.
• You are getting opinions and personal observation rather than hard, empirical data from the exercise, and the experts’ own background, attitudes, preferences might colour the verdicts.
• You have to do a good deal of analysis and thinking to make sure you choose the right heuristics in the first place. If this is wrong, no matter how good the experts are, you are likely to get less than optimum results.
• Often the problems identified are not critical (or even real in some cases).

Walk-Through

In software testing, a walkthrou gh is a method of reviewing documents with peers, managers, and other team members while being guided by the document’s author to obtain input and establish a consensus. A walkthrough might be scheduled ahead of time or organized dependent on the need. The walkthrough process usually involves colleagues who are working on the same project.

The audience is selected from different backgrounds in order to have a diverse point of view and thus, provide different dimensions to a common objective. This is not a formal process but is specifically used for high-level documents like requirement specifications or functional specifications, etc.

Web analytics

Web analytics is the collection, reporting, and analysis of website data. The focus is on identifying measures based on your organizational and user goals and using the website data to determine the success or failure of those goals and to drive strategy and improve the user’s experience.

Web analytic tools

1.Content analytics tools

2.Customer analytics tools

3.Usability (UX) analytics tools

4.A/B and multivariate testing tools

5.Social media analytics tools

6.SEO analytics tools

7.General enterprise analytics tools

8.Open source web analytics tools

9.Product analytics tools

Web analytics tools collect data to show you how visitors arrive at your website and what they do once they’re there. These tools let you compare data over time to see patterns. This data also lets you measure performance against benchmarks and goals to see how your website is performing, where performance can be improved, and the effects of the actions you take to improve it.

Some of the things that website analytics tools can tell you include:

• How do people find your site? What do they do after they get there?
• Which content on your site do people engage with? When and how are they engaging with it?
• Why do some people buy and others don’t? How can you get more of them to take action?

A/B Testing

A/B testing, also known as split testing, refers to a randomized experimentation process wherein two or more versions of a variable (web page, page element, etc.) are shown to different segments of website visitors at the same time to determine which version leaves the maximum impact and drive business metrics.

A/B testing, in essence, removes all of the guesswork from website optimization and allows experienced optimizers to make data-driven judgments. The ‘control’ or original testing variable is referred to as A in A/B testing. B stands for ‘variant,’ which is a new version of the original testing variable.

The ‘winner’ is the version that improves your company metric(s) in a good way. Implementing the successful variation’s adjustments on your tested page(s) / element(s) can help you optimize your website and increase your business’s ROI.

Each website’s conversion stats are distinct. For example, in the case of eCommerce, it may be product sales. In the meanwhile, it may be the creation of qualified leads for B2B.

Predictive Models

Predictive modeling is a statistical technique using machine learning and data mining to predict and forecast likely future outcomes with the aid of historical and existing data. It works by analyzing current and historical data and projecting what it learns on a model generated to forecast likely outcomes. Predictive modeling can be used to predict just about anything, from TV ratings and a customer’s next purchase to credit risks and corporate earnings.

Types of Predictive Models

1. Classification model: Considered the simplest model, it categorizes data for simple and direct query responses. An example use case would be to answer the question “Is this a fraudulent transaction?”
2. Clustering model: This model nests data together by common attributes. It works by grouping things or people with shared characteristics or behaviors and plans strategies for each group at a larger scale. An example is in determining credit risk for a loan applicant based on what other people in the same or a similar situation did in the past.
3. Forecast model: This is a very popular model, and it works on anything with a numerical value based on learning from historical data. For example, in answering how much lettuce a restaurant should order next week or how many calls a customer support agent should be able to handle per day or week, the system looks back to historical data.
4. Outliers model: This model works by analyzing abnormal or outlying data points. For example, a bank might use an outlier model to identify fraud by asking whether a transaction is outside of the customer’s normal buying habits or whether an expense in a given category is normal or not. For example, a $1,000 credit card charge for a washer and dryer in the cardholder’s preferred big box store would not be alarming, but $1,000 spent on designer clothing in a location where the customer has never charged other items might be indicative of a breached account.
5. Time series model: This model evaluates a sequence of data points based on time. For example, the number of stroke patients admitted to the hospital in the last four months is used to predict how many patients the hospital might expect to admit next week, next month, or the rest of the year. A single metric measured and compared over time is thus more meaningful than a simple average.