USABILITY + ROI
Summary:
I used 4 different papers to evaluate findings that relate to the topic of return on investment (ROI) and usability. These papers include Usability for Internal Systems: What’s the Payoff, Technology and the Return on Investment, Cost-Justifying Usability Engineering: A Real World Example, and Return on Investment for Usable User-Interface Design: Examples and Statistics.
Seth Earley in Usability for Internal Systems: What’s the Payoff? connects poor user experience with poor data hygiene on the part of the organization. If the company does not provide its employees with usable systems then the inevitable frustration that an employee encounters will trickle down and be felt by the customer as well. Investing in these systems to improve internal processes leads to greater efficiency, ultimately “[enhancing] the customer journey.”
Karen Coyle wrote of a similar issue in Technology and the Return on Investment. An important and interesting point she makes is how malleable the terms ‘return’ and ‘investment’ are. These terms can mean whatever the organization is looking for them to mean; by pointing this out she allows ROI to be a concept that can fit a library’s needs. ROI when used to look at libraries can be understood as costs and benefits since libraries do not operate to make a profit. In this scenario the return (or benefit) is “doing something better than before.”
In Cost-Justifying Usability Engineering: A Real World Example the study set out to show that prioritizing usability when creating a design would have the most impact on cost. Two usability tests were done; the baseline was set with the first usability test which had a set of participants interact with the current (pre-redesign) interface. The second usability test had another set of participants interact with the interface after it was redesigned using findings from the first study. The findings showed a decrease in errors (2.4 to 0.8) and decrease in time spent on task (83 seconds to 36 seconds) with the improved window, and both of these factors lead to less time needed for training. Saving the end-user (whether that be an employee or the customer) time necessarily involves the information systems that act as an infrastructure for an organization. Making systems more user-centered through findings from usability testing appears to garner a significant cost benefit.
The paper Return on Investment for Usable User-Interface Design: Examples and Statistics sets out to debunk the idea held by “Most software and Website development managers [that] usability costs [are an] added effort and expense.” A driving factor for this is that it is more costly to fix a problem down the line. He writes, “Once a system is in development, correcting a problem costs 10 times as much as fixing the same problem in design. If the system has been released, it costs 100 times as much relative to fixing in design.” It is the underlying infrastructure of a product or system that has a rippling or trickle down effect on the end user.
Findings:
Return on investment when related to usability can be understood in two different ways: internal and external. A project that incorporates usability aspects will see a benefit; benefits may include greater productivity, fewer errors, and less time and effort needed for training. The two key factors that I’ve gleaned from the papers I looked at are: usability techniques when implemented at the beginning of a project will yield the highest benefit and usability techniques effect the underlying infrastructure of information. The underlying infrastructure of information determines how many errors and how much time an employee or customer will spend interacting with an interface, ultimately determining the cost for an organization. If an organization is not aware of the benefits of incorporating usability techniques from the beginning of a project then the costs of fixing a product can be insurmountable.
The Paper:
Return on investment (ROI) when related to usability can be understood in two different ways: internal and external. A project that incorporates usability aspects will see a benefit; benefits may include greater productivity, fewer errors, and less time and effort needed for training.[1] I will be using different papers to evaluate findings that relate to the topic of ROI and usability. These papers include Usability for Internal Systems: What’s the Payoff, Technology and the Return on Investment, Cost-Justifying Usability Engineering: A Real World Example, and Return on Investment for Usable User-Interface Design: Examples and Statistics.
Seth Earley in Usability for Internal Systems: What’s the Payoff? draws attentions to the trend of increasing expectations on the part of consumers. By discussing this scenario he describes the factors that go into a poor user experience, namely poor data hygiene on the part of the organization. By developing better—more intuitive, easier to use—internal systems, then there will be “adaptability, agility, and greater customer intimacy.”[2] Earley’s point is that the customer experience goes beyond a company’s face or frontend, it includes when a customer may contact the company because of an issue with an order. If the company does not provide its employees with usable systems then the inevitable frustration that an employee encounters will trickle down and be felt by the customer as well.
Easier access to information doesn’t provide business value unless there’s corresponding improvement in efficiency or a new capability. Organizations are often hard pressed to justify the significant investments required to address enterprise-wide applications when the outcome is uncertain and the payoff unclear.[3]
Earley’s hypothesis is that organizations must invest in areas that rarely see change, i.e. systems that are used to structure information. Improving internal processes leads to greater efficiency, ultimately “[enhancing] the customer journey.”[4]
This article discusses IT organizations specifically and it is important to point out the troubling notion that information technology organizations do not invest in their information systems. However, this is not a new issue. Earley wrote his article in 2014, but Karen Coyle wrote of a similar issue in 2006. Coyle writes in Technology and the Return on Investment on libraries and makes clear their need for information services as they operate “in an interlocking web of information services.”[5] It is irksome that it took 8 years for the private IT sector to catch up to an often-denigrated public institution. I am surprised because I would have thought that a private sector organization would be able to assess better their internal funding strategy but I suppose that is what makes ‘return on investment’ an interesting topic.
Coyle attempts to explain ROI and how easily manipulated both ‘return’ and ‘investment’ can be. These terms can mean whatever the organization is looking for them to mean; by pointing this out she allows ROI to be a concept that can fit a library’s needs.
The business calculation of Return on Investment (ROI) is based on a view of investment that is deeply embedded in the concepts of capitalism: you invest capital to create profit. […] It is one of the few ways that we have to measure costs against benefits. There are many different things ROI can measure, mainly because you can define “investment” and “return” to suit your situation. […] Because we can define our own concepts of investment and return, it is possible that we can use ROI for library activities. [This] general concept of costs and benefits […] is used in contexts that are relevant to libraries and their service functions.[6]
ROI when used to look at libraries can be understood as costs and benefits since libraries do not operate to make a profit. In this scenario the return (or benefit) is “doing something better than before.”[7] Just as investment and return are fluid terms and can be molded to the library’s specific situation, looking at this concept as fluid allows ROI to be understood as relating to services and social benefit as well. Because of this ROI can relate and be tied to usability.
In Cost-Justifying Usability Engineering: A Real World Example the usability aspect includes time on task when participants interact with different graphical user interfaces (GUIs). The study set out to show that prioritizing usability when creating a design would have the most impact on cost. The baseline was set by having one set of participants interact with the current (pre-redesign) interface and another set of participants interact with the interface after it was redesigned. The participants in the groups did not overlap but were all employed at the same company.
An interesting piece of information addressed in this study was that it presented different usability goals as conflicting. I think this is to emphasize how complicated it is to design a usable interface that also includes the many factors that the business needs or wants it to include. The designers of the GUI needed to “[include] functionality, performance, cost, reliability, maintenance, and the quality of the user interface”[8] in their creation. Furthermore GUI development exhausts 30% of a project’s budget and a substantial amount of time.[9]
The study utilized two usability studies, one of the original GUI and a second with “subtle”[10] tweaks. The findings showed that there was a decrease in errors (2.4 to 0.8) and decrease in time spent on task (83 seconds to 36 seconds) with the improved GUI window, and both of these factors lead to less time needed for training.
We assumed this hypothetical market typically uses the account window 1,150 times per day by 50 CSRs (1000 customer service calls and 150 credit and activation requests). The analysis indicated a savings of $23,540 while incurring a cost of $2,676. Both the software developer and the end-user share the resulting $20,864 net benefit from this single window change.[11]
The findings from this study echo Earley’s sentiments. Saving the end-user (whether that be an employee or the customer) time necessarily involves the information systems that act as an infrastructure for an organization. In this study the information systems are navigated through a GUI, the GUI dictates the data that the user can access and is faced with. Making systems more user-centered through findings from usability testing appears to garner a significant cost benefit.
Aaron Marcus thoroughly explains the benefit of usability through ROI in Return on Investment for Usable User-Interface Design: Examples and Statistics. The paper sets out to debunk the idea held by major industry players that “Most software and Website development managers view usability costs as added effort and expense[.]”[12] The crucial factor for the greatest gain (or return) is that usability efforts be targeted at the beginning of a project, whether that project is for internal (employee) or external (client) use. The following list shows the benefits:
Internal ROI
• Increased user productivity
• Decreased user errors
• Decreased training costs
• Savings gained from making changes earlier in design life cycle
• Decreased user support
External ROI
• Increased sales
• Decreased customer support costs
• Savings gained from making changes earlier in the design life cycle
• Reduced cost of providing training (if training is offered through the vendor company[)] (Bias & Mayhew, 1994)[13]
Involving usability techniques from the beginning reaps the most benefit; a major reason for this is that it is more costly to fix a problem in the development stage rather than the design stage (design comes before development).[14]
Once a system is in development, correcting a problem costs 10 times as much as fixing the same problem in design. If the system has been released, it costs 100 times as much relative to fixing in design. (Gilb, 1988)[15]
It is easier to incorporate usability into the design, as the design stage is most flexible. By featuring usability techniques as a requirement (along with the requirements submitted by the client) then the design can be more efficient and have a clearer focus (or purpose). This point speaks to previous takeaways gleaned from the papers I previously looked at. It is the underlying infrastructure of a product or system that has a rippling or trickle down effect on the end user. The beginning stage of a project addresses the way information will be organized, this ultimately affects the end-user and whomever they may be interacting with.
The two key factors that I’ve gleaned from the papers I looked at are: usability techniques when implemented at the beginning of a project will yield the highest benefit and usability techniques effect the underlying infrastructure of information. The underlying infrastructure of information determines how many errors and how much time an employee or customer will spend interacting with an interface, ultimately determining the cost for an organization. The myth addressed by Aaron Marcus makes sense; if an organization is not aware of the benefits of incorporating usability techniques from the beginning of a project then the costs of fixing a product can be insurmountable.
Bibliography:
Cope, Michael E. and Uliano, Kevin C. 1995. “Cost-Justifying Usability Engineering: A Real World Example.” Proceedings of the Human Factors and Ergonomics Society 39th Annual Meeting: 263-267.
Coyle, Karen. 2006. “Technology and the Return on Investment.” The Journal of Academic Librarianship, 32(5): 537-539.
Earley, Seth. 2014. “Usability for Internal Systems: What’s the Payoff?” IT Pro: 66-69.
Marcus, Aaron. 2004. “Return on Investment for Usable User-Interface Design: Examples and Statistics.” AM+A: 1-23.
Endnotes:
[1] Marcus, Aaron. 2004. “Return on Investment for Usable User-Interface Design: Examples and Statistics.” AM+A: 2.
[2] Earley, Seth. 2014. “Usability for Internal Systems: What’s the Payoff?” IT Pro: 69.
[3] Earley, Seth. 2014. “Usability for Internal Systems: What’s the Payoff?” IT Pro: 66
[4] Earley, Seth. 2014. “Usability for Internal Systems: What’s the Payoff?” IT Pro: 68
[5] Coyle, Karen. 2006. “Technology and the Return on Investment.” The Journal of Academic Librarianship, 32(5): 537.
[6] Coyle, Karen. 2006. “Technology and the Return on Investment.” The Journal of Academic Librarianship, 32(5): 537.
[7] Coyle, Karen. 2006. “Technology and the Return on Investment.” The Journal of Academic Librarianship, 32(5): 537.
[8] Cope, Michael E. and Uliano, Kevin C. 1995. “Cost-Justifying Usability Engineering: A Real World Example.” Proceedings of the Human Factors and Ergonomics Society 39th Annual Meeting: 263.
[9] Cope, Michael E. and Uliano, Kevin C. 1995. “Cost-Justifying Usability Engineering: A Real World Example.” Proceedings of the Human Factors and Ergonomics Society 39th Annual Meeting: 263.
[10] Cope, Michael E. and Uliano, Kevin C. 1995. “Cost-Justifying Usability Engineering: A Real World Example.” Proceedings of the Human Factors and Ergonomics Society 39th Annual Meeting: 263.
[11] Cope, Michael E. and Uliano, Kevin C. 1995. “Cost-Justifying Usability Engineering: A Real World Example.” Proceedings of the Human Factors and Ergonomics Society 39th Annual Meeting: 265.
[12] Marcus, Aaron. 2004. “Return on Investment for Usable User-Interface Design: Examples and Statistics.” AM+A: 2.
[13] Marcus, Aaron. 2004. “Return on Investment for Usable User-Interface Design: Examples and Statistics.” AM+A: 2.
[14] Marcus, Aaron. 2004. “Return on Investment for Usable User-Interface Design: Examples and Statistics.” AM+A: 4.
[15] Marcus, Aaron. 2004. “Return on Investment for Usable User-Interface Design: Examples and Statistics.” AM+A: 4.