Observe and Adapt

A coworker sent me the following blog after he heard me rant about how most software is engineered to be unusable: http://www.codinghorror.com/blog/

The latest post ("The organism will do whatever it .... well pleases") points out lessons learned from the Lucasfilm's Habitat project. These lessons learned were presented in 1990 at a Cyberspace conference held at UT Austin.

It’s interesting that Randy Farmer and Chip Morningstar reached conclusions on customer behavior 20+ years ago which are applicable today. A couple of passages are highlighted, but the statements below especially resonated with me:

·         “We could influence things, we could set up interesting situations, we could provide opportunities for things to happen, but we could not predict nor dictate the outcome.”

·         “Instead of trying to push the community in the direction we thought it should go, an exercise rather like herding mice, we tried to observe what people were doing and aid them in it.”

A key takeaway is the importance of observing behavior and adapting to it after a system goes live. This idea is embedded in multiple agile development principles. The ironic thing is that most of the projects I worked on in the past seemed to underutilize observation as a tool to understand the possibilities of a system. During the development process, customers watched demos and provided comments. After releases, change requests were collected verbally or in written formats and moved through the ‘change control’ process (Side note: “Change control” is such a misnomer, almost implies people can control changes).

The value of verbal/written impressions may come from the notion that customers can accurately understand their behavior and pinpoint challenges and ways to fix them. There is some truth to this idea – as a customer, I can say with confidence which tasks take up a lot of time (e.g. answering email), which tasks I do more frequently (e.g. filing out time sheets) and which tasks I strongly like / strongly dislike. For things I care about, I invest the time to reflect and identify workarounds.

To translate this to system speak, if I just see the demo of a tool, I provide feedback on whether it looks cool or not. Moreover, when I give feedback after using a system I state aspects that make me very happy or very mad and improvements for the things that made me very mad. This is why most online book reviews seem highly positive or highly negative.

As designers, overall impressions and very good/very bad feedback is somewhat useful but it’s not enough to create a better product - we need to understand the subtleties of behavior so we can make minor tweaks that can significantly improve utility.  A complementary technique is to also use direct observation of behavior to figure out what works and what didn’t. For example:

·         Review logs to see which features are used most often by various personas

·         Observe trends and themes across customer posts / change requests

·         Video record people using your system. Afterwards, view the tape, take detailed notes of what people did, in what sequence, and observe how long each task took and bottlenecks. Use this information to re-design!

I suspect Facebook utilizes the techniques of observation and adaptation to evolve their system. They seem to frequently release new features and adjust based on customer response. Some of their recent changes have frustrated users and increased discussions on privacy tradeoffs, yet people still haven’t stopped using Facebook. However, it’ll be interesting to see if they are using it differently now.