Storyboarding multi-touch interactions

I think it was around half a year ago that I wrote “UX design­ers should get into every­ware”. Back then I did not expect to be part of a ubi­comp project any­time soon. But here I am now, writ­ing about work I did in the area of mul­ti-touch interfaces. 

Background

The peo­ple at InUse (Swe­den’s pre­mier inter­ac­tion design con­sul­tan­cy firm) asked me to assist them with visu­al­is­ing poten­tial uses of mul­ti-touch tech­nol­o­gy in the con­text of a gat­ed com­mu­ni­ty. That’s right—an actu­al real-world phys­i­cal real-estate devel­op­ment project. How cool is that?

InUse storyboard 1

This res­i­den­tial com­mu­ni­ty is aimed at well-to-do seniors. As with most gat­ed com­mu­ni­ties, it offers them con­ve­nience, secu­ri­ty and pres­tige. You might shud­der at the thought of liv­ing in one of these places (I know I have my reser­va­tions) but there’s not much use in judg­ing peo­ple want­i­ng to do so. Planned ameni­ties include sports facil­i­ties, fine din­ing, onsite med­ical care, a cin­e­ma and on and on…

Social capital

One of the known issues with these ‘com­mu­ni­ties’ is that there’s not much evi­dence of social cap­i­tal being high­er there than in any reg­u­lar neigh­bour­hood. In fact some have argued that the glob­al trend of gat­ed com­mu­ni­ties is detri­men­tal to the build-up of social cap­i­tal in their sur­round­ings. They throw up phys­i­cal bar­ri­ers that pre­vent free inter­ac­tion of peo­ple. These are some of the things I tried to address: To see if we could sup­port the emer­gence of com­mu­ni­ty inside the res­i­den­cy using social tools while at the same coun­ter­act­ing phys­i­cal bar­ri­ers to the out­side world with “vir­tu­al inroads” that allow for free inter­ac­tion between res­i­dents and peo­ple in the periphery.

Being in the world

Anoth­er con­cern I tried to address is the dif­fer­ent ways mul­ti-touch inter­faces can play a role in the lives of peo­ple. Recent­ly Matt Jones addressed this in a post on the iPhone and Noki­a’s upcom­ing mul­ti-touch phones. In a com­mu­ni­ty like the one I was design­ing for, the worst thing I could do is make every instance of mul­ti-touch tech­nol­o­gy an atten­tion-grab­bing pres­ence demand­ing full immer­sion from its user. In many cas­es ‘my’ users would be bet­ter served with them behav­ing in an unob­tru­sive way, allow­ing almost uncon­scious use. In oth­er words: I tried to bal­ance being in the world with being in the screen—apply­ing each par­a­digm based on how appro­pri­ate it was giv­en the user’s con­text. (After all, some­times peo­ple want or even need to be immersed.)

Process

InUse had already pre­pared sev­er­al per­sonas rep­re­sen­ta­tive of the future res­i­dents of the com­mu­ni­ty. We went through those togeth­er and exam­ined each for sce­nar­ios that would make good can­di­dates for sto­ry­board­ing. We want­ed to come up with a range of sce­nar­ios that not only showed how these per­sonas could be sup­port­ed with mul­ti-touch inter­faces, but also illus­trate the dif­fer­ent spaces the inter­ac­tions could take place in (pri­vate, semi­pri­vate and pub­lic) and the scales at which the tech­nol­o­gy can oper­ate (from small key-like tokens to full wall-screens). 

InUse storyboard 2

I draft­ed each sce­nario as a tex­tu­al out­line and sketched the poten­tial sto­ry­boards on thumb­nail size. We went over those in a sec­ond work­shop and refined them—making adjust­ments to bet­ter cov­er the con­cerns out­lined above as well as improv­ing clar­i­ty. We want­ed to end up with a set of sto­ry­boards that could be used in a pre­sen­ta­tion for the client (the real-estate devel­op­ment firm) so we need­ed to bal­ance user goals with busi­ness objec­tives. To that end we thought about and includ­ed exam­ples of API-like inte­gra­tion of the plat­form with ser­vice providers in the periph­ery of the com­mu­ni­ty. We also tried to cre­ate self-ser­vice expe­ri­ences that would feel like being wait­ed on by a per­son­al butler.

Outcome

I end­ed up draw­ing three sce­nar­ios of around 9 pan­els each, digi­tis­ing and clean­ing them up on my Mac. Each sce­nario intro­duces a per­sona, the phys­i­cal con­text of the inter­ac­tion and the per­son­a’s moti­va­tion that dri­ves him to engage with the tech­nol­o­gy. The inter­ac­tions visu­alised are a mix of ges­tures and engage­ments with mul­ti-touch screens of dif­fer­ent sizes. Usu­al­ly the per­sona is sup­port­ed in some way by a social dimension—fostering serendip­i­ty and emer­gence of real relations.

InUse storyboard 3

All in all I have to say I am pret­ty pleased with the result of this short but sweet engage­ment. Col­lab­o­ra­tion with the peo­ple of InUse was smooth (as was expect­ed, since we are very much the same kind of ani­mal) and there will be fol­low-up work­shops with the client. It remains to be seen how much of this mul­ti-touch stuff will find its way into the final gat­ed com­mu­ni­ty. That as always will depend on what makes busi­ness sense. 

In any case it was a great oppor­tu­ni­ty for me to immerse myself ful­ly in the inter­re­lat­ed top­ics of mul­ti-touch, ges­ture, urban­ism and social­i­ty. And final­ly, it gave me the per­fect excuse to sit down and do lots and lots of drawings.

UX designers should get into everyware

I’ve been read­ing Adam Greenfield’s Every­ware on and off and one of the things that it has me won­der­ing the most late­ly is: are UX pro­fes­sion­als mak­ing the move to design for ubiq­ui­tous computing?

There’re sev­er­al places in the book where he explic­it­ly men­tions UX in rela­tion to every­ware. Let’s have a look at the ones I man­aged to retrieve using the book’s trusty index…

On page 14 Green­field writes that with the emer­gence of ubi­comp at the dawn of the new mil­len­ni­um, the user expe­ri­ence com­mu­ni­ty took up the chal­lenge with “vary­ing degrees of enthu­si­asm, scep­ti­cism and crit­i­cal dis­tance”, try­ing to find a “lan­guage of inter­ac­tion suit­ed to a world where infor­ma­tion pro­cess­ing would be every­where in the human environment.” 

So of course the UX com­mu­ni­ty has already start­ed con­sid­er­ing what it means to design for ubi­comp. This stuff is quite dif­fer­ent to inter­net appli­ances and web sites though, as Green­field points out in the­sis 09 (pp.37–39):

Con­sis­tent­ly elic­it­ing good user expe­ri­ences means account­ing for the phys­i­cal design of the human inter­face, the flow of inter­ac­tion between user and device, and the larg­er con­text in which that inter­ac­tion is embed­ded. In not a sin­gle one of these dimen­sions is the expe­ri­ence of every­ware any­thing like that of per­son­al com­put­ing.” (p.37)

That’s a clear state­ment, on which he elab­o­rates fur­ther on, men­tion­ing that tra­di­tion­al inter­ac­tions are usu­al­ly of a “call-and-response rhythm: user actions fol­lowed by sys­tem events.” Where­as every­ware inter­ac­tions “can’t mean­ing­ful­ly be con­struct­ed as ‘task-dri­ven.’ Nor does any­thing in the inter­play between user and sys­tem […] cor­re­spond with […] infor­ma­tion seek­ing.” (p.38)

So, UX design­ers mov­ing into every­ware have their work cut out for them. This is vir­gin territory:

[…] it is […] a rad­i­cal­ly new sit­u­a­tion that will require the devel­op­ment over time of a doc­trine and a body of stan­dards and con­ven­tions […]” (p.39)

Now, UX in tra­di­tion­al projects has been prone to what Green­field calls ‘val­ue engi­neer­ing’. Com­mer­cial projects can only be two of these three things: fast, good and cheap. UX would sup­port the sec­ond, but sad­ly it is often sac­ri­ficed for the sake of the oth­er two. Not always though, but this is usu­al­ly depen­dent on who is involved with the project:

[…] it often takes an unusu­al­ly ded­i­cat­ed, per­sis­tent, and pow­er­ful advo­cate […] to see a high-qual­i­ty design project through to com­ple­tion with every­thing that makes it excel­lent intact. […] the painstak­ing­ly detailed work of ensur­ing a good user expe­ri­ence is fre­quent­ly hard to jus­ti­fy on a short-term ROI basis, and this is why it is often one of the first things to get val­ue-engi­neered out of an extend­ed devel­op­ment process. […] we’ve seen that get­ting every­ware right will be orders of mag­ni­tude more com­pli­cat­ed than achiev­ing accept­able qual­i­ty in a Web site, […] This is not the place for val­ue engi­neers,” (p.166)

So if tra­di­tion­al projects need UX advo­cates on board with con­sid­er­able influ­ence, com­pa­ra­ble to Steve Jobs’s role at Apple, to ensure a descent user expe­ri­ence will it even be pos­si­ble to cre­ate ubiq­ui­tous expe­ri­ences that are enjoy­able to use? If these projects are so com­plex, can they be even got­ten ‘right’ in a com­mer­cial con­text? I’m sor­ry to say I think not…

Design­ers (used broad­ly) will be at the fore­front of decid­ing what every­ware looks like. If you don’t think they will, at least I’m sure they should. They’re not the only ones to deter­mine its shape though, Green­field points out that both reg­u­la­tors and mar­kets have impor­tant parts to play too (pp.172–173):

[…] the inter­lock­ing influ­ences of design­er, reg­u­la­tor, and mar­ket will be most like­ly to result in ben­e­fi­cial out­comes if these par­ties all treat every­ware as a present real­i­ty, and if the deci­sion mak­ers con­cerned act accord­ing­ly.” (p.173)

Now there’s an inter­est­ing notion. Hav­ing just come back from a pre­mier venue for the UX com­mu­ni­ty to talk about this top­ic, the IA Sum­mit, I’m afraid to say that I didn’t get the impres­sion IAs are tak­ing every­ware seri­ous­ly (yet.) There were no talks real­ly con­cerned with tan­gi­ble, per­va­sive, ubiq­ui­tous or ambi­ent tech­nolo­gies. Some basic fare on mobile web stuff, that’s all. Wor­ry­ing, because as Green­field points out:

[UX design­ers] will best be able to inter­vene effec­tive­ly if they devel­op appro­pri­ate insights, tools, and method­olo­gies ahead of the actu­al deploy­ment of ubiq­ui­tous sys­tems.” (pp.173–174)

This stuff is real, and it is here. Green­field points to the exis­tence of sys­tems such as Octo­pus in Hong Kong and E‑ZPass in the US. Hon­est­ly, if you think beyond the tools and meth­ods we’ve been using to com­mu­ni­cate our designs, IxDs and IAs are well-equipped to han­dle every­ware. No, you won’t be required to draw wire­frames or sitemaps; but you’ll damn well need to put in a lot of the think­ing design­ers do. And you’ll still need to be able to com­mu­ni­cate those designs. It’s time to get our hands dirty:

What ful­ly oper­a­tional sys­tems such as Octo­pus and E‑ZPass tell us is that pri­va­cy con­cerns, social impli­ca­tions, eth­i­cal ques­tions, and prac­ti­cal details of the user expe­ri­ence are no longer mat­ters for con­jec­ture or sup­po­si­tion. With ubiq­ui­tous sys­tems avail­able for empir­i­cal enquiry, these things we need to focus on today.” (p.217)

So, to reit­er­ate the ques­tion I start­ed with: are there any UX design­ers out there that have made the switch from web-work to ubi­comp? Any­one con­sid­er­ing it? I’d love to hear about your experiences.