Hardware interfaces for tuning the feel of microinteractions

In Dig­i­tal Ground Mal­colm McCul­lough talks about how tun­ing is a cen­tral part of inter­ac­tion design prac­tice. How part of the chal­lenge of any project is to get to a point where you can start tweak­ing the vari­ables that deter­mine the behav­iour of your inter­face for the best feel.

Feel” is a word I bor­row from game design. There is a book on it by Steve Swink. It is a fun­ny term. We are try­ing to sim­u­late sen­sa­tions that are derived from the phys­i­cal realm. We are try­ing to make things that are pure­ly visu­al behave in such a way that they evoke these sen­sa­tions. There are many games that heav­i­ly depend on get­ting feel right. Basi­cal­ly all games that are built on a physics sim­u­la­tion of some kind require good feel for a good play­er expe­ri­ence to emerge.

Physics sim­u­la­tions have been find­ing their way into non-game soft­ware prod­ucts for some time now and they are becom­ing an increas­ing part of what makes a prod­uct, er, feel great. They are often at the foun­da­tion of sig­na­ture moments that set a prod­uct apart from the pack. These sig­na­ture moments are also known as microin­t­er­ac­tions. To get them just right, being able to tune well is very impor­tant.

The behav­iour of microin­t­er­ac­tions based on physics sim­u­la­tions is deter­mined by vari­ables. For exam­ple, the feel of a spring is deter­mined by the mass of the weight attached to the spring, the spring’s stiff­ness and the fric­tion that resists the motion of the weight. These vari­ables inter­act in ways that are hard to mod­el in your head so you need to make repeat­ed changes to each vari­able and try the sim­u­la­tion to get it just right. This is time-con­sum­ing, cum­ber­some and resists the easy explo­ration of alter­na­tives essen­tial to a good design process.

In The Set­up game design­er Ben­nett Fod­dy talks about a way to improve on this work­flow. Many of his games (if not all of them) are playable physics sim­u­la­tions with pun­ish­ing­ly hard con­trols. He sug­gests using a hard­ware inter­face (a MIDI con­troller) to tune the vari­ables that deter­mine the feel of his game while it runs. In this way the loop between chang­ing a vari­able and see­ing its effect in game is dra­mat­i­cal­ly short­ened and many dif­fer­ent com­bi­na­tions of val­ues can be explored eas­i­ly. Once a sat­is­fac­to­ry set of val­ues for the vari­ables has been found they can be writ­ten back to the soft­ware for future use.

I do believe such a set­up is still non-triv­ial to make work with todays tools. A quick check ver­i­fies that Framer does not have OSC sup­port, for exam­ple. There is an oppor­tu­ni­ty here for pro­to­typ­ing envi­ron­ments such as Framer and oth­ers to sup­port it. The approach is not lim­it­ed to motion-based microin­t­er­ac­tions but can be extend­ed to the tun­ing of vari­ables that con­trol oth­er aspects of an app’s behav­iour.

For exam­ple, when we were mak­ing Stand­ing, we would have ben­e­fit­ed huge­ly from hard­ware con­trols to tweak the sen­si­tiv­i­ty of its motion-sens­ing func­tions as we were using the app. We were forced to do it by repeat­ed­ly chang­ing num­bers in the code and build­ing the app again and again. It was quite a pain to get right. To this day I have the feel­ing we could have made it bet­ter if only we would have had the tools to do it.

Judg­ing from sna­fus such as the poor feel of the lat­est Twit­ter desk­top client, there is a real need for bet­ter tools for tun­ing microin­t­er­ac­tions. Just like pen tablets have become indis­pens­able for those design­ing the form of user inter­faces on screens. I think we might soon find a small set of hard­ware knobs on the desks of those design­ers work­ing on the behav­iour of user inter­faces.

Design without touching the surface

I am prepar­ing two class­es at the moment. One is an intro­duc­tion to user expe­ri­ence design, the oth­er to user inter­face design. I did not come up with this divi­sion, it was part of the assign­ment. I thought it was odd at first. I wasn’t sure where one dis­ci­pline ends and the oth­er begins. I still am not sure. But I made a prag­mat­ic deci­sion to have the UX class focus on the high lev­el process of design­ing (soft­ware) prod­ucts, and the UI class focus on the visu­al aspects of a product’s inter­face. The UI class deals with a product’s sur­face, form, and to some extent also its behav­iour, but on a micro lev­el. Where­as the UX class focus­es on behav­iour on the macro lev­el. Sim­ply speaking—the UX class is about behav­iour across screens, the UI class is about behav­iour with­in screens.

The solu­tion is work­able. But I am still not entire­ly com­fort­able with it. I am not com­fort­able with the idea of being able to prac­tice UX with­out ‘touch­ing the sur­face’, so to speak. And it seems my two class­es are advo­cat­ing this. Also, I am pret­ty sure this is every­day real­i­ty for many UX prac­ti­tion­ers. Notice I say “prac­ti­tion­er”, because I am not sure ‘design­er’ is the right term in these cas­es. To be hon­est I do not think you can prac­tice design with­out doing sketch­ing and pro­to­typ­ing of some sort. (See Bill Buxton’s ‘Sketch­ing User Expe­ri­ences’ for an expand­ed argu­ment on why this is.) And when it comes to design­ing soft­ware prod­ucts this means touch­ing the sur­face, the form.

Again, the real­i­ty is, ‘UX design­er’ and ‘UI design­er’ are com­mon terms now. Cer­tain­ly here in Sin­ga­pore peo­ple know they need both to make good prod­ucts. Some prac­ti­tion­ers say they do both, oth­ers one or the oth­er. The lat­ter appears to be the most com­mon and expect­ed case. (By the way, in Sin­ga­pore no-one I’ve met talks about inter­ac­tion design.)

My con­cern is that by encour­ag­ing the prac­tice of doing UX design with­out touch­ing the sur­face of a prod­uct, we get shit­ty designs. In a process where UX and UI are seen as sep­a­rate things the risk is one comes before the oth­er. The UX design­er draws the wire­frames, the UI design­er gets to turn them into pret­ty pic­tures, with no back-and-forth between the two. An iter­a­tive process can mit­i­gate some of the dam­age such an arti­fi­cial divi­sion of labour pro­duces, but I think we still start out on the wrong foot. I think a bet­ter prac­tice might entail includ­ing visu­al con­sid­er­a­tions from the very begin­ning of the design process (as we are sketch­ing).

Two things I came across as I was prepar­ing these class­es are some­how in sup­port of this idea. Both result­ed from a call I did for resources on user inter­face design. I asked for books about visu­al aspects, but I got a lot more.

  1. In ‘Mag­ic Ink’ Bret Vic­tor writes about how the design of infor­ma­tion soft­ware is huge­ly indebt­ed to graph­ic design and more specif­i­cal­ly infor­ma­tion design in the tra­di­tion of Tufte. (He also men­tions indus­tri­al design as an equal­ly big prog­en­i­tor of inter­ac­tion design, but for soft­ware that is main­ly about manip­u­la­tion, not infor­ma­tion.) The arti­cle is big, but the start of it is actu­al­ly a pret­ty good if unortho­dox gen­er­al intro­duc­tion to inter­ac­tion design. For soft­ware that is about learn­ing through look­ing at infor­ma­tion Vic­tor says inter­ac­tion should be a last resort. So that leaves us with a task that is 80% if not more visu­al design. Touch­ing the sur­face. Which makes me think you might as well get to it as quick­ly as pos­si­ble and start sketch­ing and pro­to­typ­ing aimed not just at struc­ture and behav­iour but also form. (Hat tip to Pieter Diepen­maat for this one.)

  2. In ‘Jump­ing to the End’ Matt Jones ram­bles enter­tain­ing­ly about design fic­tion. He argues for pay­ing atten­tion to details and that a lot of the design he prac­tices is about ‘sig­na­ture moments’ aka micro-inter­ac­tions. So yeah, again, I can’t imag­ine design­ing these effec­tive­ly with­out doing sketch­ing and pro­to­typ­ing of the sort that includes the visu­al. And in fact Matt men­tions this more or less at one point, when he talks about the fact that his team’s deliv­er­ables at Google are almost all visu­al. They are high fideli­ty mock­ups, ani­ma­tions, videos, and so on. These then become the start­ing points for fur­ther devel­op­ment. (Hat tip to Alexan­der Zeh for this one.)

In sum­ma­ry, I think dis­tin­guish­ing UX design from UI design is non­sense. Because you can­not prac­tice design with­out sketch­ing and pro­to­typ­ing. And you can­not sketch and pro­to­type a soft­ware prod­uct with­out touch­ing its sur­face. In stead of tak­ing visu­al design for grant­ed, or talk­ing about it like it is some innate tal­ent, some kind of mag­i­cal skill some peo­ple are born with and oth­ers aren’t, user expe­ri­ence prac­ti­tion­ers should con­sid­er being less enam­oured with acquir­ing more skills from busi­ness, mar­ket­ing and engi­neer­ing and in stead prac­tice at the skills that define the fields user expe­ri­ence design is indebt­ed to the most: graph­ic design and indus­tri­al design. In oth­er words, you can’t do user expe­ri­ence design with­out touch­ing the sur­face.