So, some notes for the past week. They will be very short because there is not much to tell really. It was one of those weeks with a lull in between projects, and also virtually devoid of meetings, today being the exception.
I went over to LikeMind to catch up with my fellow Dutchman Mark, who seems to be doing great, living and working between Copenhagen and London. Not the shabbiest pair of cities.
I also had another chat with the guys running the U‑Turm project who seem to be making nice progress with the image tracking side of things and are now really getting into the game design. Which means, as I told them, they need to start iterating on the rules like crazy, doing paper prototype after paper prototype.
Aside from this I’ve been doing some preparations for PLAY, reaching out to creative teams and individuals who I’d like to see be part of the productions we’re going to be running. Getting lots of positive responses so that is great. I also posted some background on the work we’ve been doing so far for this at the Hubbub blog.
Julius and Karel ran a successful second urban games workshop in Leidsche Rijn for Cultuur19 last saturday. We are now gearing up to assist with NU Grounds, a games festival in the same neighborhood where there will be several urban games on offer. That’s going to take place two weeks form now and should be good fun.
The relative quiet this week has allowed me some time to tinker with Processing, particularly the Box2D library that Mr. Shiffman has put together. I’m just making a little silly software toy with that to kind of flex my flimsy programming skills. Nothing special but I might post some screenshots and maybe a screencast later anyway.
And also, I bit the bullet, installed XCode, and had a go at OpenFrameworks, mostly to have a look at some of the Box2D stuff out there that is controlled with OpenCV (a computer vision library). That seems to be a really nice basis for gaming in public space using urban projections and such. I don’t see myself working in OpenFrameworks though, it really is an increase in complexity as opposed to Processing. Still, by messing with it, I can now appreciate it better.
Next week is going to be my last one here in Copenhagen and looks like it’ll be slightly more busy, with another trip to Malmö and a lecture at CIID. After that it’s back to NL. Time really flies.
So a few weeks ago, before he went surfing in Morocco, m’colleague Alper reported in an elaborate fashion on the project he and I did for the artist Sarah van Sonsbeeck. If you’re into data visualization and information design and you haven’t read it already, I encourage you to do so right now. I thought I’d post some additions and comments to Alper’s post here, going into some details related to working with 3D graphics in Processing, aesthetic considerations, and some other bits.
Dealing with 3D in Processing
As Alper writes, part of what we were doing involved generating a 3D landscape from the sound volume and location data Sarah had gathered. What I found was that working with 3D in Processing can be cumbersome, especially with regards to camera controls. I had a hard time getting a sense of the 3D model we were generating, since the camera controls I had at my disposal were limited and a hassle to use. I can imagine that if you do this kind of work a lot, you have a kind of container Processing sketch that has all the camera controls in place. I didn’t, so ultimately I decided to go to a tool that had all the camera controls already: Google SketchUp. I could have gone to a more professional tool such as 3ds Max or Maya, but SketchUp is freely available and suited me fine. Using Marius Watz’s unlekkerLib, I exported the geometry of our landscape to a DXF file and imported it to SketchUp and that was that.
Rendering
SketchUp is fine for manipulating and exploring a 3D model, but its rendering left something to be desired. It had been a while since I dabbled in 3D graphics (back in art school, making games) but I did recall that global illumination rendering yields pretty pictures. Sarah had told us from the outset that above everything else, it was important for the output of our exercise to be aesthetically pleasing. According to her, in the art world, beauty is paramount. So I did some Googling (as one does) and bumped into Sunflow. Karsten Schmidt had experimented with using Sunflow as a renderer directly from Processing, but this library turned out to be outdated for the current version of Processing. There is however, a Sunflow exporter for Sketchup. So I used Sketchup to set up the basics of the scene I wanted to render (camera angle and such) exported and then manually edited the resulting Sunflow file. The Sunflow wiki was a great help for understanding the anatomy of the Sunflow file format. In addition, this page, which shows examples of many shader settings, was very helpful when it came to figuring out the materials we ended up using. A snow-like material with a “sun sky” light, which makes the whole thing look like Antarctica, seemed like a good fit for the subject; silence.
Aesthetics
Alper rightly points out neither of us is a graphic designer. But this does not mean certain aesthetic considerations came into play during this project. For instance, towards the end, we had renderings of the landscape floating above an infinite plane, as if it’s an object of sorts. I felt this did not do justice to the concept we were pursuing, so we eventually decided to merge the mesh with the underlying plane. We achieved this by simply adding a band of average noise level around the datascape and regenerating it. Optically, thanks to the nice illumination in Sunflow, there is no border between the landscape and the plane that recedes to the horizon.
There’s more to be said about this project but I feel like wrapping up. A few final words with regards to the utility of this piece as data visualization then. I think from this perspective it is practically useless. As a piece of graphic art that provides a visceral sense of the data gathered by Sarah during her walks however, I think it is quite successful. Keep in mind she’s used this as part of another publication, where a set of annotations is overlaid on it.
I would also love to do an interactive version of this, allowing for free movement through the 3D space, as well as additional information layers with annotations by the artist and geographic context. Who knows, we might come around to this some time.
But for now, this is it: a picture of a landscape, generated from silence.
I am a bit nervous about doing this, but since several people asked, here goes: You can now download the travel-time map of the Netherlands I made in Processing. I have exported applications for Linux, Mac OS X and Windows. Each download includes the source files, but not the data file. For that, you will need to head to Alper’s site (he’s the guy who pulled the data from 9292 and ANWB). I hope you’ll enjoy playing around with this, or learn something from the way it was put together.
Some notes, in no particular order:
Please remember I am not a programmer. The vast majority of this sketch was put together from bits and pieces of code I found in books and online. I have tried to credit all the sources in the code. The full write-up I posted earlier should point you to all the sources too. In short; all the good bits are by other people, the bad code is mine. But who cares, it’s the end-result that counts (at least for me).
Related to the previous point is the fact that I cannot figure out under which license (if any) to release this. So the usual CC by-nc-sa license applies, as far as I’m concerned.
If this breaks your computer, offends you, makes you cry, or eats your kittens, do not come knocking. This is provided as is, no warranties whatsoever, etc.
Why am I nervous? Probably because for me the point of the whole exercise was the process, not the outcome.
Subscribers to my Flickr stream have probably noticed a number of images of some kind of map flowing past lately. They were the result of me tracking my progress on a pet project. I have more or less finished work on it this week, so I thought I’d detail what I did over here.
Background
Following my Twitter dataviz sketches, I thought I’d take another stab at prototyping with Processing. On the one hand I wanted to increase my familiarity with the environment. On the other, I continued to be fascinated with data-visualization, so I wanted to do another design exercise in this domain. I was particularly interested in creating displays that assist in decision making and present data in a way that allows people to ‘play’ with it — explore it and learn from it.
The seed for this thing was planted when I saw Stamen’s work on the mySociety travel-time maps. I thought the idea of visually overlaying two datasets and allowing the intersection to be manipulated by people was simple but powerful. But, at that time, I saw no way to ‘easily’ try my hand at something similar. I had no ready access to any potentially interesting data, and my scraping skills are limited at best.
Luckily, I was not the only one whose curiosity was piqued. After seeing Ben Cerveny demoing the same maps at The Web and Beyond 2008, Alper wondered how hard it would be to create something similar for the Netherlands. He presented a way to do it with freely available tools and data (to an extent) in a workshop at a local unconference.1
I did not attend the event, but after seeing his blog post, I sent him an email and asked if he was willing to part with the data he had collected from the Dutch public transport travel planning site 9292. Alper being the nice guy he is, he soon emailed me a JSON containing of the data.
So that’s the background. I had an example, I had some data, and I had a little experience with making things in Processing.
JSON
The first step was to read the data in the JSON file from Processing. I followed the instructions on how to get the JSON library into Processing from Ben Fry’s book (pages 315–316). On the Processing boards, a cursory search unearthed some code examples. After a little fiddling, I got it to work and could print the data to Processing’s console.
Plotting
Next up was to start visualizing it. I used the examples of scatterplot maps in Visualizing Data as a starting point, and plugged in the JSON data. Pretty soon, I had a nice plot of the postal codes that actually resembled the Netherlands.
Coloring
From there, it was rather easy to show each postal code’s travel time.2 I simply mapped travel times to a hue in the HSB spectrum. The result nicely shows colored bands of travel-time regions and also allows you to pick out some interesting outliers (such as Groningen in the north).
Selecting
At this point, I wanted to be able to select travel-time ranges and hide postal codes outside of that range. Initially, I used the keyboard for input. This was OK for this stage of the project, but of course it would need to be replaced with something more intuitive later on. In any case, I could highlight selected points and dim others, which increased the display’s explorability considerably.
Coloring, again
The HSB spectrum is quick and easy way of getting access to a full a range of colors. It served me well in my Twitter visualizations. However in this case it left something to be desired, aesthetically speaking. Via Tom Carden I found the wonderful cpt-city, which catalogues gradients for cartography and the like. Initially I struggled with ways to get these colors into Processing, but then it turned out you could easily read out the colors of pixels from images. This allowed me to cycle through many palettes just by adding the files to my Processing sketch. I discovered that a palette with a clear division in the middle was best, because that provides you with an extra reference point besides the beginning and end.
Selecting, again
I next turned to the interaction bits. I knew I wanted a so-called dual slider that would allow people to select the upper and lower limit of travel time. In the Processing book, there is code for plenty of interface widgets, but sadly no dual slider. I looked around on the Processing board and could find none either, to my surprise. Even in the UI libraries (such as controlP5 and Interfascia) I could not locate one.
So I decided to lower the bar and first include two horizontal sliders, one for the upper and one for the lower limit. These I made using the code on pages 448–452 of the Processing book. Not perfect, but an improvement over the keyboard controls.
Selecting, yet again
Next, I decided I’d see if I could modify the code of the horizontal scrollbar so that I would end up with a dual slider. After some messing about (which did increase my understanding of the original code considerably) I managed to get it to work. This was an unexpected success. I now had a decent dual slider.
Exploring
So far there was no way of telling which point corresponded to which postal code. So, I added a rollover that displayed the postal code’s name and travel time. At this point it became clear the data wasn’t perfect — some postal codes were erroneously geocoded by GeoNames. For instance, code 9843 (which is Grijpskerk, 199 minutes to the Dam) was placed on the map as Amsterdam Noord-Oost!
Adding more data
Around this point I visited Alper in Delft and we discussed adding a second dataset. Although housing prices à la mySociety would have been interesting, we decided to take a different route and add a second travel-time set for cars.3 My first step in integrating this was to simply generate a map each for the public transport and car travel data and manually juxtapose them. What I liked about this was that even though you know intuitively that traveling by car is faster, the two maps next to each other provide a dramatic visual confirmation of this piece of knowledge.
Representing
Moving ahead with the extra data, I started to struggle with how to represent both travel times. My first effort was to draw two sets of dots on top of each other (one for car travel times and one for public transport) and color each accordingly. For each set I introduced a separate slider. I wasn’t very satisfied with the result of this. It did not help in understanding what was going on that much.
Showing differences
After discussions with Alper and several other people, I decided it would make more sense to show the difference between travel times. So I calculated the percentage difference between public transport and car travel time for each postal code. This value I mapped to a color. Here, a simple gradient worked better than the palettes used earlier for travel times.
I also discarded the idea of having two dual sliders and simply went with one travel time selector. Although more user-friendly, it created a new problem: for some points both travel times would fall within the selected range, and for others one or the other. So I needed an extra visual dimension to show this. This turned out to be the greatest challenge.
After trying many approaches, I eventually settled on using the shape of the point to show which travel times fell within the range. A small dot meant that only the public transport travel time is within the range, a donut means only the car travel time is selected, and a big dot represents selection of both times.
Final tweaks
Around this point I felt that it was time to wrap up. I had learnt about all I could from the exercise and any extra time spent on the project would result in marginal improvements at best. I added a legend for both the shapes and color, improved the legibility of the rollover and increased the visual affordance of the slider, and that was it.
Thoughts
It is becoming apparent to me that the act of building displays like this is playful in its own way. Through sketching in code, you can have something like a conversation with the data and get a sense of what’s there. Perhaps the end result is merely a byproduct of this process?
I’m amazed at how far a novice programmer like myself, with a dramatic lack of affinity for anything related to mathematics or physics, can get by simply modifying, augmenting and combining code that is already out there. I have no ambition whatsoever of becoming a professional developer of production-quality code. But building a collection bits and pieces of code that can do useful and interesting things seems like a good strategy for any designer. I am learning to trust my innate reluctance to code stuff from scratch.
Also, isn’t it cool that it is becoming increasingly feasible for regular citizens to start analyzing data that is — or at least should be — publicly available? Government still has a long way to go. Why do we need to go through the painstaking process of scraping this data from sources such as 9292 which for all intents and purposes is a public service?4
I will probably make the final prototype available online at some point in the future. For now, if you have any questions or comments I would love to hear them here, or via email.
Update:Alper has released a JSON file containing all the data I used to make this. Go on and grab it, and make some displays of your own!
And another update: I’ve decided to make this application available for download, including source files.
Sketching is the defining activity of design writes Buxton and I tend to agree. The genius of his book is that he shows sketching can take on many forms. It is not limited to working with pencils and paper. You can sketch in 3D using wood or clay. You can sketch in time using video, etc. Buxton does not include many examples of sketching in code, though.1 Programming in any language tends to be a hard earned skill, he writes, and once you have achieved sufficient mastery in it, you tend to try and solve all problems with this one tool. Good designers can draw on a broad range of sketching techniques and pick the right one for a given situation. This might include programming, but then it would need to conform to Buxton’s defining characteristics of sketching: quick, inexpensive, disposable, plentiful, offer minimal detail, and suggest and explore rather than confirm.
I have been spending some time broadening my sketching repertoire as a designer. Before I started interaction design I was mostly into visual arts (drawing, painting, comics) so I am quite comfortable sketching in 2D, using storyboards, etc.2 Sketching in code though, has always been a weak spot. I have started to remedy this by looking into Processing.
As an exercise I took some data from Twitter — one data set was the 20 most recent tweets and the other my friends list — and decided to see how quick I could create a few different visualizations of that data. The end results were:
one: a timeline that spatially plots the latest tweets from my friends — showing density at certain points in time; or how ‘noisy’ it is on my Twitter stream,
two: an ordering of friends based on the percentage of their tweets that take up my timeline — who’s the loudest of my friends?,
three: a graph of my friends list, with number of friends and followers on the axes and their total number of tweets mapped to the size of each point.
The aim was not to come up with groundbreaking solutions, or finished applications.3 The goal was to exercise this idea of sketching in code and use it to get a feel for a ‘complex’ data set, iterating on many different ways to show the data before committing to one solution. In a real-world project I could see myself as a designer do this and then collaborate with a ‘proper’ programmer to develop the final solution (which would most likely be interactive). I would choose different sketching techniques to design the interactive aspects of a data-visualization. For now I am content with Processing sketches that simply output a static image.
Tools & resources used were:
Processing — a free and open source programming environment for visual folks
The Twitter4J library for working with the Twitter API
If as a designer you are confronted with a project that involves making a large amount of data understandable, sketching in code can help. You can use it to ‘talk’ to the data, and get a sense of its ‘shape’.
There is one involving Phidgets and Max/MSP, a visual programming solution for physical computing. [↩]
Last sunday I sat down and coded a prototype of a casual game in Mobile Processing. I got the idea for it the evening before: You’re a bee who needs to collect as much honey as possible in his hive while at the same time keeping a flower-bed blooming by pollinating… Play it and let me know what your high score is in the comments!
Thinking and making
I’ve been looking for an excuse to get some experience with Processing (particularly the variant suitable for developing mobile stuff) for a while. I also felt I needed to get back into the making part of the field I’ve been thinking about so much lately: Game Design. I agree with Saffer, Webb and others — making is an important part of the design practice, it cannot be replaced by lots of thinking. The things learnt from engaging with the actual stuff things are made of (which in the case of digital games is code) aren’t gained in any other way and very valuable.
Get the game
I’ve uploaded the first version of the game here. You can play it in the emulator in your browser or if your phone runs Java midlets, download the file and play it like you’re supposed to: While out and about. The source code is provided as well, if you feel like looking at it.1
How to play
You’re the yellow oval. The orange triangle in the top left corner is your hive. Green squares are grass, brown squares are seeds, red squares are flowers and pink squares are pollinated flowers. The field is updated in columns from left to right (indicated by the yellow marker in the bottom). A seed will turn into a flower (in rare cases a pollinated flower). A flower will die, a pollinated flower will die and spread seeds to grass around it. Move your bee with the directional keys, use the centre key to grab nectar from a flower. You can cary a maximum of 100 nectar. Drop your nectar off at the hive (again using the centre key) to up your score. When you first grab nectar from a pollinated flower and subsequently from a normal flower, the latter is pollinated. Try to keep the flower-bed in bloom while at the same time racking up a high-score!
You’ll get 10 nectar from a flower (in bloom or not). Pollinating a flower costs 5 nectar. If you try to take nectar more than once from the same flower, you’ll loose 10 nectar.2
Improvements
Stuff not in here that I might put into a next version (whenever I get around to it):
Animation — I need to get my feet wet with some scripted animation. Thing is I’ve always sucked at this. For now it’s all tile-based stuff.
Better feedback — For instance show the points you earn near the bee and the hive. I think that’ll make the game a lot easier to understand and therefore more fun.
Menus, pause, game over — It’s a prototype, so you get dumped into the action right away. (The game starts on the first key you press.) And there’s no actual game over message, the field just turns green and you’re left to wonder what to do.
Balance — I’m not sure if the game like it stands is balanced right, I will need to play it a lot to figure that out. Also there’s probably a dominant strategy that’ll let you rack up points easily.
The aim was to create a relatively casual game experience that will almost allow you to zone out while playing. I think it is far too twitchy now, so perhaps I really should sit down and do a second version sometime soon.
Mobile Processing
I enjoy working with Mobile Processing. I like the way it allows you to program in a very naive way but if you like structure things in a more sophisticated fashion. It really does allow you to sketch in code, which is exactly what I need. The emphasis on just code also prevents me from fiddling around with animations, graphics and so on (like I would in Flash for instance.) Perhaps the only thing that would be nice is an editor that is a bit more full-featured.3 Perhaps I should grab an external editor next time?
Feedback
If you played the game and liked it (or thought it was too hard, boring or whatever) I’d love to get your feedback in the comments. Anyone else out there prototyping games in Processing? Or using it to teach game design? I’d be very interested to hear about it.
Not that it’s particularly good, I’m an amateur coder at best. [↩]
I’m not sure this is the right kind of negative reinforcement. [↩]
The automatic code formatting refused to work for me, requiring me to spend a bit too much effort on formatting by hand. [↩]
