The making of a travel-time map of the Netherlands

Sub­scribers to my Flickr stream have prob­a­bly noticed a num­ber of images of some kind of map flow­ing past late­ly. They were the result of me track­ing my progress on a pet project. I have more or less fin­ished work on it this week, so I thought I’d detail what I did over here.

Background

Fol­low­ing my Twit­ter dataviz sketch­es, I thought I’d take anoth­er stab at pro­to­typ­ing with Pro­cess­ing. On the one hand I want­ed to increase my famil­iar­i­ty with the envi­ron­ment. On the oth­er, I con­tin­ued to be fas­ci­nat­ed with data-visu­al­iza­tion, so I want­ed to do anoth­er design exer­cise in this domain. I was par­tic­u­lar­ly inter­est­ed in cre­at­ing dis­plays that assist in deci­sion mak­ing and present data in a way that allows peo­ple to ‘play’ with it — explore it and learn from it.

The seed for this thing was plant­ed when I saw Sta­men’s work on the mySo­ci­ety trav­el-time maps. I thought the idea of visu­al­ly over­lay­ing two datasets and allow­ing the inter­sec­tion to be manip­u­lat­ed by peo­ple was sim­ple but pow­er­ful. But, at that time, I saw no way to ‘eas­i­ly’ try my hand at some­thing sim­i­lar. I had no ready access to any poten­tial­ly inter­est­ing data, and my scrap­ing skills are lim­it­ed at best.

Luck­i­ly, I was not the only one whose curios­i­ty was piqued. After see­ing Ben Cer­ve­ny demo­ing the same maps at The Web and Beyond 2008, Alper won­dered how hard it would be to cre­ate some­thing sim­i­lar for the Nether­lands. He pre­sent­ed a way to do it with freely avail­able tools and data (to an extent) in a work­shop at a local uncon­fer­ence.1

I did not attend the event, but after see­ing his blog post, I sent him an email and asked if he was will­ing to part with the data he had col­lect­ed from the Dutch pub­lic trans­port trav­el plan­ning site 9292. Alper being the nice guy he is, he soon emailed me a JSON con­tain­ing of the data. 

So that’s the back­ground. I had an exam­ple, I had some data, and I had a lit­tle expe­ri­ence with mak­ing things in Processing.

JSON

The first step was to read the data in the JSON file from Pro­cess­ing. I fol­lowed the instruc­tions on how to get the JSON library into Pro­cess­ing from Ben Fry’s book (pages 315–316). On the Pro­cess­ing boards, a cur­so­ry search unearthed some code exam­ples. After a lit­tle fid­dling, I got it to work and could print the data to Processing’s console.

Plotting

Next up was to start visu­al­iz­ing it. I used the exam­ples of scat­ter­plot maps in Visu­al­iz­ing Data as a start­ing point, and plugged in the JSON data. Pret­ty soon, I had a nice plot of the postal codes that actu­al­ly resem­bled the Netherlands.

Playing with some data Alper gave me

Coloring

From there, it was rather easy to show each postal code’s trav­el time.2 I sim­ply mapped trav­el times to a hue in the HSB spec­trum. The result nice­ly shows col­ored bands of trav­el-time regions and also allows you to pick out some inter­est­ing out­liers (such as Gronin­gen in the north).

Second pass

Selecting

At this point, I want­ed to be able to select trav­el-time ranges and hide postal codes out­side of that range. Ini­tial­ly, I used the key­board for input. This was OK for this stage of the project, but of course it would need to be replaced with some­thing more intu­itive lat­er on. In any case, I could high­light select­ed points and dim oth­ers, which increased the display’s explorabil­i­ty considerably.

Pass 3

Coloring, again

The HSB spec­trum is quick and easy way of get­ting access to a full a range of col­ors. It served me well in my Twit­ter visu­al­iza­tions. How­ev­er in this case it left some­thing to be desired, aes­thet­i­cal­ly speak­ing. Via Tom Car­den I found the won­der­ful cpt-city, which cat­a­logues gra­di­ents for car­tog­ra­phy and the like. Ini­tial­ly I strug­gled with ways to get these col­ors into Pro­cess­ing, but then it turned out you could eas­i­ly read out the col­ors of pix­els from images. This allowed me to cycle through many palettes just by adding the files to my Pro­cess­ing sketch. I dis­cov­ered that a palette with a clear divi­sion in the mid­dle was best, because that pro­vides you with an extra ref­er­ence point besides the begin­ning and end.

Playing with palettes (pass 4)

Selecting, again

I next turned to the inter­ac­tion bits. I knew I want­ed a so-called dual slid­er that would allow peo­ple to select the upper and low­er lim­it of trav­el time. In the Pro­cess­ing book, there is code for plen­ty of inter­face wid­gets, but sad­ly no dual slid­er. I looked around on the Pro­cess­ing board and could find none either, to my sur­prise. Even in the UI libraries (such as controlP5 and Inter­fas­cia) I could not locate one. 

So I decid­ed to low­er the bar and first include two hor­i­zon­tal slid­ers, one for the upper and one for the low­er lim­it. These I made using the code on pages 448–452 of the Pro­cess­ing book. Not per­fect, but an improve­ment over the key­board controls.

Pass 5 – some basic interactivity

Selecting, yet again

Next, I decid­ed I’d see if I could mod­i­fy the code of the hor­i­zon­tal scroll­bar so that I would end up with a dual slid­er. After some mess­ing about (which did increase my under­stand­ing of the orig­i­nal code con­sid­er­ably) I man­aged to get it to work. This was an unex­pect­ed suc­cess. I now had a decent dual slider.

A proper dual slider

Exploring

So far there was no way of telling which point cor­re­spond­ed to which postal code. So, I added a rollover that dis­played the postal code’s name and trav­el time. At this point it became clear the data wasn’t per­fect — some postal codes were erro­neous­ly geocod­ed by GeoN­ames. For instance, code 9843 (which is Gri­jpskerk, 199 min­utes to the Dam) was placed on the map as Ams­ter­dam Noord-Oost!

Rollovers

Adding more data

Around this point I vis­it­ed Alper in Delft and we dis­cussed adding a sec­ond dataset. Although hous­ing prices à la mySo­ci­ety would have been inter­est­ing, we decid­ed to take a dif­fer­ent route and add a sec­ond trav­el-time set for cars.3 My first step in inte­grat­ing this was to sim­ply gen­er­ate a map each for the pub­lic trans­port and car trav­el data and man­u­al­ly jux­ta­pose them. What I liked about this was that even though you know intu­itive­ly that trav­el­ing by car is faster, the two maps next to each oth­er pro­vide a dra­mat­ic visu­al con­fir­ma­tion of this piece of knowledge.

Compare: travel by public transport or car

Representing

Mov­ing ahead with the extra data, I start­ed to strug­gle with how to rep­re­sent both trav­el times. My first effort was to draw two sets of dots on top of each oth­er (one for car trav­el times and one for pub­lic trans­port) and col­or each accord­ing­ly. For each set I intro­duced a sep­a­rate slid­er. I wasn’t very sat­is­fied with the result of this. It did not help in under­stand­ing what was going on that much.

The gap

Showing differences

After dis­cus­sions with Alper and sev­er­al oth­er peo­ple, I decid­ed it would make more sense to show the dif­fer­ence between trav­el times. So I cal­cu­lat­ed the per­cent­age dif­fer­ence between pub­lic trans­port and car trav­el time for each postal code. This val­ue I mapped to a col­or. Here, a sim­ple gra­di­ent worked bet­ter than the palettes used ear­li­er for trav­el times.

I also dis­card­ed the idea of hav­ing two dual slid­ers and sim­ply went with one trav­el time selec­tor. Although more user-friend­ly, it cre­at­ed a new prob­lem: for some points both trav­el times would fall with­in the select­ed range, and for oth­ers one or the oth­er. So I need­ed an extra visu­al dimen­sion to show this. This turned out to be the great­est challenge.

After try­ing many approach­es, I even­tu­al­ly set­tled on using the shape of the point to show which trav­el times fell with­in the range. A small dot meant that only the pub­lic trans­port trav­el time is with­in the range, a donut means only the car trav­el time is select­ed, and a big dot rep­re­sents selec­tion of both times.

Return of the map

Final tweaks

Around this point I felt that it was time to wrap up. I had learnt about all I could from the exer­cise and any extra time spent on the project would result in mar­gin­al improve­ments at best. I added a leg­end for both the shapes and col­or, improved the leg­i­bil­i­ty of the rollover and increased the visu­al affor­dance of the slid­er, and that was it.

It's hard to stop tweaking

Thoughts

It is becom­ing appar­ent to me that the act of build­ing dis­plays like this is play­ful in its own way. Through sketch­ing in code, you can have some­thing like a con­ver­sa­tion with the data and get a sense of what’s there. Per­haps the end result is mere­ly a byprod­uct of this process?

I’m amazed at how far a novice pro­gram­mer like myself, with a dra­mat­ic lack of affin­i­ty for any­thing relat­ed to math­e­mat­ics or physics, can get by sim­ply mod­i­fy­ing, aug­ment­ing and com­bin­ing code that is already out there. I have no ambi­tion what­so­ev­er of becom­ing a pro­fes­sion­al devel­op­er of pro­duc­tion-qual­i­ty code. But build­ing a col­lec­tion bits and pieces of code that can do use­ful and inter­est­ing things seems like a good strat­e­gy for any design­er. I am learn­ing to trust my innate reluc­tance to code stuff from scratch. 

Also, isn’t it cool that it is becom­ing increas­ing­ly fea­si­ble for reg­u­lar cit­i­zens to start ana­lyz­ing data that is — or at least should be — pub­licly avail­able? Gov­ern­ment still has a long way to go. Why do we need to go through the painstak­ing process of scrap­ing this data from sources such as 9292 which for all intents and pur­pos­es is a pub­lic ser­vice?4

I will prob­a­bly make the final pro­to­type avail­able online at some point in the future. For now, if you have any ques­tions or com­ments I would love to hear them here, or via email.

Update: Alper has released a JSON file con­tain­ing all the data I used to make this. Go on and grab it, and make some dis­plays of your own!

And anoth­er update: I’ve decid­ed to make this appli­ca­tion avail­able for down­load, includ­ing source files.

  1. Those of you who under­stand Dutch might enjoy his walk­through on Vimeo. []
  2. Inci­den­tal­ly, all trav­el times in this project were from the Dam in Ams­ter­dam to all the postal codes in NL. []
  3. This we retrieved from the ANWB site. The time of day was set to 12:00 noon. []
  4. Tools like Mech­a­nize make this eas­i­er, but still. []