Download my travel-time map

I am a bit ner­vous about doing this, but since sev­er­al peo­ple asked, here goes: You can now down­load the trav­el-time map of the Nether­lands I made in Pro­cess­ing. I have export­ed appli­ca­tions for Lin­ux, Mac OS X and Win­dows. Each down­load includes the source files, but not the data file. For that, you will need to head to Alper’s site (he’s the guy who pulled the data from 9292 and ANWB). I hope you’ll enjoy play­ing around with this, or learn some­thing from the way it was put togeth­er.

Some notes, in no par­tic­u­lar order:

  • Please remem­ber I am not a pro­gram­mer. The vast major­i­ty of this sketch was put togeth­er from bits and pieces of code I found in books and online. I have tried to cred­it all the sources in the code. The full write-up I post­ed ear­li­er should point you to all the sources too. In short; all the good bits are by oth­er peo­ple, the bad code is mine. But who cares, it’s the end-result that counts (at least for me).
  • Relat­ed to the pre­vi­ous point is the fact that I can­not fig­ure out under which license (if any) to release this. So the usu­al CC by-nc-sa license applies, as far as I’m con­cerned.
  • If this breaks your com­put­er, offends you, makes you cry, or eats your kit­tens, do not come knock­ing. This is pro­vid­ed as is, no war­ranties what­so­ev­er, etc.
  • Why am I ner­vous? Prob­a­bly because for me the point of the whole exer­cise was the process, not the out­come.
  • I can’t think of any­thing else. Have fun.

The making of a travel-time map of the Netherlands

Sub­scribers to my Flickr stream have prob­a­bly noticed a num­ber of images of some kind of map flow­ing past late­ly. They were the result of me track­ing my progress on a pet project. I have more or less fin­ished work on it this week, so I thought I’d detail what I did over here.

Background

Fol­low­ing my Twit­ter dataviz sketch­es, I thought I’d take anoth­er stab at pro­to­typ­ing with Pro­cess­ing. On the one hand I want­ed to increase my famil­iar­i­ty with the envi­ron­ment. On the oth­er, I con­tin­ued to be fas­ci­nat­ed with data-visu­al­iza­tion, so I want­ed to do anoth­er design exer­cise in this domain. I was par­tic­u­lar­ly inter­est­ed in cre­at­ing dis­plays that assist in deci­sion mak­ing and present data in a way that allows peo­ple to ‘play’ with it — explore it and learn from it.

The seed for this thing was plant­ed when I saw Sta­men’s work on the mySo­ci­ety trav­el-time maps. I thought the idea of visu­al­ly over­lay­ing two datasets and allow­ing the inter­sec­tion to be manip­u­lat­ed by peo­ple was sim­ple but pow­er­ful. But, at that time, I saw no way to ‘eas­i­ly’ try my hand at some­thing sim­i­lar. I had no ready access to any poten­tial­ly inter­est­ing data, and my scrap­ing skills are lim­it­ed at best.

Luck­i­ly, I was not the only one whose curios­i­ty was piqued. After see­ing Ben Cer­ve­ny demo­ing the same maps at The Web and Beyond 2008, Alper won­dered how hard it would be to cre­ate some­thing sim­i­lar for the Nether­lands. He pre­sent­ed a way to do it with freely avail­able tools and data (to an extent) in a work­shop at a local uncon­fer­ence.1

I did not attend the event, but after see­ing his blog post, I sent him an email and asked if he was will­ing to part with the data he had col­lect­ed from the Dutch pub­lic trans­port trav­el plan­ning site 9292. Alper being the nice guy he is, he soon emailed me a JSON con­tain­ing of the data.

So that’s the back­ground. I had an exam­ple, I had some data, and I had a lit­tle expe­ri­ence with mak­ing things in Pro­cess­ing.

JSON

The first step was to read the data in the JSON file from Pro­cess­ing. I fol­lowed the instruc­tions on how to get the JSON library into Pro­cess­ing from Ben Fry’s book (pages 315–316). On the Pro­cess­ing boards, a cur­so­ry search unearthed some code exam­ples. After a lit­tle fid­dling, I got it to work and could print the data to Processing’s con­sole.

Plotting

Next up was to start visu­al­iz­ing it. I used the exam­ples of scat­ter­plot maps in Visu­al­iz­ing Data as a start­ing point, and plugged in the JSON data. Pret­ty soon, I had a nice plot of the postal codes that actu­al­ly resem­bled the Nether­lands.

Playing with some data Alper gave me

Coloring

From there, it was rather easy to show each postal code’s trav­el time.2 I sim­ply mapped trav­el times to a hue in the HSB spec­trum. The result nice­ly shows col­ored bands of trav­el-time regions and also allows you to pick out some inter­est­ing out­liers (such as Gronin­gen in the north).

Second pass

Selecting

At this point, I want­ed to be able to select trav­el-time ranges and hide postal codes out­side of that range. Ini­tial­ly, I used the key­board for input. This was OK for this stage of the project, but of course it would need to be replaced with some­thing more intu­itive lat­er on. In any case, I could high­light select­ed points and dim oth­ers, which increased the display’s explorabil­i­ty con­sid­er­ably.

Pass 3

Coloring, again

The HSB spec­trum is quick and easy way of get­ting access to a full a range of col­ors. It served me well in my Twit­ter visu­al­iza­tions. How­ev­er in this case it left some­thing to be desired, aes­thet­i­cal­ly speak­ing. Via Tom Car­den I found the won­der­ful cpt-city, which cat­a­logues gra­di­ents for car­tog­ra­phy and the like. Ini­tial­ly I strug­gled with ways to get these col­ors into Pro­cess­ing, but then it turned out you could eas­i­ly read out the col­ors of pix­els from images. This allowed me to cycle through many palettes just by adding the files to my Pro­cess­ing sketch. I dis­cov­ered that a palette with a clear divi­sion in the mid­dle was best, because that pro­vides you with an extra ref­er­ence point besides the begin­ning and end.

Playing with palettes (pass 4)

Selecting, again

I next turned to the inter­ac­tion bits. I knew I want­ed a so-called dual slid­er that would allow peo­ple to select the upper and low­er lim­it of trav­el time. In the Pro­cess­ing book, there is code for plen­ty of inter­face wid­gets, but sad­ly no dual slid­er. I looked around on the Pro­cess­ing board and could find none either, to my sur­prise. Even in the UI libraries (such as controlP5 and Inter­fas­cia) I could not locate one.

So I decid­ed to low­er the bar and first include two hor­i­zon­tal slid­ers, one for the upper and one for the low­er lim­it. These I made using the code on pages 448–452 of the Pro­cess­ing book. Not per­fect, but an improve­ment over the key­board con­trols.

Pass 5 – some basic interactivity

Selecting, yet again

Next, I decid­ed I’d see if I could mod­i­fy the code of the hor­i­zon­tal scroll­bar so that I would end up with a dual slid­er. After some mess­ing about (which did increase my under­stand­ing of the orig­i­nal code con­sid­er­ably) I man­aged to get it to work. This was an unex­pect­ed suc­cess. I now had a decent dual slid­er.

A proper dual slider

Exploring

So far there was no way of telling which point cor­re­spond­ed to which postal code. So, I added a rollover that dis­played the postal code’s name and trav­el time. At this point it became clear the data wasn’t per­fect — some postal codes were erro­neous­ly geocod­ed by GeoN­ames. For instance, code 9843 (which is Gri­jpskerk, 199 min­utes to the Dam) was placed on the map as Ams­ter­dam Noord-Oost!

Rollovers

Adding more data

Around this point I vis­it­ed Alper in Delft and we dis­cussed adding a sec­ond dataset. Although hous­ing prices à la mySo­ci­ety would have been inter­est­ing, we decid­ed to take a dif­fer­ent route and add a sec­ond trav­el-time set for cars.3 My first step in inte­grat­ing this was to sim­ply gen­er­ate a map each for the pub­lic trans­port and car trav­el data and man­u­al­ly jux­ta­pose them. What I liked about this was that even though you know intu­itive­ly that trav­el­ing by car is faster, the two maps next to each oth­er pro­vide a dra­mat­ic visu­al con­fir­ma­tion of this piece of knowl­edge.

Compare: travel by public transport or car

Representing

Mov­ing ahead with the extra data, I start­ed to strug­gle with how to rep­re­sent both trav­el times. My first effort was to draw two sets of dots on top of each oth­er (one for car trav­el times and one for pub­lic trans­port) and col­or each accord­ing­ly. For each set I intro­duced a sep­a­rate slid­er. I wasn’t very sat­is­fied with the result of this. It did not help in under­stand­ing what was going on that much.

The gap

Showing differences

After dis­cus­sions with Alper and sev­er­al oth­er peo­ple, I decid­ed it would make more sense to show the dif­fer­ence between trav­el times. So I cal­cu­lat­ed the per­cent­age dif­fer­ence between pub­lic trans­port and car trav­el time for each postal code. This val­ue I mapped to a col­or. Here, a sim­ple gra­di­ent worked bet­ter than the palettes used ear­li­er for trav­el times.

I also dis­card­ed the idea of hav­ing two dual slid­ers and sim­ply went with one trav­el time selec­tor. Although more user-friend­ly, it cre­at­ed a new prob­lem: for some points both trav­el times would fall with­in the select­ed range, and for oth­ers one or the oth­er. So I need­ed an extra visu­al dimen­sion to show this. This turned out to be the great­est chal­lenge.

After try­ing many approach­es, I even­tu­al­ly set­tled on using the shape of the point to show which trav­el times fell with­in the range. A small dot meant that only the pub­lic trans­port trav­el time is with­in the range, a donut means only the car trav­el time is select­ed, and a big dot rep­re­sents selec­tion of both times.

Return of the map

Final tweaks

Around this point I felt that it was time to wrap up. I had learnt about all I could from the exer­cise and any extra time spent on the project would result in mar­gin­al improve­ments at best. I added a leg­end for both the shapes and col­or, improved the leg­i­bil­i­ty of the rollover and increased the visu­al affor­dance of the slid­er, and that was it.

It's hard to stop tweaking

Thoughts

It is becom­ing appar­ent to me that the act of build­ing dis­plays like this is play­ful in its own way. Through sketch­ing in code, you can have some­thing like a con­ver­sa­tion with the data and get a sense of what’s there. Per­haps the end result is mere­ly a byprod­uct of this process?

I’m amazed at how far a novice pro­gram­mer like myself, with a dra­mat­ic lack of affin­i­ty for any­thing relat­ed to math­e­mat­ics or physics, can get by sim­ply mod­i­fy­ing, aug­ment­ing and com­bin­ing code that is already out there. I have no ambi­tion what­so­ev­er of becom­ing a pro­fes­sion­al devel­op­er of pro­duc­tion-qual­i­ty code. But build­ing a col­lec­tion bits and pieces of code that can do use­ful and inter­est­ing things seems like a good strat­e­gy for any design­er. I am learn­ing to trust my innate reluc­tance to code stuff from scratch.

Also, isn’t it cool that it is becom­ing increas­ing­ly fea­si­ble for reg­u­lar cit­i­zens to start ana­lyz­ing data that is — or at least should be — pub­licly avail­able? Gov­ern­ment still has a long way to go. Why do we need to go through the painstak­ing process of scrap­ing this data from sources such as 9292 which for all intents and pur­pos­es is a pub­lic ser­vice?4

I will prob­a­bly make the final pro­to­type avail­able online at some point in the future. For now, if you have any ques­tions or com­ments I would love to hear them here, or via email.

Update: Alper has released a JSON file con­tain­ing all the data I used to make this. Go on and grab it, and make some dis­plays of your own!

And anoth­er update: I’ve decid­ed to make this appli­ca­tion avail­able for down­load, includ­ing source files.

  1. Those of you who under­stand Dutch might enjoy his walk­through on Vimeo. []
  2. Inci­den­tal­ly, all trav­el times in this project were from the Dam in Ams­ter­dam to all the postal codes in NL. []
  3. This we retrieved from the ANWB site. The time of day was set to 12:00 noon. []
  4. Tools like Mech­a­nize make this eas­i­er, but still. []