What’s the next challenge for Open Government data?

So three years in to data.gov.uk and the inaugural National Hack the Government Day and now there is a tick box exercise to “run a hack day”… please… someone… anyone?

Open data is not about hack days and running one does not achieve “engagement with the developer community”.

Background

I met Liz Azyan today. Someone whom I have been aware of for the last few years: blogs great stuff, is principled and keeps herself gainfully employed with a plethora of socially ethical social media support (if you know what I mean).

I was blindsided by her, she is awesome and I think really trying her damnedest to do the right thing in an environment that she totally understands, but with a community she is less accustomed to – yet. Watch this space, and government data geeks: I urge you to chat to her if you get a chance.

One of the questions she asked me today was: What is the next challenge for open government data? So thank you Liz for the inspiration for this blog post, it got me thinking about something I have not thought about much, recently.

The environment

Government has opened up quite a bit of data through data.gov.uk, and has encouraged engagement with keen developers who have been hankering after such information for years.

Industry too has embraced Open, with a small number of notable businesses throwing open their data doors, with good results. I wrote a post about this, I shan’t repeat myself and bore you.

APIs are being released almost every day – developer information overload has maxed out, and now we risk lethal developer apathy.

Developers have attended hack days, meetings in Whitehall – indeed many of them have joined AlphaGov. This is all fabulous; but not scalable to the extreme that the open data dream promises.

The challenge

Making it all work.

It’s all very well having developers working away with this data, but if government is not ready for it, it’s a waste of time.

Take just one example: two incredibly talented developers worked together over the course of a weekend hack last year, coding through the night to create a notification engine for the government Tell Us Once programme. It worked, it would have saved oodles of time and bucketloads of cash – but government was simply unable to implement it. This is one simplified example of 100s of apps created by Rewired State hack days alone, and there are many others.

Now, if you can imagine for a minute being a developer, donating your time – granted, sometimes the hack days are paid, but always weekends away from family – year on year creating apps that would help government and citizens. Solving problems time and time again – quick example, every year the Young Rewired State coders create apps to help them define safe routes to school/friends. Year on year we showcase these to the Home Office – nothing happens. Still no government supported/approved app to meet this obviously critical need.

Why would you bother?

Open data? Awesome, and we are making tracks.

Open Government? HARD, and we are not banging on that door yet.

The reality

The developers who work on government data often do so either out of personal frustration, or a genuine commitment to making the world a little bit better.

Rarely can they reach an audience that would benefit from their app/widget/website on their own and in their spare time, at least not without considerable support. Nor are they doing this for profit, so they are not going to get investor cash.

Helping government do its work better is not a good proposition for your typical angel or VC – the target is government; and only government can utilise the genius that they are being offered.

Lots of tiny arrows

Right now lots of tiny arrows are rained on the government portals day on day, by an increasingly disparate and desolate group of extremely talented people.

Is there any success anywhere? No. Well unless you count the oft-reported GovSpark created by Issy in Young Rewired State 2010, curated by a plethora of supportive geeks and designers and some financial and hosting support from The Stationery Office. But that was a ‘nice to have’ addition to a Prime Ministerial commitment. It was not a revolutionary way to interact with central or local government.

So what’s the next challenge for Open Government data?

Forget the data.

Find a way to enable these revolutionary ideas, apps, websites and widgets that save time, money and mind-numbing frustration from those who have to engage with government.

Do that, and only that.

And when you have done that – then engage the developers again around your open data through hack days, geek advisory boards or whatever means you can.

Until then, let them have a break. They’ll still be there if you do this. If you don’t, they won’t.

And that is ridiculous.

Also, please don’t insist people ‘do hack days’ for you. Here’s the point of a hack day.

What’s the point of a hack day?

I get asked constantly what my favourite app was that was built at any of the many hack days I have run through Rewired State. I am often ashamed that I struggle to answer, although there are many. This is because hack days are rarely about the prototype.

To cover briefly what a hack day is, it is:

• one or two days long (often belying the name)
• any number of developers, for me a minimum of 10 devs are needed to make it buzz a bit, but 20+ makes it exciting
• a subject, challenge, dataset (the broader the better)
• developers are given a brief of the subject or challenge at the beginning of day 1
• they code/design/engineer over the course of a free form period of around 24 hours to create prototype solutions or ideas
• they present back to their hack peers and any inquisitive viewers, as well as the sponsor, client or group who put the event together
• prizes are awarded
• beer and pizza is essential

Many people will not experience a hack day, but if you can, please do. Show and tells are usually open to anyone who wants to attend and twitter and lanyrd are quite good at curating such event information.

However, the reason for this blog post is to explain the point of a hack day, now in 2011 (it will definitely be different in a year’s time, but to chart right now).

If you take a little time to look at the above list of what a hack day is you can understand that the common question might be: yes but what did they make and what happened next?

My response to that is that you are jumping the gun.

What we do at hack days is show you the future. Here’s why.

Why do developers turn up?

Well, in the current climate: API bonkers, information overload (yes devs get that too), tablet shmablet, toy shmoy world that we live in, there needs to be a little peace, as well as a challenge. As I have explained in a previous post about developers it is up to the rest of the world not to risk developer apathy (already here IMHO), and to look at what really matters.

Developers are simply awesome and if you know one I dare you to go try your million dollar idea out on them – they will have deconstructed and reconstructed it in minutes. Tell them your *save the world* idea and they will probably risk divorce to build it for you – please don’t do this.

Developers who know hack days turn up for the buzz, the competition and to learn, mainly to learn. Those who have never been to one come for the challenge.

I have been running hack days for three years now, and one veteran of the Rewired State hack days was at this weekends’ hactivate event. He spent the weekend coding a composting app, it’s cool, you can see it and many more here. But the big thing for him was spending 1.5 hours playing with a web server, in peace, legitimately, on a Sunday (and learning). Another group (and this is usual for a hack weekend) were hack day virgins, and have adopted the amaze-balls face of pride at what they can actually build when challenged by time (hack days are ruthless) as well as taking home the contact details of the colleagues who are as talented as themselves, at other stuff.

One developer gave himself this hack weekend as a Father’s day present. To have a weekend to spend with his peers, although coding was his day job, to work on his own projects, surrounded by like-minded awesomes, fed, watered – that’s the point.

Most developers will leave a hack day with new knowledge or at least new contacts, that can lead to extending their ability to deliver the awesomeness.

It’s probably fair to say that most would not admit to being so excited by the non-coder audience blinking at what they have managed to create in a two-day period, nor the prizes showered upon them. And, from those I know, it is always the afterthought – although I am now really clever and spend my life finding flipping brilliant geek prizes that they can’t ignore :).

Which is why it is important to understand all this before you ask: what is the point of a hack day?

What’s in it for the non-coders/organisations/brands?

So, there is an immediate and very obvious benefit for anyone engaging a number greater than ten developers on your own idea/API/bit of kit, and hack days seem to be de rigueur. Is not hard to be confident that good things will come of the weekend.

But is it the list of prototypes at the end? That no good hack day host would ever be able to predict?

No, it is engagement with the development community. Gifting your idea/API/bit of kit and enabling some free time for developers to engage with and over said idea/API/bit of kit. Yes of course you will get any number of good prototypes and even working applications – but better you will get to meet a number of developers, showing off their skills and often their newly acquired ones – this is really as rare as hen’s teeth (usually because they are fully employed fulfilling other peoples’ ambitions) engaging over a dedicated period, with peers they may not have yet met, over your technology or challenge. Yes, your super-sexy next bazillion idea might come out of this – but you created the environment for that conversation, that dev-to-dev spark.

But yet…

The thing I have noted today after Hactivate is that the sponsors are actually dedicated to seeing the apps go beyond the hack day. The winning app was one built to try to address human trafficking, and it was created to make the interface so simple that anyone could take it up without needing access to anything too technical; then we could crowdsource peoples’ safety.

The judges are determined – from a human pov, not only the brand they represented – to help collate the necessary charity network information and wherewithall to make it happen. However the geeks who thought it needed to happen and were so passionate about beating human trafficking that they spent their weekend building an application to make people a little bit more safe, found it hard to adjust to the jump of someone actually taking it on and helping make it happen (within 24 hours). Possibly because they had been coding non-stop for 24 hours, presenting to Press, sponsors and co-hackers – more probably because they were not used to their ideas being taken up so strongly and immediately by the kind of brands that can really make it a reality.

Such is the magic of a hack day.

This is why I love hack days… dilemma 🙂

And so…

The point of a hack day for a developer is to be with like-minded people, work on your own stuff, learn and be celebrated; for the rest of us, it is to create the environment for magic to happen.

Maybe in the next few years they may become simply about the prototype, but I hope that day is a long way off. The point is developers, living and learning from each other in an environment that is created by you: the challenger.

Finally…

As ever, my cry is: please, do not take the piss, developers are for life not just for your *next million* or *save the world* idea. They are an asset to be cherished and nurtured and they do not necessarily always value the same things you do. It is rarely money or jobs – most developers are awash with job offers, and extra-curricular *cash* offers.

Hack days do work, right now, because everyone wins when they are run well and with consideration. But please don’t ask me what my favourite app is that was ever built at a hack day! I can’t tell you, I have no idea. I do however now know 200+ developers whom I would be able to call in a heartbeat, and know their skills, passions and talents – but I would never sell them to anyone.

Developers are a talent to be nurtured in our open data and open society world. Hack days respect this and act as breathing spaces for devs.

It is rarely about the prototype, and when it is, I will probably go buy that flower shop I have been promising myself.

Too much information

On day two of my week of blogging every day on what is niggling and making me think (see yesterday’s post if you are confused) I am going to write about a topic of daily discussion with colleagues and people in the digital industry and see if there is any more light to be shared on all this.

We seem to find ourselves in a world of over-communication, over-sharing and in the battle cry of Open: too much information. I am not sure that there is anything wrong with this, but what feels iffy is the fact that – again – there seem to be not enough people out there on the horizon carving out the future for us all in the following areas:

• personal data and rights management
• developer fatigue

Personal data and rights management

Working as I do in the Open data world I know for a fact that we are very careful to keep the data we work with non-personal, and endeavour always to make sure that cross-referencing data will not enable identification of an individual. We also do not go anywhere near personal information. William Heath has long been *the* voice in my world for identity, a fully paid up member of the Open Rights Group and jointly owns the company Mydex. William is one of those people to whom I was referring yesterday – we need him out there on the edges of reality and I would love to hear a lot more from him about his future vision. Personal data is obviously the next big discussion, what is the personal decision making/prioritisation that happens sub-consciously when a person builds their facebook page and sets their privacy settings? Why do people say yes or no to a store card? What is the value metric for personal data? Why is everyone (almost) religiously determined to hold back personal information from government, or treat government requests for personal information with caution or even suspicion? I don’t know the answers to this – but I hope to find out more. Please do point me to people who are researching this publicly and with a horizon view (other than William!)

I also am a bit surprised that the Open Rights Group are not being clasped to the bosom of every organisation opening their data – rights management, believe you me, is the conversation du jour; and getting it right for both data owners, developers and organisations has to be one of the highest priorities. Open data and an open society needs clearly defined and refreshed rules and perhaps it is time to start Rights camp or somesuch – it seems to me that it requires the heads of many specialists to get it right, not just one group – and that is always interesting to me.

Developer fatigue

This could well deserve a post of its own, I am not sure yet, we will see. In my (slightly controversial sorry about that) post I wrote last year about developers I touched on the risk of developers turning away from publicly released data if there was an eternal demand on their free time and expertise. To an extent this is beginning to happen now and I would hope that those who are trying to solve the problem of:

We released our data but no one is playing with it, where are all the developers?

… can recognise that there is a very real requirement to engage with developers in smarter ways and to honour their work ethic and abilities. There is no need for me to re-write the developer post from last year, but developer fatigue is very real, is very much here and should be (along with rights management) something that open organisations and industries are addressing with fresh minds. I know it is my utmost priority and is not easily solved, certainly not by simply throwing cash at the problem – although that never goes amiss; but also:

• working with their schedules and optimal way of working, this may not be 9-5
• finding a variety of very real challenges and apparently unsolvable problems
• realising the relevance and value of geek work and utilising that

Looking at the future landscape of a professional relationship between Open organisations and the developer community in a sustainable and respectful fashion is the main focus for me really, and I really, REALLY would love some suggestions if you have them for who is scoping this work – again the edge of reality and future world stuff – not the immediate environment.

So that’s it for today. See you tomorrow!