So three years in to data.gov.uk and the inaugural National Hack the Government Day and now there is a tick box exercise to “run a hack day”… please… someone… anyone?
Open data is not about hack days and running one does not achieve “engagement with the developer community”.
Background
I met Liz Azyan today. Someone whom I have been aware of for the last few years: blogs great stuff, is principled and keeps herself gainfully employed with a plethora of socially ethical social media support (if you know what I mean).
I was blindsided by her, she is awesome and I think really trying her damnedest to do the right thing in an environment that she totally understands, but with a community she is less accustomed to – yet. Watch this space, and government data geeks: I urge you to chat to her if you get a chance.
One of the questions she asked me today was: What is the next challenge for open government data? So thank you Liz for the inspiration for this blog post, it got me thinking about something I have not thought about much, recently.
The environment
Government has opened up quite a bit of data through data.gov.uk, and has encouraged engagement with keen developers who have been hankering after such information for years.
Industry too has embraced Open, with a small number of notable businesses throwing open their data doors, with good results. I wrote a post about this, I shan’t repeat myself and bore you.
APIs are being released almost every day – developer information overload has maxed out, and now we risk lethal developer apathy.
Developers have attended hack days, meetings in Whitehall – indeed many of them have joined AlphaGov. This is all fabulous; but not scalable to the extreme that the open data dream promises.
The challenge
Making it all work.
It’s all very well having developers working away with this data, but if government is not ready for it, it’s a waste of time.
Take just one example: two incredibly talented developers worked together over the course of a weekend hack last year, coding through the night to create a notification engine for the government Tell Us Once programme. It worked, it would have saved oodles of time and bucketloads of cash – but government was simply unable to implement it. This is one simplified example of 100s of apps created by Rewired State hack days alone, and there are many others.
Now, if you can imagine for a minute being a developer, donating your time – granted, sometimes the hack days are paid, but always weekends away from family – year on year creating apps that would help government and citizens. Solving problems time and time again – quick example, every year the Young Rewired State coders create apps to help them define safe routes to school/friends. Year on year we showcase these to the Home Office – nothing happens. Still no government supported/approved app to meet this obviously critical need.
Why would you bother?
Open data? Awesome, and we are making tracks.
Open Government? HARD, and we are not banging on that door yet.
The reality
The developers who work on government data often do so either out of personal frustration, or a genuine commitment to making the world a little bit better.
Rarely can they reach an audience that would benefit from their app/widget/website on their own and in their spare time, at least not without considerable support. Nor are they doing this for profit, so they are not going to get investor cash.
Helping government do its work better is not a good proposition for your typical angel or VC – the target is government; and only government can utilise the genius that they are being offered.
Lots of tiny arrows
Right now lots of tiny arrows are rained on the government portals day on day, by an increasingly disparate and desolate group of extremely talented people.
Is there any success anywhere? No. Well unless you count the oft-reported GovSpark created by Issy in Young Rewired State 2010, curated by a plethora of supportive geeks and designers and some financial and hosting support from The Stationery Office. But that was a ‘nice to have’ addition to a Prime Ministerial commitment. It was not a revolutionary way to interact with central or local government.
So what’s the next challenge for Open Government data?
Forget the data.
Find a way to enable these revolutionary ideas, apps, websites and widgets that save time, money and mind-numbing frustration from those who have to engage with government.
Do that, and only that.
And when you have done that – then engage the developers again around your open data through hack days, geek advisory boards or whatever means you can.
Until then, let them have a break. They’ll still be there if you do this. If you don’t, they won’t.
And that is ridiculous.
Also, please don’t insist people ‘do hack days’ for you. Here’s the point of a hack day.
Emma Mulqueeny 
Um. Am I allowed to ask why developers wasted time coding something which couldn’t be implemented? Where were civil servants in this process? Did anyone consult with ‘government’?
I ask because as a local govvie who’s less important but still trying to fight the good fight from within, I sat in a meeting yesterday where the benefits of open data were assumed by ‘us’. We’re expecting analysis, visualisation and benchmarking to be done nationally, with cross referencing datasets included such as IMD’s etc to ensure comparison of apples with apples.
Now, we’d buy that, if it was priced sensibly. Our procurement process can cope just fine with paying them as well. We pay SME’s all the time. Suddenly, however, after reading this post, I have a faint cold fear that no one is going to bother to do that and sell it back cos it’s not an app and it aint cool. If we end up in a situation where we release data and no one does owt with it because it isn’t cool data and it’s not being perceived to change the world I shall be cross. So I guess the onus is on me to explain the internal processes of performance management within local government to explain that that visualisation and analysis could save a tonne of money.
Yep. Government was consulted. Government was there, members of the civil service, local government and Ministers are always involved in these. Civil servants tried, Ministers tried, certainly in the case of Tell Us Once – but for one reason or another, the obstacles were just too much. Indeed two of the events last year were commissioned by government directly in line with problems and issues that had been raised as being needed to be addressed. Yet still the internal processes could not be enabled, even to endorse whichever digital solution had been created.
I am not sure about your last sentence – of course some apps made are ‘cool’ as you say, or ‘fun’. Most aren’t – and yes, the onus is on government to work out how to implement these. In my opinion.
In some ways that was the best response I could have hoped for but it makes me feel very sad and quite….
So yes, I guess, lots of tiny arrows. And maybe if us local govvies prove the worth of data in service design and delivery (there’s great work going on all over the place), then perhaps central will be shamed into keeping up? I don’t know. Some days I sorely wish we could scrap the whole damn lot and start again from scratch.You must have the patience of a saint.
Lou, I think you’re looking to the wrong people. The hack days are mostly about clever people having fun and opening minds. They will probably continue to focus on cool apps and whizzy visualisations using data of interest to them and therefore in general reflecting their demographic’s demands. Most of this will be quite useless to people experienced in the fields the apps and visualisations address.
It’s much more likely that data will, over time, become more central to the way local gov already does stuff. Those SMEs we already work with will start to offer services making more use of data.
I think it’s a bit naive to expect a loose network of developers to take an interest in, and seek to understand, the needs of local government.
Local gov didn’t suddenly change its software stack and ICT procurement processes when Richard Stallman unveiled his free software license in 1983. Hordes of hackers didn’t suddenly knock on the doors of councils offering to re-programme their databases. Gradually, companies came along who could bring the spirit and methodology of free/open source software into meeting local government’s needs.
Lots of tiny arrows. yup.
tiny scratches on the digital dinosaur.
Perhaps developers need to be more entrepreneurial. Government is happy to buy software from businesses provided that there is sufficient support over the medium term. Indie developers’ apps usually don’t offer that.
Developers either need to set up their own businesses or find someone who can market, sell, maintain and support their apps for them.
Agreed. We’d happily buy open source software, for example, if the vendor was also bundling support.
Oh sorry, I obviously didn’t make myself clear at the beginning there. The reason Liz had come to see me was because she had been asked to run a hack day on local government data, for Local Directgov and the Transparency Board. This is not developers doing stuff for shits and giggles – this is government asking them to do things. I said that this would be utterly pointless until the internal mechanisms for making such things a reality are in place. It is not the developers who need to be more entrepreneurial! Perhaps now the rest of the blog post makes more sense?
I agree with the thrust of the argument, open data is not the end itself and I very much share the concern that we’re running the risk of alienating the developer community. I think the problem is that government itself is far too disparate and disorganised to manage the kind of dynamic shifts in internal behaviours required to really move from open data to open government. In practice, the in-house geek advocate may well be as distant from the relevant decision making process as an interested/active member of the public with a neat line in letter-writing.
For government’s part, it has to listen better and fast. Across all themes, all levels and with all interested parties. Seeing Hack days (or any engagement equivalent) as a solution in and of itself without thinking any further about what it means for the organisation is pretty standard practice, and you’re absolutely right, that must change before the goodwill vanishes forever. I’d like to say I was optimistic, but whilst ‘engagement with the developer community’ remains a New Thing, then people will read the best practice, do an event, tick their box, miss the point and waste everyone’s time.
But on the issue of engagement, then there is a side issue of tone; language like “raining arrows” on “dinosaurs” makes it appear like a conflict, a fight. Everyone has a great idea how to make government better and a lot are willing to put time and energy into making their point. It may feel illogical (to the point of hair-tearing frustration when you know your idea/app is or could be *the* thing to drive efficiencies and make the world a better place) but knowing who and how to influence will change the world just as assuredly as a killer app.
The solution? Not entirely sure; but perhaps it’s about a more casual conversation with the developer community, proper engagement, not 1-off good-will erosions. This will involve government dealing with risk in its own procurement nightmares but there is a role role here in stimulating a local knowledge community, from developers through local academics and the host of data vis freelancers, market researchers and random interested parties in the middle. Once that trust exists, then government will have trust, and developers get influence.
As an aside, to @AdrianShort’s comment: More practically, on the issue around clustering about larger suppliers – it seems to me that the biggest issue is risk of onward support. IME government procurement people are a nervy bunch at the best of times and government IT procurement people even more so, they want assurance that they’ve got someone to help when it all falls over on a damp February. I’m sure there’s a way of doing this without forcing developers into the arms of the larger companies, some kind of support co-op across a geography or sector, perhaps?
Pingback: The RaxList for July 12th through July 15th | raxraxrax.com
Please people, this is not the time to start losing faith in open data! Dave Briggs posted a very skeptical piece about it earlier in the week, and this feels like another.
Open data is NOT the answer to all government’s problems. It is about transparency, scrutiny, and accountability, and I think most of us understand that holding government to account increases problems for people on the inside; it doesn’t make them go away.
Also, developers are free – they are not cheap! Developers value their freedom, and won’t give it up lightly. They may build something to show you what’s possible, but don’t be surprised if they look at the internal politics of your organisation and shudder. (Developers tend to be hyper-rational, which is why I’ve made a career out of being a diplomatic buffer between developers and the people they call ‘users’.)
Issy Long’s GovSpark project was well received, because it was a one way extract and a mashup of feeds from government departments, and it fitted with targets that had already been set. For an example of a more challenging problem, I’d commend the NHS Appointments Booking application that was prototyped at the Enabled By hack day a few weeks back. It’s a very useful idea that could save the NHS money,but it couldn’t be implemented without treading on a lot of toes, and putting a lot of people’s noses out of joint.
Organisations are not the rational things developers would like them to be. They are social-technical networks that have evolved over decades, where information is power, and control is power, and saying ‘No’ is power. The humility of middle management is a quality you can absolutely not rely on.
Developers are the wrong people to re-shape old organisations. The developers’ instinct is always to form a start up, so they can build something new and logical without having to bother with power and politics and interests and imperfections.
This is why I still believe in Open Data, but I’m a lot less optimistic about the future of hack days. I introduced a hacking culture into a very old ,crusty, conservative organisation, by building things that users could use, and sitting with them, day after day, helping them understand their work and understand what my team could do, so that they had a new vision of their work that empowered them. I see lots in my Twitter stream and on blog posts about people who go away for a day and get really excited with a bunch of people who feel the same, then go back to their day jobs and go ‘Meh’. I’m sorry, but that’s not going to scale, or ever reach a tipping point. Not until you inspire the people who make you go ‘Meh’.
Thanks Emma, this is a great article. I am a developer (loosely speaking anyway, I rely on colleagues for the tricky stuff). And I love going to hackdays. I’ve been to quite a few in the last couple of years and even run one or two of my own. I get a lot out of them (though I’ve never been paid to attend one – some people get all the luck it seems!)
Their strength is in getting a bunch of like-minded people together to show/discover what is possible. Most of the events I’ve been to have involved quite a few government folks, both central and local. They are generally enthusiastic and creative and I’ve learned a lot from talking to them, understanding much better what their problems are – both the problems of the organisation trying to achieve the things they want to achieve, and often the problems of the individual trying to bring new ways of doing things into rather conservative organisations. The gov folks at hackdays give me great faith in humanity and in government! The system might leave a lot to be desired, but these people are going to make it work anyway.
I run a small business – we help people publish Linked Data – and of course going to such events is ‘marketing’ for us in a broad sense, but it’s also about networking with peers, learning, experimenting and usually having a bit of fun.
However at one or two events I’ve been at I get the impression that the organisers think that hackdays are in themselves a way to achieve something concrete and lasting. Which usually they are not.
It’s more about sowing lots of seeds. Most of them will die and that’s ok. One or two might grow into something useful, but only if they are nurtured. It’s usually not about the developers not being entrepreneurial enough. Developers generally have lots of interesting ideas – the difficult bit is having the focus to pick just one and develop it to the point of being something real, keeping your enthusiasm when everyone tells you it’s crap or pointless, rebuilding it a couple of times as you realise what you actually should have done, polishing it again and again, finding innovators and like-minded people amongst your potential customers, building relationships, battling procurement processes, battling risk-averseness, all the while worrying about how you are going to fund it, whether it’s the best of the twenty candidate ideas mulling around in your mind etc etc (sorry started ranting there).
Hackdays do lead to sustainable businesses and commercial applications, but only by a very indirect route. And there will be a lot of hackdays per sustainable business/app. In fact, they usually serve to steer proto-businesses that are already in motion, or to alert the government folks to what is possible and put developers and problem-owners in touch, so that together they might achieve something.
And sometimes I come across the assumption that developers will just do stuff for free. Either because they just love doing it (we do, but there are limits) or because they can ‘make an app and sell it’. Yes, we’ll do that too, but only if someone will buy it! And building a commercially viable business is a long road (see previous rant). I know Emma understands this (eg https://mulqueeny.wordpress.com/2010/11/18/developers/) and many folks in government do too, realising that even techies need to pay for food and accommodation. And beer.
If it’s about meeting the needs of government via hackdays, think of it as a starting off point. It won’t be free in the long run, though it will usually be a lot cheaper and better than the traditional write-tender/have-competition/assign-contract-to-BigCo approach. It won’t happen by itself – the nasty mess that just about works at the end of one or two days needs to be beaten about until it actually does something good; and probably more importantly the government folks need to have an idea of how they will build this into their real work and the will to make it happen.
So I reckon, let’s keep having open gov/open data hackdays, but understand what they can achieve and use them for that purpose. And if you are running a hackday for a government organisation, do it with the intention of acting on and investing in any good stuff that comes out of it.
Sorry, that comment ended up a bit of an essay.
Pingback: Open data ingebed in Amsterdam-Cetrum — Hack de Overheid!
interesting post & well considered responses 🙂
have your say..
OPEN DATA EVENT – ENSURING ANALYTICAL DATA SERVE THE PUBLIC GOOD
The Statistics User Forum and the National Statistician’s Office The Royal Statistical Society, 12 Errol Street, London, EC1Y 8LX, 10.00 (09.30 arrival for registration) 16.15, Friday 30 September 2011
The Open Data agenda presents both opportunities and potential concerns for the management, accessibility and use of analytical data. The outcome of the Cabinet Office consultation http://data.gov.uk/opendataconsultation
(running until 27 October) will set the future direction as to how the agenda progresses and so it is vital that users and producers of analytical data are both well informed of the opportunities and well positioned to shape the outcomes.
This event, co-organised by the Statistics User Forum and the National Statistician’s Office, will bring together users and producers of analytical data. Speakers from the user and producer communities (including a keynote address from the National Statistician) will inform discussion sessions, covering areas such as:
– The opportunities presented by Open Data
– How to ensure that the data are useful
– Data quality and disclosure issues
– Resource implications
Entry is free of charge and a sandwich lunch will be provided. Please pass this invitation on to policy officials and IT specialists in your departments, and also to external users of your data who may be interested. A provisional agenda is attached.
There are limited spaces so early registration is advised. Please email events@rss.org.uk by 16 September with the following information:
Name:
Organisation:
Role:
Special dietary requirements:
Places at the event will be confirmed by email. A final agenda and event pack will be available in due course.
also, Cabinet Office have launched a consultation ‘Making Open Data Real’ at http://www.cabinetoffice.gov.uk/resource-library/making-open-data-real-public-consultation
Interesting article and comments. I work in a central government programme call UK Location. I won’t go full into what we do, but opening up access to location data throughout the UK sums it up more details on the website – link to contact us. http://location.defra.gov.uk/resources/contact-us/
As communications manager I’ve been looking to set up a hack day, or collaborating with others to do so. That said I have seen quite a few of the different hack day sites and the applications that have been created, and will review these for any that use location data and seek to promote both internally and externally, hopefully that helps – so if you know of any concepts, ideas, apps, algorithms or analysis using location data (especially that which has been published on data.gov.uk since May this year) then do contact us – link above.
We run hackdays and have done for government for years. You can contact me on emma@rewiredstate.org if you need more help!