Every day I love you less and less

Communication used to be fun for me. Digital communication especially so. In the mid 90s it was a blank sheet of paper, or one only scribbled all over in pencil. Common sense was all it really took to say what you wanted to say, online, to the audience you wanted to reach.

Since the digital revolution of the last decade (at least) – and as ‘organisations’ make their online presence a strategic priority – it has become increasingly hard to keep that clear line of sight.

Take website rationalisation in the UK government. It is a perfectly simple and absolutely right policy. The information was often badly managed, not maintained and completely impossible to find, notwithstanding the cash that was being poured into a plethora of websites.

Put in its simplest form, website rationalisation means that all public sector information for citizens can be found on Directgov, and for business on businesslink.gov.uk (corporate information stays with the departmental websites) by 2011. This requires convergence of the content on the two main sites and throws up the inevitable cry of: what about the old stuff? Clearly, content that was written yonks ago needs to be re-written and there are new style guides to consider &c &c. But we can’t just switch off the old sites, it is wrong to have broken links in recorded answers to PQs/PMQs, that information must remain in perpetuity; and once you go down that path you end up in all sorts of mind-boggling complications. The National Archives provides the obvious solution (but that is so not as simple is it sounds – because I am nice I will not drive you down through that particular ‘detail devil’). Nor can you switch off urls, as to do so risks cyber squatting (on non-.gov domains) by questionable folk.

*sigh* you see… by the time you have wound yourself up in knots about this, the simple pleasure of getting the right information to the right audience is swept up in such a maelstrom, you wish you never started! but you can’t do that…

Then along comes a new lovely clean simple way of communicating online: one that is not simply a push of content…

WEB TWO (twenty if you’re cool)

Oh how attractive this is to the frankly ragged people like me; and to be fair the bemused policy units, communication and marketing teams, press officers and the rest: aching to be relieved from the too complicated discussions around getting the ‘old, flat’ content to the spangly new macro-sites (and keeping the… yes you get where I am going).

And so we have seen the remarkable rise in supremely fantastic new work across the public sector digital arena, using social media tools: monitoring, influencing and engaging in the *hopefully* appropriate digital communities… so much so that I cannot keep up (unless I give up the day job and simply watch).

In the last 18 months the most desired digital skill set has not been the ability to craft and manage online content, rather the canny knowledge of the community manager: someone who understands how everything works NOW, and can steer a department/organisation into utilising crowdsourcing, cloud computing and Open Source software.

This is all well and good; it honestly is the Good Life of the internet: community based communication.

But it’s not that simple.

Now we have embraced social technologies we come to the problem of data. In order to continue with this trend of ‘going to the people where they are communing’ we must listen to what they need – and increasingly those who enable us to utilise these social tools demand that the raw data be free. I don’t mean personal data about you and I, I mean the data feeds. Give it to us, they say, and we will make our own stuff in a way that we understand.

The answer to the eternal cry of ‘How can we engage the young people’? Give them the data and let those who know what they are doing, create something that their peers will understand.

And so we find ourselves in a quandary. Not because anyone is precious about the data, rather it is not ready; often it has not been held in any format that is easily shared; sometimes data sets have been held in different formats and updated by a variety of people; borders and boundaries differ &c &c.

In order to free this data, a cross-government (central and local) audit needs to take place; and as with the rationalisation of content onto Directgov and businesslink.gov.uk, a redrafting and ordering of the raw data needs to occur, APIs created, ratification of the accuracy, maintenance contracts drawn up, SLAs…


It’s just never as simple as it seems, but we need to do this work. All of it.

I just wanted you to understand how complicated this all is 🙂

Oh and by the way, go and sign up to this: http://www.mashthestate.org.uk/index


PS Apologies to the Kaiser Chiefs… er not sure what I am legally up for when using a song as a blog title.

16 responses

  1. You say “But we can’t just switch off the old sites, it is wrong to have broken links in recorded answers to PQs/PMQs, that information must remain in perpetuity” but isn’t that exactly what happened with paper? If a file needed to be kept in the national interest or because the curious like me might want to know some obscure fact fo the next pub quiz then, yes sure it went off to the National Archive, in much the same way as websites and digital media should. The trick though, is that only that which is worth saving should be archived. The rest can be consigned to the equivalent of the shredding sack.

    And it might be wise not to turn off urls, but then if departments had been doing as they should and only used dot gov domains all would be well and safe from cyber squatting. Still, renewing a domain name for 10 years at a time doesn’t require too much effort does it?

    As for web 2, I read the other day that to the ultra cool, or maybe it’s those who are complete behind the times,. it’s web squared.

    Could I suggest a remedy for your ills? Take some time out for you. Relax. Have a weekend with twitter, facebook or barcamps. Turn your phone off, don’t turn on your pc and forget about all things e. It can be done. Honest. And it will all still be there waiting for you on Monday

  2. Sorry for monopolising your blog, but can we discuss “The answer to the eternal cry of ‘How can we engage the young people’? Give them the data and let those who know what they are doing, create something that their peers will understand.” over a coffee sometime soon. I’d like to hear your views in answering a similar question that has been put to us.

  3. In order to free this data, a cross-government (central and local) audit needs to take place;


    Were Facebook and MySpace built by some door-to-door survey of the profile and interests of unwilling worldwide citizens? Was Twitter built by asking everyone to enter their current activity into a spreadsheet for publication? Was eBay built by some rag-and-bone man knocking on doors?

    My point is: there are other ways to get people to do socially useful things. Competitions. Keeping up with the Joneses. Ego. Curiosity. And best of all, good old private interest. What would it take to get people to open up their data? An easy platform to do it? Some financial benefit to them or their team when data is reused? Some fame and adultation, or even time off to do it instead of the day job? Who knows. All I know is that audits are dull, expensive and generally futile. Let’s find some other way to get people to do this Good Thing.

  4. This is only daunting if you try to do it all at once and try to get it “right” first time. The web has never worked like that and never will. Broken links are a nuisance but only a crime when they’re broken unnecessarily and no-one even bothered to redirect to the new equivalent URL, if one exists.

    As for data, there have to be stacks of data sets out there that are ready to be published, not least because many of them are, just not in convenient, machine-readable formats. Everywhere you see a list or a table on a web page you’re looking at a likely RSS feed or CSV file.

    That’s why with Mash the State I intentionally started on what I hoped would be the easiest data to set to get. Council news is already published, just not as an RSS feed in most councils. I’m also insisting that these RSS feeds are autodiscoverable through the site home page as then finding all the feeds of all the councils becomes as simple as running a feedfinder on a rarely-changing set of council homepage URLs.

    Frankly, if we can’t get RSS news feeds out of all the councils fairly soon you can pretty much forget about open-API gov.uk on any grander scale.

  5. @Tony: you might have noticed that I have actually been offline for a while now 🙂 I am just blowing the dust off the blog. With regard to Hansard links and paper vs online PQs/PMQs it is a matter of public record, and there is no simple way to know where each and every link went to/goes to: so the simplest way is to stash the lot.

    @Steph I think you are missing the point, although I hesitate to say this to you as you don’t normally. It might be the way I wrote this (badly). The point is that the data sets are not in a fit state to be opened up: ref the data sets we used on Rewired State, most had to be converted; some held out of date/poorly entered data, which then offers up duff results; such as happened randomly on Job Centre Pro Plus, and written about by Harry Metcalfe here: http://www.thedextrousweb.com/2009/03/jobcentreproplus-geocoding-jobcentre-unreliable-datasets/

    Yes audits are tedious, so is migrating/converging and rewriting content for citizen/business. It’s not sexy, that is the point. Doing great stuff using social tools requires clean, maintained data sets offered up in a form that can be re-used.

    I can assure you that a great deal of work needs to happen before we can know for certain that this is the case with the data currently held. Not from any sinister point, rather that the devolved management has meant human error on top of human error, as well as countless ways of collecting and storing the raw information means that one system cannot talk to another: and before we know it, well we do know it, we have the equivalent of the website mush that the transformational government aimed to address.

    It’s boring, its not sexy. but it must be done.

    Sorry… and obviously IMHO only

  6. @Adrian re data sets already ready yes there are many that I know of from a central government POV, but not so clear on local, that’s your bag baby. Some central govnt APIs/and other feeds here http://wiki.rewiredstate.org/?page=APIs

    The Power of information report does recommend that all data should be available in a single place, and I understand is working towards the end of 2009 to do so.

    My point is that there is a job to do here to clean and sort these data sets is all…

  7. @Emma: You’re right that you need clean data. But to stand any chance of getting people to provide it, you need some carrots as well as sticks. To use the Facebook/eBay examples, you get your structured, useful data from which interesting social things are possible by providing benefits which motivate the contributors to go through the pain of reformatting and republishing it. Right now, the benefits for opening up data are unclear, and are generally felt a long way from the source.

    Adrian’s point is a good one – I suspect a lot of the most useful stuff, such as postcoded lists of branches of X, neat spreadsheets of spending on Y – exist somewhere but nobody thinks to convert those Access databases, or commission their web agency to build an RSS feed interface to that particular listing page. If we can find a technical and process solution to overcome those barriers, and provide the incentives to undertake the work, who knows what might be possible.

  8. @Mulqueeny,

    Can we assume that if data is good enough to already be published as a human-readable web page then it’s good enough to be APId?

    A case in point is my council list data which was scraped from a government site. Before I launched my site I spotted a dupe in it (admittedly I didn’t look very hard). After launch someone else spotted another couple of dupes in it.

    For my purposes for that data, it wasn’t a problem. Many eyeballs soon found the bugs and any anomalies were short-lived and easily rectified.

    For the data that’s already out there in one format or another it sounds like what’s needed is not an audit or cleaning exercise but just a usable bug tracker.

    Irresponsible or just pragmatic?


    With the council news data I’m working on at the moment one of the problems some councils have with providing feeds is that their CMSs won’t allow it and they’re not allowed to hack their back ends.

    Microformats may well provide a useful stop-gap solution if not a more permanent one. An hAtom feed can be hacked into an HTML template without touching the back end code. I like to keep different representations of the same data as close together as possible and you can’t get any closer than microformats, which make XHTML both human- and machine-readable.


  9. Yes, you are right. This data can be scraped and can be used in whatever format it is in and anomalies sorted… for some apps.

    And there is some stuff about now…


    and see appendix 3 page 99:


    There are different rules for locally held and centrally held data sets. I can only vouch for centrally held data. It needs sorting basically but yes of course, we can make do.

    re CMSs 🙂 I hear much about the semantic future of government online, but the proliferation of CMS implementation scuppers this: that’s a whole other post…

  10. can I suggest that (a?) step zero is a ‘a cross-government (central and local) audit ‘ of outsourcing contracts, with a view as to which ones have resulted in ‘core business’; i.e. interaction with the Public, being outsourced.; and this ‘core business’ being brought back into the hands of the relevant bodies as soon as cost efficiently possible?
    It is SO much easier and cheaper to change the underlying structures/formats etc, when you own them and don’t have a commercial interest in inertia…

  11. Pingback: Posts about Open Source Software as of April 14, 2009

  12. hi all
    fascinating discussion – it made me feel that we’re trying to stop the ocean from flowing. I’m not technical, so expressions like ‘hacking the back end’ make me smile! I’m an old ‘boomer’ intrigued by the social/cultural implications of the shifting patterns that are emerging and morphing- so maybe we just need to be a bit more zen about it.


  13. @SubtleBlade I like the natural assumption that “core business” automatically equates to “interaction with the Public”, and only wish it flowed through like that more often.

    In reality, once you peer under the skirts of heavyweight public service outsourcing, you rarely see such objectives couched with anything approaching that clarity.

    Core business in many cases means processing transactions, ensuring content (of whatever quality) is there to be consumed, or that security and other hygiene factors are maintained. There’s a big distinction between digital public services of the “democratic engagement” type, and those of the “service delivery” type. This is something that much of the current digital engagement debate seems to gloss over, and which is keeping my mind very busy these days…

  14. Emma

    Well done – nearly two weeks without a post – how does it feel ? Have you rediscovered the family, the outdoors ; let’s hope so. You can come back re-invigorated

    Data sets – I am with the people who say let them be opened, and we will try to use what there is, iterate and improve.

    I wonder what William Caxton thought 2 years after setting that printing press to “ON”. He probably went for a walk through the Kent orchards, and then decided to continue.

    Look forward to seeing you back on-line and take your time.

  15. Alex! Yes, you have not been following me on twitter (lucky you). I actually have been so ashamed at how appallingly I worded this post, so that it sounded as if I was saying the complete opposite to what I meant.

    I am saying free the data, but am also saying that it is muddled, messy and could take a while to find it all.

    Soon I will have something to say, but right now I am concentrating on Rewired State for young people and the freedom of this bloke: http://www.justicefortom.com


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: