Facebook/Cambridge Analytica — the meerkat moment for platforms and people

This has been a long time coming. Platforms have utilised the easiest business model they could and closed their eyes and crossed their fingers that it would be too annoying, too complicated or too late by the time people started wanting to take control of their own data. That business model being that platforms make money by selling your data to organisations public and private for marketing/advertising purposes.

In Facebook’s case they use what they know about you through your data to offer targeted marketing, whilst they retain the deep data knowledge, a pyramid of access, but it still the same model. It is old fashioned but it works. (I know that Cambridge Analytica take this data and do monstrous things with it, but I am talking specifically of the source: the platform).

The irony that such ‘disruptive and innovative’ social platforms that purport to drive the future of the digital revolution are actually just old stuff dressed up in fairly shouty and shiny new clothes is easily lost.

For years I have been waiting for the other shoe to drop with regards to people wanting ownership over their own data, so that they can *choose* when to share what information with whom and in exchange for what.

I thought this moment would come with health information. In the UK certainly, the NHS struggles with communication between surgery, hospital, clinic and other medical establishments, meaning that whoever is treating you never has the whole picture. It is a relatively simple solution, that I know doctors and surgeons across the land would love to happen, and that is for every patient to own their own records, keeping them under digital lock and key and sharing that information with the relevant medical practitioners at the right time. Much less frustrating all around.

Of course once this information is held and controlled by the individual, the smarter developers, researchers and platform owners would then have to come to you and ask for access, you can then choose to give access to all, some or none of your data — and for what in return. I believed this would be the tipping point and people would then start collecting their data from everywhere and so the tide would turn.

However it was not to be. Facebook and Cambridge Analytica have managed to scare the bejesus out of everyone — leading to a mass exodus, kind of, and a lot of finger pointing and noise, but not very much substance.

Does this fundamentally break the trust between people and social platforms? It has certainly rocked it to the core and I am not sure that Mark Zuckerberg has gone far enough to reassure or to reboot that relationship. But neither does just leaving Facebook solve anything — unless of course you were just fed up of it anyway, but leaving in fear worries me.

It would be good if we could use this opportunity to persuade the social platforms to change their business model. To be grateful that they have got away with it for so long, they must have known this day would come.

Social platforms should give everyone their own data, they should not have it — it should be held in a digital account that a person would own and manage as they would their bank account, (or personal health records if we had got that far yet!).

New platform business models should be built around that premise, making it simple and unerringly transparent for people to share or trade their data.

We all know that there is a multi billion — if not trillion — dollar market in worldwide data trade, if you want to get geeky on this look up the Annual Revenue Per User (ARPU) figures. In 2017 this just tipped over the $5 mark for Facebook alone and it was growing at an increase of 26% in that year, with users in Canada and the US having an ARPU of $21.20. That’s fine, don’t cut it off, just be smarter about it: give people control over their own data, a fair price for use of it — and make the whole transaction simple and transparent.

Here are a couple of links to explore how you can request your medical health records https://digital.nhs.uk/article/6851/How-to-make-a-subject-access-request and here https://www.nhs.uk/chq/pages/1309.aspx?categoryid=68

2014: Pistorious and Wilson: fear and firearms: where *ism equals death and triumph

The world went a little bit more wrong this year. Logic and reason were cast aside. Never were we ever presented with two stories so bound up in data and facts to justify actions and emotion. The result? Chaos. Nothing makes sense. We are all a little bit more scared – of the law, and logic.

Scenario 1: a beautiful South African model was walking along the middle of the road with her friend in the US. A US policeman asked her to move onto the sidewalk. Shortly after this, an altercation ensues and the South African model is shot 12 times for not walking on the sidewalk and being scary in her argument with the police officer. She dies. The policeman was scared.

Scenario 2. a teenager from Missouri is with his paralympic friend in South Africa. After a series of unfortunate events and misunderstandings about who was sleeping when and where, the teenage boy is shot multiple times through a toilet door by his friend. He dies. The paralympian was scared.

Two people died: a white girl and a black boy

Two people shot them

Of course I know I muddled the stories here, on purpose. Here’s why.

In one scenario – let’s just accept that neither Reeva nor Michael should have died – there is a court case with a non-televised judgment by a jury on whether there is a case to try. It is found that there is no case to answer and the killer can walk free. In the other scenario there is a prolonged televised court case that the world watches transfixed, where the killer is grilled from here to hereafter on the public stage – and is handed five years for culpable homicide.

But in the rule of morals and ethics, neither of these results are justified – they cannot be excused; no logic or reason can make either of these deaths OK – even when I swop them back between the white girl and the black boy.

What is pivotal for me about these two cases is that they come in the year the 97er, the kids who grew up with social media and know nothing else, ‘come of age’. They are pretty much ready for the working world now, and this is everywhere: developed and developing markets, these kids are now a universal, unified, socially digitally savvy crew who are insanely mature in communication, identity and influence, and normally immature with regards to work and societal norms. (I write about them a lot here).

Because of these two cases, I really believe that the future of law and crime/punishment will be fundamentally upended by these 97ers. When they become politicians, law-makers, jurors and media story-tellers. Pretty much starting now…

But what does need to happen, as I said in the post I wrote last night, (accidentally on theme), is that we need to become more clever about how we use data, and reason. And for that to happen we need to accept the digital renaissance goes beyond the smartphone and 3D printing – it affects our very base of reason and understanding.

We have access to so much more information, we cannot and must not ignore these data points just because they were not statistically valid for Socrates or the Victorians.

I wrote the following post on my private Facebook account about the Ferguson shooting, and the 97ers I know asked me to post it more publicly so that they could share it. It is my own view of course…

I have to say that when I looked at the articles themed: Darren Wilson shows injuries sustained (…at the hands of an unarmed teenager) I was genuinely expecting to see quite shocking injuries, that would test the patience of a well-meaning public servant. Really they were like when one of the dorts shows me the thing that is totally really hurting and needs a massive plaster – there was barely a scratch. (And the mark on the back of his head I am pretty sure is the birth mark most of us carry from being pressed against our mother’s spine in the womb).

I believe in democracy and the jury service, heartily. So I thought there must have been pictures too graphic to show, seen only by the jury, we are only seeing the sanitised ones. But they really were not – I found this article in my FB feed http://www.rawstory.com/…/fanciful-and-not-credible-cnn-le…/

When I read the facts as stated in the case about why he even was in contact with these two teenagers, I was like… right… and then… waiting for the *wince* moment when I would find it hard to sympathise with the child who was shot. It never came.

I was not on the jury, I believe in the system enough to know that I cannot heap blame on those who found nothing wrong with this child’s death in the eyes of the law.

But when I told my children about why ‪#‎Ferguson‬ was blowing up their social media stream, and the facts of the case (because you should try to do this if there is a big case that will invade their social world), the car journey home settled into an uneasy feeling and conversation that the world became a little less sane and therefore a little less safe.

Teachers worldwide know how to deal with teenage rages and teenage angst. Also how and when to remonstrate and insist with someone, especially someone young. They also know that when they do choose to do so, that there will potentially be some mad rage – either totally justified or for no reason whatsoever. Teachers talk these kids down every day, every day – and face the fear that policeman felt. They are trained to know how to deal with it, and it all starts with the decision to enter into the conflict.

Michael Brown was walking in the road. Darren Wilson wanted him to walk on the sidewalk. It was immaterial really, in the grand scheme of things; but I accept that there were tensions in the community that we will never know who do not live in Ferguson. A bit like in school with uniforms and conformity, there does need to be some rule of law that young people learn – but teachers know how to do this better.

No teacher I know would ever shoot a kid 12 times.

Maybe teachers need to be police officers, or train police officers.

If it were me and I picked a fight with a teenager who became scary as the argument escalated, I would not end the discussion by whipping out a gun and shooting him 12 times.

It is an end. But it is not OK just because he is a teenage boy (can be scary) and most definitely not OK because he is a black teenage boy (moronic use of the frontal lobe).

I wrote about sexism last night, finally, not knowing that the verdict was out on Ferguson, but it does mean I was already in the *ism zone – and I realised as I wrote that I was far more naturally enraged by racial discrimination than gender, because race transcends gender: obviously we have all colours of boys and girls, and the judging starts with colour and gets worse from there.

Then I awoke this morning to this Ferguson decision, and I just had to know more about the facts. Here is a great article about what you can do to learn and know more http://qz.com/…/12-things-white-people-can-do-now-because-…/

To which one of my greatest FB mates Jon Harman replied with a link to data on the teenage brain: http://www.edinformatics.com/news/teenage_brains.htm

*isms: racism, sexism, feminism… &c

{note: I wrote this the night before the Ferguson result. I shall leave it as it is. As for Ferguson these images say it all http://www.slate.com/articles/news_and_politics/politics/2014/11/ferguson_protest_photos_grand_jury_decides_not_to_indict_darren_wilson_in.html}

I tend to retreat from all gender based discussions, only ever tempted out occasionally in discussions on how to encourage more girls into technology. But my words are usually not heartfelt enough – because I am unsure how I feel about it all.

Ultimately the issue is this. Here is what I believe:

That we should all respect everyone we meet, regardless of race, gender, size, age, hair colour etc (I cannot believe that this is news)

It makes no sense to base any decision any person can reasonably make, based on whether you are a man or woman; it is not a sensible delineator. Whether that be who is good enough to apply for a job, or who should be paid what, or who should drive/cook/clean/cry/laugh/play/work/sleep/fly/invent/code/speak/manage/define/write/lead…

So news headlines that read like this in 2014 are vapid, not incendary Turkey president Erdogan: Women are not equal to men

On the topic of men vs women, and men or women being better than the other – well, what are we talking about? Breast feeding? Women will be better. Weeing standing up? Men will be better. There is not general *thing* that makes one or other gender better than the other. It is ridiculous and we *all* know it!

However

The decades of this ridiculous delineation, and all other *isms (including racism which really does get my wick up more than anything), means that our poor children/the next generations+ are victims of unthinking and historic educational/parental rhetoric.

We were all subjected to this, and it is annoying to me that little has changed with boy/girl toys, books, games, career advice – just pure laziness really on behalf of the people who should be doing this as a part of their job or life.

It means that people like me, who have a very little influence in this space, can only really add a lobbying voice in our spare time and in our actions as parents and entrepreneurs.

But we have to in the absence of proper activism in all those flogging stuff to kids, whether it be schools, apps, games or whatever – they reinforce the gaps, because it is an easy dollar/pound if you get the parent market. And let’s face it, the affluent parents are more likely to be looking for conformism and the *in* club, not the challengers.

It is so annoying!!! (Although Mattel got a bit of a bite in the bum with the recent Barbie book – so it is starting to bleed over, thank goodness).

I have to say that just as damaging is the rhetoric of charity songsters, pleading for everyone to pity and pay for “Africa”. This article puts it much better than I could.

The world is flat. No, the world is round.

We just must stop making these assumptions based on such a massive slicing judgment: gender, race, politics, religion.

I know it is geeky, and not everyone’s bag, but really everyone knows that we are all made up of a complex mixture of stuff, and our “data” that makes up who we are is rarely finally defined by gender/race/politics etc (religion aside, I accept that it will define all things in life for the devout).

We are too complex for sexism and racism and any flipping *ism you can throw at me.

Humanity realised a while ago, through learning and science and wondering, that the earth was not flat. We know it is round now. We also know a hell of a lot more than this and the earth being round, not flat, does not define the way we consider our multi-complex relationship with it.

Let’s just please do everything we can to minimise these divides, and at every opportunity look for the other data points that really are relevant.

What’s the next challenge for Open Government data?

So three years in to data.gov.uk and the inaugural National Hack the Government Day and now there is a tick box exercise to “run a hack day”… please… someone… anyone?

Open data is not about hack days and running one does not achieve “engagement with the developer community”.

Background

I met Liz Azyan today. Someone whom I have been aware of for the last few years: blogs great stuff, is principled and keeps herself gainfully employed with a plethora of socially ethical social media support (if you know what I mean).

I was blindsided by her, she is awesome and I think really trying her damnedest to do the right thing in an environment that she totally understands, but with a community she is less accustomed to – yet. Watch this space, and government data geeks: I urge you to chat to her if you get a chance.

One of the questions she asked me today was: What is the next challenge for open government data? So thank you Liz for the inspiration for this blog post, it got me thinking about something I have not thought about much, recently.

The environment

Government has opened up quite a bit of data through data.gov.uk, and has encouraged engagement with keen developers who have been hankering after such information for years.

Industry too has embraced Open, with a small number of notable businesses throwing open their data doors, with good results. I wrote a post about this, I shan’t repeat myself and bore you.

APIs are being released almost every day – developer information overload has maxed out, and now we risk lethal developer apathy.

Developers have attended hack days, meetings in Whitehall – indeed many of them have joined AlphaGov. This is all fabulous; but not scalable to the extreme that the open data dream promises.

The challenge

Making it all work.

It’s all very well having developers working away with this data, but if government is not ready for it, it’s a waste of time.

Take just one example: two incredibly talented developers worked together over the course of a weekend hack last year, coding through the night to create a notification engine for the government Tell Us Once programme. It worked, it would have saved oodles of time and bucketloads of cash – but government was simply unable to implement it. This is one simplified example of 100s of apps created by Rewired State hack days alone, and there are many others.

Now, if you can imagine for a minute being a developer, donating your time – granted, sometimes the hack days are paid, but always weekends away from family – year on year creating apps that would help government and citizens. Solving problems time and time again – quick example, every year the Young Rewired State coders create apps to help them define safe routes to school/friends. Year on year we showcase these to the Home Office – nothing happens. Still no government supported/approved app to meet this obviously critical need.

Why would you bother?

Open data? Awesome, and we are making tracks.

Open Government? HARD, and we are not banging on that door yet.

The reality

The developers who work on government data often do so either out of personal frustration, or a genuine commitment to making the world a little bit better.

Rarely can they reach an audience that would benefit from their app/widget/website on their own and in their spare time, at least not without considerable support. Nor are they doing this for profit, so they are not going to get investor cash.

Helping government do its work better is not a good proposition for your typical angel or VC – the target is government; and only government can utilise the genius that they are being offered.

Lots of tiny arrows

Right now lots of tiny arrows are rained on the government portals day on day, by an increasingly disparate and desolate group of extremely talented people.

Is there any success anywhere? No. Well unless you count the oft-reported GovSpark created by Issy in Young Rewired State 2010, curated by a plethora of supportive geeks and designers and some financial and hosting support from The Stationery Office. But that was a ‘nice to have’ addition to a Prime Ministerial commitment. It was not a revolutionary way to interact with central or local government.

So what’s the next challenge for Open Government data?

Forget the data.

Find a way to enable these revolutionary ideas, apps, websites and widgets that save time, money and mind-numbing frustration from those who have to engage with government.

Do that, and only that.

And when you have done that – then engage the developers again around your open data through hack days, geek advisory boards or whatever means you can.

Until then, let them have a break. They’ll still be there if you do this. If you don’t, they won’t.

And that is ridiculous.

Also, please don’t insist people ‘do hack days’ for you. Here’s the point of a hack day.

Open data then and now – and er next?

We at Rewired State have been asked to supply evidence to the Public Administration Select Committee. Adam McGreggor is going to go and represent us – as he is very clever at speaking – and answer questions on “IT’s role in government” on the 15th March. Today we have been speaking to the clerk and discussing one particular point: Why has there been no progress in people using public data since 2002?

So I thought in the spirit of my daily blogging we could chat about that here a bit. Firstly, in spite of calling all of our mates and getting hold of none of them, trying the Office of National Statistics and getting nowhere, tweeting and googling – we cannot for the life of us find anything relevant at all from 2002.

The genius we did find came, as ever, from twitter (thanks @harryharrold), when Tim Davies shared this with us from his dissertation:

It is utterly, utterly brilliant – but we are still bewildered about the 2002 thing. We are poring over this this afternoon, it would be ace to have some of your thoughts and ‘interesting points’ on it – and also how you think this line might extend.

I guess the more pertinent question is what happens next? How might this be shaped? It would be seriously interesting to run a Shell future scenarios piece of work on this; now that would be fun. Shall we?

Too much information

On day two of my week of blogging every day on what is niggling and making me think (see yesterday’s post if you are confused) I am going to write about a topic of daily discussion with colleagues and people in the digital industry and see if there is any more light to be shared on all this.

We seem to find ourselves in a world of over-communication, over-sharing and in the battle cry of Open: too much information. I am not sure that there is anything wrong with this, but what feels iffy is the fact that – again – there seem to be not enough people out there on the horizon carving out the future for us all in the following areas:

• personal data and rights management
• developer fatigue

Personal data and rights management

Working as I do in the Open data world I know for a fact that we are very careful to keep the data we work with non-personal, and endeavour always to make sure that cross-referencing data will not enable identification of an individual. We also do not go anywhere near personal information. William Heath has long been *the* voice in my world for identity, a fully paid up member of the Open Rights Group and jointly owns the company Mydex. William is one of those people to whom I was referring yesterday – we need him out there on the edges of reality and I would love to hear a lot more from him about his future vision. Personal data is obviously the next big discussion, what is the personal decision making/prioritisation that happens sub-consciously when a person builds their facebook page and sets their privacy settings? Why do people say yes or no to a store card? What is the value metric for personal data? Why is everyone (almost) religiously determined to hold back personal information from government, or treat government requests for personal information with caution or even suspicion? I don’t know the answers to this – but I hope to find out more. Please do point me to people who are researching this publicly and with a horizon view (other than William!)

I also am a bit surprised that the Open Rights Group are not being clasped to the bosom of every organisation opening their data – rights management, believe you me, is the conversation du jour; and getting it right for both data owners, developers and organisations has to be one of the highest priorities. Open data and an open society needs clearly defined and refreshed rules and perhaps it is time to start Rights camp or somesuch – it seems to me that it requires the heads of many specialists to get it right, not just one group – and that is always interesting to me.

Developer fatigue

This could well deserve a post of its own, I am not sure yet, we will see. In my (slightly controversial sorry about that) post I wrote last year about developers I touched on the risk of developers turning away from publicly released data if there was an eternal demand on their free time and expertise. To an extent this is beginning to happen now and I would hope that those who are trying to solve the problem of:

We released our data but no one is playing with it, where are all the developers?

… can recognise that there is a very real requirement to engage with developers in smarter ways and to honour their work ethic and abilities. There is no need for me to re-write the developer post from last year, but developer fatigue is very real, is very much here and should be (along with rights management) something that open organisations and industries are addressing with fresh minds. I know it is my utmost priority and is not easily solved, certainly not by simply throwing cash at the problem – although that never goes amiss; but also:

• working with their schedules and optimal way of working, this may not be 9-5
• finding a variety of very real challenges and apparently unsolvable problems
• realising the relevance and value of geek work and utilising that

Looking at the future landscape of a professional relationship between Open organisations and the developer community in a sustainable and respectful fashion is the main focus for me really, and I really, REALLY would love some suggestions if you have them for who is scoping this work – again the edge of reality and future world stuff – not the immediate environment.

So that’s it for today. See you tomorrow!

Small essay on Rewired State, Open Data and future of public service

So I have been out of government now for over two months… seems a lot longer. It has been incredibly good to focus on Rewired State properly and to try to grab hold of and contain what we started, so that it does not spiral into something useless. The Guardian have been utterly generous and supportive in this move and I have been working with them as well – which continues to be fun.

Some of you may have noticed that we are talking quite a bit with the dev community and our friends about the future of Rewired State; these discussions have been lively and brain aching – but very good. We are running one in Manchester on the 22nd October, as there is a large and engaged RS dev community there.

So, I thought I would do a very small update (that escalated) on what seems to be coming out of this time of looking at RS’s future.

The data goldmine

Firstly, there is still a great need for people and businesses to centre themselves around the trojan work of the Cabinet Office in opening government data through data.gov.uk and legislation.gov.uk. Those of you who have been fans of Rewired State may remember that when we ran National Hack the Government Day in March 2009, there was no such thing; Richard Pope scraped all the data he could lay his hands on in the days and weeks before the event and the potential for what could happen with open data was laid bare for all to see.

Within no time Cabinet Office were working up data.gov.uk and brought in all manner of luminaries, futurologists and geeks – as well as a small number of us already working in departments across Whitehall – we set to work: teasing, coaxing and cajoling the data out of startled officials, who had no idea of the value their spreadsheet/database or even micro fiche (on one occasion). It was an incredible achievement and one that we should be proud of; it was speedy, open and a bit messy – but how fabulous and refreshing, and what superb grounding for creating a crude base that works for everyone – something that we can build together as well as tailor to individual needs. (I would like to rally people back to this cause now – it needs to be worked at and supported as a community, we can do that – there is no *way* of doing this, but I know that Thayer Prime @thayer, Richard Stirling @rchards and James Forrester (sorry have forgotten yr twitter handle, James!) would gladly tell you how you might help).

Data.gov.uk provides us with a rich seam of sustainable information, information that could be the building blocks of the revitalisation of enterprise in this country. Right now, the people who are realising the benefits are developers with defined and respected skill sets – either for worthy social causes that have always bugged them, or perhaps more commercial use – like timetric.com <- those boys were always years ahead of the market and I am so pleased to see them growing steadily and continuing their extraordinary business. Yet still, even though there are mobile app developers out there making stuff that we all find useful, it is still really the preserve of the geek – this data wash – and in order for government to really see the value of this, it needs to translate into value for the general public, a circular feed of data that washes through the community bringing information, perhaps income  and brings communities together – as is the wont of this digital age.

The work now needs to focus on how we interpret the information on data.gov.uk into something of value to everyone, not just in the way they can receive it, the ‘cut’ of the data that might perhaps give them a differing view of a school, rather than just Ofsted and their data, for example – but how can everyone have a go.

Efforts like Landshare really grab my attention. If you go and have a look at what they do, then imagine that local councils work with Landshare, and use the data they have to perhaps build a view of their community – perhaps the elderly lady who has given her garden to Landshare, has a drive that she would also like to rent out as she is no longer driving and has no car. Now she can create an income and join a new community. If she is encouraged to do this online, perhaps she would be willing to add to an information drive to map – say – all the disused land in the UK, and provide feedback online to build such a data set; or post boxes in her area, or anything really – you see where I am going with this? So from the open data drive, there is potential for a person who would be the last person you would expect to derive direct financial and community benefit – is that not the whole Big Idea?

We are not far off this, but we are drifting a bit, I feel, and very fragmented. With the retirement of Andrew Stott – who is a great and, when necessary, brutal champion of open data – and an absence of an obvious figure-head – it has crossed my mind that the Coalition may not see this as a focus for their agenda. If this is the case then I think we need to build our own head of steam, and drive this movement to the tipping point we need to enable the explosion of innovation and potential revenue.

It’s tricky at the moment because we are in the inevitable chaos stage, with data not exactly pouring but stumbling and limping from departments into data.gov.uk – Martha working to get people online with raceonline.org, Helen working with UK Online centres, Open Knowledge Foundation, My Society and us working on a variety of challenges that err on the side of the geeky as well as the Guardian Open Platform and their more commercial work with government and industry. The emergence of initiatives such as linkedgov.org – a dev community based effort to make the data make sense shows the tiny shift outwards from the information trough that has been feeding the data-hungry devs – if I remember one thing from every single Rewired State event, it has been the constant cry for more data (so much so that on many occasions the audience has joined in as the developers end their presentations with pleas for data) – well luckily, there is always more, and always will be: lovely sustainable stuff that it is 🙂

But taking the big vision, the proper head above the parapet moment, what has to happen as a big leap into translating this stream of data and tables into a valuable source of information and commerce to everyone who is not blessed with binary brilliance. This is unlikely to be one thing, or enabled by any one person, but it will be a steady rise in the number of initiatives that realise value of this information for many communities, that weave themselves into the heart of every day life that will bring us to this epiphany.

So… please can we all rally back around data.gov.uk and start having a look over our shoulders as we work the code, and see where we can sling something of value out there, the more we sling, the more likely we are to build value for everyone.

To this end, these are the events Rewired State is running over the next few months. We will create many prototypes from the public sector data, some will go on to great things, some may become parts of other things and others will just slumber on until they may be useful in the future. In the mean time, we have found some brilliantly fun ways of playing!

Very important point to note here is that Rewired State will work with everyone if they are looking to do things better, are asking for our help in order to do things better – you will definitely see a rise in a number of events that are sponsored by companies that may seem to be from ‘the past’, we always look at the ethics and drivers for working with such bodies, so please trust that we are not just taking any buck we get. We work on a 50:50 balance. 50% of the work we do we try to be fun/creative/worthy and 50% is commissioned, paid help. This way, certainly for the next 6 months, we should be able to move forward and bring value, whilst remaining true to our original plan of showing government what is possible, whilst they show us what is needed.

So, here goes – in date order:

6th October sees the start of a very exciting few months, with developers in the community in and around Rewired State working with NHS Choices data and digital signage boards in UK train stations and bus stops http://rewiredstate.org/events/nhs-big-screens. The idea here is to set a challenge to see what can be done with raw public data, using a slightly different medium. These boards can play such a great part in games, mobile and interactive web applications and we are very excited about what will come of this developer challenge. We still have places, do sign up, it is for glory not pay – but will definitely inspire the old creative developer juices. You will be able to see what we did, live, on these screens between the 13th and 31st December across the UK.

On the weekend of the 30th and 31st of October we are running a Carbon and Energy hack event with 10:10, Carbon Culture and the Guardian http://rewiredstate.org/events/carbon-and-energy. This is a true hack weekend. There is so much that everyone would love to do, but so little time, so this is a bit of a playground event for developers of all creeds. It is not paid, we do need sponsors for this, however – as we need to cover some costs – but it will be fun.

On the 13th and 14th November we are running a developer weekend on behalf of DotGovLabs http://rewiredstate.org/events/dotgovlabs_weekender. They are going to be launching a platform to bring together Big Society challenges and it is a massive experiment. From the beginning they have asked for Rewired State to be involved, after we ran an event with them last year, and we are very happy to throw our hat in to see if this is a potential way to match data and real need – using agile development as one of the potential solutions, or a part of the solution at least. It is an experiment. It is paid and we can accomodate a few more developers (as I write this).

There are a few more events lined up for this year, including a postponed one for the Technology Strategy Board in November, but we won’t recruit again until we have signed everything up properly.

Next year we will be running National Hack the Government Day as ever in March, Young Rewired State is currently billed for May, and we are testing the waters to see if there is any interest in Rewired Stately: an event aimed at developers aged 50+. We will also run events that we are asked to run, and we will maintain the balance between paid/sponsored/free.

Currently we are an incorporated Limited Company, needs must to work with government – but we are working right now as a not for profit, we don’t have shareholders who wait for dividends – well right now the only shareholders are myself, James Darling and Richard Pope, and we are not taking dividend payments. Any profit we make goes back into running the unpaid RS events; (in the past we put our profit into the community, such as HackSpace, but that is getting a bit more tricky as everyone has less cash to spend). But we are growing, and we will need help to grow, so it may well be that in the next year or so the basis of our funding may change, but we won’t hide this if and when it happens.

Rewired State: Justice and Home Affairs Hack day

On Thursday, 11th March 2010, ten Rewired State developers (including two of our young rewired staters) hacked Home Office and Ministry of Justice (MoJ) data for a few increasingly panicked hours (panicked because of time restraints 10am until 4pm).

The lovely people at Osmosoft hosted the day and we presented the 9 final apps in the Home Office to Home Office Chief Information Officer: Annette Vernon, the Cabinet Office Director of Digital Egagement: Andrew Stott, Sir Tim Berners-Lee, Nigel Shadbolt, and roughly 80 Home Office and MoJ policy, statisticians, ecomms and Press Officers.

Here are the hacks:

Better Crime Maps Richard Pope created the application he has always wanted to, whereby he took away the fear created by crime maps that made people aware of crime in their area, but perhaps could not afford to move away from – by creating a ‘not the crime map, crime map’ – here is the true detail of crime in your area, and here are the contact details of people who can help affect where the police focus their efforts… ’nuff said

Crime Stats facebook quiz A real life facebook quiz of only six questions that enables you to guess about the crime stats in your area based on your conception vs the reality – not only is this an unobtrusive way of you finding out the real stats, but it is also a great way of offering the Home Office a feel for how people are feeling about crime in their local area. Valuable to both department and society

Police complaints data A new visualisation of police complaints data that shows a different view other than text for police complaints in an area, per type of complaint, rank and so on. The only thing borking this is that there is no data that gives the number of police in each ward – so it could look skewed if there are high complaints in an area with few police etc etc. However, after the hack day, the CIO for the Home Office is determined to get that data made available for the application to work properly  – WIN

BBC Data.gov.uk this was really exciting for me. Ben Griffiths (who always comes through with the most insightful hack at any event we run) started work on an application that works a bit like sidewiki – whereby any BBC news article (and of course this could be syndicated out) is marked up to link to the relevant data set that informs the story it is telling. I have been following Ben Hammersley’s work on how the publishing industry needs to change the way they are creating content, thereby data, online. Ben Griffith’s hack plays beautifully into this, news industry take note.

Safer journeys this is a simple principle hack from one of the Osmosoft guys, Simon McManus, who created an idea for a hack that gives information on crime in an area per underground station. useful as it is time lead, however, not so good as it is per area not station – we need to get the station crime data – a set that is not yet created, but hopefully we can get it made.

How’s my Town was one of the creations of Josh (Young Rewired Stater) that called up all information on an area and gives it a percentage rating  – as he says ‘for the ultimate snob value’. Currently it seems to be a bit broken but he assures me it will be working again soon!

Two other apps: Met Complaints (an iphone app) and Helicoptr (lovely vis of where the police helicopters are and have been over set periods of time – by Stephen – the other Young Rewired Stater there) have not yet been loaded but will be.

Paul Clarke took some great photos (which I have liberally reused in this post):

Photos of the hacking here: http://www.flickr.com/photos/paul_clarke/sets/72157623477510033/show/
Photos of the presentations here: http://www.flickr.com/photos/paul_clarke/sets/72157623602064232/

There was no wifi or reception in the conference room so no tweeting, but actually the point was to enthuse and inspire the Home Office and MoJ officials, not everyone else. And it worked – we will see what happens next but everyone was very buzzy afterwards. The Q&A was challenging at points, but the usual subjects were well-debated: you don’t understand the data, how will people know what we mean, loss of control etc as well as the shiny, bouncy, happy people wanting more and wanting to race back to their office to tell their colleagues.

Then we all went to the pub.

Next up: National Hack the Government Day if you are a dev sign up, sign up, if you want to come and watch – sign up

Rewired State’s massive March

Those of you who know me, or follow me on twitter (@hubmum), can’t have failed to notice that we (Rewired State) are putting on quite a few events in March. Also, the more eagle-eyed will have noticed that we have also turned ourselves into a grown up Limited Company.

Why?

As James Darling observed on our blog post about this – we were ready to hang up our collective boots and move aside after the launch of data.gov.uk and the apparent very public commitments to opening all data and enabling transparency. But then there began a series of conversations and online discussions about how the value of data.gov.uk could be explored, departments were asking for help unlocking some of the stories their data could tell; this, alongside the unsettling assumption that developers would carry on playing with this for free and would eventually come up with the *big* one (other than THE newspaper – a defining moment in data realisation), meant that we thought we still had a point.

We decided to reconvene and see how we could help government departments get to grips with the untold value of the data they were releasing, whilst showcasing the talents of the Rewired State developers in not only creating exciting applications, but also in problem solving using Agile methodologies. And so Rewired State was reborn – with more of an organic message (as you will see when our brand new site is launched next week!) <- we are all about Agile.

Why a proper limited company? So that we can be paid, it was achingly difficult getting sponsorship and past procurement issues for the other hackdays without a formal company; also, it seemed the right thing to do. We are forging ahead with the view that what we are doing is right, and we will see what value we can provide for developers, and for government. It may be nothing, it may be something – we’ll see.

What’s on in March?

11th March Rewired State: Justice and Home Affairs – Ten developers are going to play with data from the Ministry of Justice and the Home Office. It is an open hack day with a presentation at the end of it to senior officials, comms teams, Press office and the CIO’s office. Statisticians will be invited to go and have a look at what is happening during the course of the day.

After the presentation, each ‘hack’ will be written up with details of what data was used, whether it was available (or scraped), how long it would take to create a fully operational version and an idea of how much. This is not to say that Rewired State would then take on the full development, but to give the department a realistic starting point, should they want to develop it themselves. (The IPR remains with the geek, of course, as with anything we do).

I have to say that everyone within the departments that I speak to, is very excited about this (as are we).

19th and 20th March Rewired State: DotGovLabs – 30 developers will work over the course of two days solving some of the more specific challenges faced by Directgov, businesslink.gov.uk and NHS Choices: including localisation, personalisation and a cross-site(s) topic of pregnancy. (We are hunting pregnant (or young parent) developers for this – so please do spread the word).

The event will start with developers working with key people from the three super-sites, as well as policy/departmental bods. Challenges and issues will be explored then the developers will be left alone to work on some technical solutions. The following afternoon the groups will meet again and see how the applications are coming along, ready for mass-presentation at 4.30pm on the Saturday to invited people from the three super-sites and across government.

Beautifully, we have decided to hold the National Hack the Government Day on the 20th, so there will be an almighty powerhouse of development going on in The Guardian offices, with the 30 developers from the dotgovlabs hack, cheered along by the wave of random Hack day developers – who may themselves come up with some interesting things for the dotgovlabs people (but the presentations from the National Hack day will be later – and over beer and pizza as opposed to tea and biscuits!) and will be completely random.

Last but not least, we have the much evangelised Rewired State: Culture event on the 27th March. Mark O’Neill, CIO for DCMS and brilliant blogger, wrote this about what is known as ‘Rewired Culture’:

Britain is a creative culture. We have a vibrant developer community, a growing and active entrepreneurial base and a vast, rich array of culture assets. How can we bring these together to create new opportunities for data owners and developers? How do we encourage links between data repositories such as museums, broadcasters and the wider community like data.gov.uk or the “London Datastore”? How do we ensure that the exciting work already underway in a number of organizations is shared more generally, so even smaller bodies and SMEs can learn from best practice and find workable routes to market? What are the cultural content business models for the 21st century? How do creators, curators, developers and entrepreneurs work together?

Rewired Culture is a day long event on 27th March 2010 organized by DCMS and Rewired State which is intended to explore these issues and more besides.

Rewired Culture has two strands – the first is a hackday bringing together data owners, data users, developers and people with ideas to see what they can create in a day. This builds on the very successful Rewired State events held in 2009.

The second strand is a halfday unconference style event starting at midday and running in parallel with the hackday for data owners, entrepreneurs, data users and communites to discuss business models, funding mechanisms and challenges.

We will be encouraging constant communication between the two strands because by the end of the day we want the event to have come up with a number of projects that people want to take foreward on technical or business grounds, preferably both!

As you can see: four very different events

And we are wanting to work up our offer back to departments/organisations around these four – plus a few other one off events during the year. Please bear with us, we won’t be able to answer too detailed questions about our future right now – but we are going to carry on, and yes, we are definitely doing another Young Rewired State (we are also pretty chuffed that some of the younger devs are rocking up to some of our other events).

This has all happened rather fast, and as with anything, the last thing we have focused on is our own website – we are working all the hours to get everything sorted, in the mean time, if you want to come to any of the days, either signing up as a developer or as a voyeur, here’s the rather haphazard sign up:

Rewired State: Justice and Home Affairs http://rewiredstate.org/home (As there is such limited space, this will be invite only)

Rewired State: DotGovLabs email info@rewiredstate.org with the subject line ‘dotgovlabs’

National Hack the Government Day email info@rewiredstate.org with ‘National Hack’ in the subject, who you are, whether you have been to a Rewired State event before, and if not – an example of something you have created

Rewired State: Culture Sign up at: http://rewiredstate.org/culture but everyone wants to come, so you have to beg

Want us to do one for you? So long as it’s not March – I’m sure we can, just email info@rewiredstate.org with the subject line: ‘It’s OK, it’s not in March’

Otherwise – we will keep you posted

*I get asked quite a bit who the Rewired State team is, here goes: James Darling (boy wonder), Richard Pope (super clever), me and a new addition Rob Carter (@hubdad) the sensible money/business man. But the extended family is definitely The Guardian who host many of our hackdays and Harry Metcalfe, founder of the Dextrous Web, who has been extremely generous with his time, thoughts and brilliance. And, of course, the team behind data.gov.uk – who we won’t individually name as we know they are a little bit busy!

** we need sponsors for The National Hack the Government Day: only beer/pizza/lunch money for the 100 geeks and the show and tell guests email us info@rewiredstate.org with ‘sponsor’ as the subject line

*** next you will hear of us will be specific hack day deets and the launch of the new website

Sentiment analysis – analysis

Last night’s supper was brilliant for a host of reasons, not least of which being the food served at the awesome Rules restaurant in London. I was there with, amongst others, Chris Condron – a wonderful man I met through Young Rewired State. Conversation ranged from the antics of Edward VII and Lillie Langtry to sentiment analysis (the former I am comfortable with, the latter I was fascinated by).

Anyone working in the world of digital media is used to the feeling of playing catchy uppy, adopting the look of the slightly baffled whilst trying desperately to keep up and learn. That was me last night.

Today I hounded Chris for an explanation of sentiment analysis, and he gave me the following:

Crudely, semantic analysis gives you a non-statistical (unlike search engines) sense of what something (say, an article) is about.

Sentiment analysis uses semantic analysis techniques to measure that against a set of known criteria, eg is a text pro or anti something?

It’s already being used in the financial world. It could be a really cool tool (especially when run across live data [such as Twitter] rather than flat text articles) for brand management. Marketers can use it to test in real time the public’s reaction to a product launch.

Before rapidly handing me over to his much heralded colleague Dr Jarred McGinnis:

It’s a computer that analyses text for keywords and phrases and determines the positive or negative sentiment of the story. For example, “Paddington Bear sucks” would probably be determined to be negative where the statement “Paddington Bear is a hero” would be positive.

The technology is not very accurate but still useful. One example of its use is to monitor mainstream and social media for negative or positive trends with respect to your company or one of its products.

Now, I am rapidly becoming a huge fan of championing the ability of talented people whilst genuflecting to the power of the computer. As the work in my field diverges ever more on information, data and ontologies, so my respect for the statisticians and analysts grows; and my understanding of the limits of computers, and the limits of humans. I am not sure whether to proudly embrace my ever-increasing knowledge of librarian skills and understanding of the importance of cataloguing languages: Dublin Core and the like – or to run away. What I do know, is that it is increasingly important to spend time making sure that the human involvement in the digital revolution is carefully balanced with the awesome power of the computer.

So we come to sentiment analysis. Whilst doing my own homework on this tonight, I understand that it is essentially an ontology of words or phrases that are assigned positive or negative associations. Using this as a framework, you can throw a whole load of content at this wall of good and bad – and have it separated cleanly into positive and negative, using the brilliant processing power of the computer.

To give you an example that Dr J showed me http://www.newssift.com/index.jsp. Using the search box, I can put in a topic. The resulting page gives me bucket loads of information; the graph on the top left is the sentiment analysis, some useful MIS and source material is included and the main centre gives me the search results being analysed. I won’t do it for you, you go and play.

However, what has kept me most intrigued is the semantic search bit, (by semantic I mean refined associated search). Once you have run your initial search, the results page lets you add search terms to refine the results and gives you ever more detailed information.

Now, I don’t know how you would use this – I would say with a note of caution: this is just data being thrown at a pretty brutal analysis tool of positive and negative feeling (something a computer can only do by cataloguing good/bad feeling words against online content) – but it is the first step I have seen in digitally automating the mood of the nation on any given topic.

Please do let me know of other tools that you know of that have refined this further, (don’t google it – I already have!), and please do let me know your thoughts on this. I will certainly be playing about a bit more with this stuff.