Archives for posts with tag: Data

You may be under the impression that when you search for something on Google the results you see are the same as anyone else that performs that search. This isn’t the case, and hasn’t been for a long time.

In 2009 Google went full steam ahead with personalized search. The idea was to look through your internet history, your Gmail and all the rest of your Google products and look for signals that would enable Google to tailor a search results to exactly what you are looking for.

As well as looking through your history, Google has always wanted to look at your social network to make your search results more relevant. The only problem with that is it doesn’t own any social network data – a social network like Facebook is a ‘walled garden’ that Google can only peek in from the outside.

The arrival of Google+ allows Google free-rein over your social data and will herald the age of a new buzzword – social search. Social search is the process whereby your social network (or social graph) affects the results of a Google search. By looking at the content that has been created or shared by people in my social graph, the results I get from a Google search will be more personalized than ever before.

I’ve already seen this in action. After searching Google for ‘SOPA’ (the Stop Online Piracy Act) I found myself reading from a website that I had never heard of. I traced how I ended up on this particular page and it turns out that someone I have in my Google Circle network was a writer for this website and had +1’ed the article.

This is great, right? Google search results will become more relevant, based upon people like me and less likely to be manipulated by dirty SEO tactics. Some people have even gone so far as to call this a ‘Socratic Revolution’ – suggesting that the era of personalized search is akin to the philosopher Socrates placing man at the center of the intellectual universe.

There is, however, a dark side to personalized search that has been recognized in a book called  ‘The Filter Bubble’ by Eli Pariser. The problem, he argues, is that this personalized ecosystem of knowledge acts as a mirror that reinforces what we believe without allowing the possibility of our views being challenged. Each new layer of personalization strengthens the walls of our own bubble – satisfying us with the information we want to see instead of offering new ideas. Or as he puts it, we are being given ‘too much candy, and not enough carrots.’

Whilst the Filter Bubble emphasizes our uniqueness, it acts as a centrifugal force – it pulls us apart from one another. With enough personalization the front page of Google News will be different for everyone, removing the kind of shared experience we used to have with a newspaper. Also, the Filter Bubble is invisible – we don’t know the maths behind how these algorithms define us. And with the increasing omnipotence of Google – it is difficult to not be a part of it.

So the arrival of Google+ social search marks a new era of ‘invisible autopropaganda’ that will continue ‘indoctrinating us with our own ideas’. What it will also mark is the start of a new form of marketing and campaigning – especially in the run-up to the 2012 US election. If I tap ‘Healthcare’ into Google I will be presented with the healthcare articles that my network has shared. Both the Democrats and the Republicans will have to fight to ensure that they have the right people inside the voters Google Circles.

Whilst we may still be at the dawn of social search – the correct techniques in this area could eventually make or break a campaign. Could 2012 be the year that Obama leverages Google+ to win the election?

The EOL is aiming to be a single online resource cataloguing all life on this planet. Collaborating globally with many other collections, the site is working to provide a webpage for every single one of the 1.9 million species on the planet.

Each page will contain photo’s, sound-clips, videos, maps and articles written by experts and verified by the scientific community. There is also prominent ‘threat status’ section, letting the viewer know how endangered the species is.

On every page there is a dedicated community page which links to all discussions that relate to that particular species. Anyone can sign up and begin a discussion and all content is licensed under creative commons. It currently has 48,000 members who have already contributed towards the 634,000 images on the website.

The ultimate goal is to ‘make high-quality, well-organized information available on an unprecedented level.’

Socrata – A variety of data sets with a social vibe.

Timetric – Timetric aggregates statistics from the the world’s leading sources of economic data

Google Public Data Explorer – Anyone can upload data to Google’s newly launched search service.

Infochimps – Find every dataset in the world. Upload datasets and point to others across the internet. Datasets are easily browsed and the metadata is contributed by users.

Datamarket – UN, World Bank, Eurostat, Gapminder and others all contribute to 13,000 dataests (both paid and free).

Data.Gov – American government data

Data.Gov.Uk – UK government data

Get The Data – A forum of data geeks helping you with your data queries.

This video shows how researchers at Stanford University mapped letters of famous figures in the 18th century – visualising their spheres of influence. This visualisation can show connections that would otherwise take ages to study.


With Facebook announcing a major new messaging system (codenamed Project Titan) and Google snapping up acquisitions all over the place (83 so far) – what is the fundamental difference between these two giants of cyberspace?

The answer lies in the kind of data that they both deal in.

Facebook lets you tell the world all about you – what you ‘like’ about culture, companies and people. It is data that you want to give away so that you can show other people just how much of an individual you are.

Google, on the other hand, is a lot more personal than that. It is about what you really get up to when it is just you and the computer. It stores data about everything from your embarrassing rash to your sexual desires.

As Sebastian Anthony puts it:

‘Facebook knows who we want to be, while Google knows who we actually are.’

We could see this as Facebook being all about your public self, whilst Google is all about your private self.

Of course, the bottom line for the companies involved is all about how this fundamental difference affects revenue. Facebook advertises to your public self, and Google advertises to your private self.

The question now is whether Project Titan will change this fundamental difference by reading your Facebook emails and targeting adverts (something which Google already do).

This would be an advertising model based on both your private and public identities. Priceless to marketers, but something that I find unsettling.

Found this gem of a list.

So now I know to say that I was tempted by a delectation of nymphs, surrounded by a nervousness of AI’s and beset upon by a torment of ogres.

We have been living in the Age of Smart for a little while now.

Thanks to the iPhone, and more recently the Android handsets, smartphones have exploded into our lives. And now, for most of us, our mobile phone has become an essential tool for much more than just calling others.

Smartphones are now used extensively for social networking, media consumption, navigation, checking the weather, receiving restaurant recommendations, playing games, watching programmes, listening to music and, of course, calling people.

Add to this the advent of augmented reality (a view of the world augmented by virtual computer-generated imagery) and it is clear that the Age of Smart is seeping into all areas of our lives.

In 2012 it is estimated that smartphones will sell more units than both laptop and desktop PCs put together. Wireless data will get faster and cheaper to access through our handset. And by 2013 27% of all mobile phones will be smartphones (Morgan Stanley). Furthermore, Google’s mobile search traffic grew 50 percent in first half of 2010 and the average smartphone user will download 5 applications a month.

This rapid adoption is causing accelerated innovation which is reinforcing this growth.

Eric Schmidt, CEO of Google, has stated that our interaction with these devices will become more fluid and natural – they will “just work”. They will grow and begin spreading even further, for example the Android operating system will be able to act as a remote control device for Google TV, understanding verbal commands perfectly. Even more exciting is the concept of a universal translator which understands your voice and translates it into a chosen language.

He goes further, stating rather creepily in an interview with The Wall Street Journal:

“Let’s say you’re walking down the street. Because of the info Google has collected about you, “we know roughly who you are, roughly what you care about, roughly who your friends are.”

Google also knows, to within a foot, where you are. Mr. Schmidt leaves it to a listener to imagine the possibilities: If you need milk and there’s a place nearby to get milk, Google will remind you to get milk.’

Obviously, it is not just Google steaming ahead with smartphone innovation. A recent picture was spread around the web showing how the new iPod Nano could look and be worn like a watch.

It is clear that smartphones will get smaller and more powerful until we can fit them comfortably anyway on our body (and maybe inside our body).

Undoubtedly this new Age of Smart will bring with it renewed discussions around privacy and security.

With our phones constantly receiving and sending information it will become easier to keep us under surveillance and expose us to abuses of power.

We will also become more susceptible to the techniques of marketing, an industry that keeps an eagle eye on technology for the next possible sales opportunity.

The latest topic in marketing is the possibilities of push notifications – the process of popping messages up onto your mobile after you opt in to receive them.

Smartphones are also a perfect platform to target and engage with the young, as they are the ones who are spedning the most amount of money and time on their smart mobile devices.

So in this new age of Smart we will become more distracted and convinced to spend online, our tiniest movements will be monitored and recorded whilst privacy is discussed more hotly than ever before.

And when our smartphones begin to do many of the tasks we do ourselves, remembering everything for us and recommending where we go and what we do – we need to ask if we are going to get less smart as our phones get smarter.

Whichever way it goes – for the first time technology will truly leave the box and spread out into our environment. And this might happen at a rate unparalleled in human existence, truly heralding the arrival of the virtual age.

Following on from my last post that looked into recent developments in crowdsourcing – here are some more reports and examples of crowdsourcing.

Crowdsourced Journalism – Imagining how the media will be reborn in 2022, trend watcher and futurist Ross Dawson has predicted that news investigation and reporting will be done by ‘hordes of amateurs overseen by professionals’.

Organising Aid Relief – Techcrunch have investigated case studies in crowdsourcing disaster relief. By ‘placing reporting power in the hands of people who might otherwise be victims’ technology is allowing armchair disaster relief experts to work with people on the ground to help build important communication channels.

In Haiti, mass collaboration enabled to provide a better map of the disaster zone than the US department of defense (and at a fraction of the cost). In any area where modern communication channels exist, there is the possibility for this form of organisation.

New Opensource Software – All Our Ideas is a new free platform that has been designed to harness the power of crowdsourcing. It cleverly combines a structured questionnaire with the added ability for a respondent to suggest new ideas. As crowdsourcing in the digital age is all about combining the benefits of quantitative research with the benefits of qualitative research, I reckon this software will be the first example of many new platforms to come.

Crowdsourcing the Truth – Truth Squad was an experiment that tested whether crowdsourcing can be used to fact-check claims made in the media and by politicians. After a discussion and voting by contributors to a website, the quote or claim in question is assessed by a panel of judges who then weigh up the evidence and make a verdict. Despite low publicity the experiment was a success and showed the crowds eagerness to put claims to the test. I expect (and hope) this to become a major new movement within crowdsourcing.

Collecting Sounds – Asking “What does Britain sound like?” the British Library is using crowdsourcing to collect and catalogue sound samples from around the country. Working with social sound recording service Audioboo, the library is asking people to record the sound of traffic, town centres or anything else around where they live. It is anticipated that over 10,000 samples will be created to generate a sound map of Britain. More details here.

Over halfway through a year which has seen the biggest rise in crowdsourcing solutions, it seems that the trend is set to continue. What seems to be the most important observation to make is the increasing use of methods to curate, filter or control the input from the crowd.

As if declining newspaper sales hadn’t made journalists fear for their jobs enough – along comes the robot reporter!

I’m not talking about C3PO with a notepad – but rather the ability computers have to analyse data and write reports automatically.

Narrative Science is a company that runs software which recieves date, analyses it and writes a piece of news copy – all without a human hand in sight.

So far the software has been used to successfully report on sports games which, due to financial constraints, couldn’t normally be reported on. But the possibilities of this new form of reporting are enormous.

This technology could potentially be applied to any piece of reporting that requires analysis of data. And with the amount of data around us piling up at a ridiculous rate, there is definitely a need for something to help make sense of it all.

Of course, I seriously doubt that a computer would be able to dig out the finer details that a trained reporting human eye would uncover. But equally as unlikely is the ability for humans to find details and patterns in data which a computer easily can.

The example given by one of the partners of Narrative Science is proof enough: “One machine-generated game story suggested that the pitcher’s excellent performance during that game indicated that he might be coming out of a slump.”

So – with governments around the world opening up their data, alongside a growing need to monitor the actions of bankers and a web that keeps the world constantly producing data, this new method of reporting could really catch on.

And how will this effect journalism? I doubt it will lead to news companies sacking their staff and hiring companies like Narrative Science.

I don’t believe that automating news reporting will diminish the role of human reporters – rather it will help reporters get closer to the story hidden within  the data. And save a lot of time.

Everyone seems to be getting more and more interested in data. Like computer gaming – data has left nerd territory and entered into popular culture. It is even (dare I say it) cool.

It is not surprising. Technological innovation changes our attitudes, opinions and beliefs in ways we couldn’t have expected.  The internet has accelerated the creation, spread and recording of data on a scale never before imagined.

First there are the data sources – today marks the release of COINS (Combined Online Information System). It is a massive 120GB of data detailing public spending – essentially showing how the government works. Data geeks around the world will begin to work out ways to sort out and organise this data into something that can be used.

There is already a vast database of government data available at Other data can be found at the click of a mouse – check out – a searchable list of available data sets from social network usage to betting odds. The Guardian runs a datablog which posts the latest and topical datasets.

It is not just records that are available – public opinion is instantaneous. I work for a company that has pioneered the first smart phone online survey software which already has thousands of users. We are able to get thousands of responses to any news story in literally minutes. Consumer attitude can be monitored and a change immediately exploited.

Secondly – there are the mash ups. Imagine the mash up music of 2 many DJ’s (mixing two or more songs together to create something new and unique) and apply that to data. Applications like Google Doc’s are allowing people to easily combine different data sets. So if you want to combine a Google map with a list of BNP members – voila! Or if you want to mash together the RSS feeds of your favourite websites and filter for things you care about – then a programme like Yahoo Pipes makes that possible.

Finally, the rise of the info-graphic. An info-graphic is a visual representation of information. The recent advent of online social-networking has bred a population of users eager to see how people are using sites like Twitter and Facebook – like this current state of Twitter info-graphic. Info-graphic’s have surged in popularity and are everywhere – from the trustworthiness of beards to the cost of the BBC. Some of my favourite sites that provide a regular stream of quality info-graphics are:

There are websites that are allowing you to create your own visualisations, such as ManyEyes (which includes this brilliant chart of movie genres over time).  Tableau has just released a free version of its graph software, Tableau Public (which i’m yet to play with).

So, technology has given us loads of tools to make sense of this increasingly confusing and information overloaded world. The emergence and spread of tablet computers will help our love of data grow as we begin whizzing through info-graphics and interactive programmes using our fingers. It might just be me that is excited about  these changes – but i’m fairly confident that a popular appropriation of data will be the next big thing for the inhabitants of our plugged-in society


Get every new post delivered to your Inbox.

Join 25 other followers