What does a year in the life of a computer look like?
Well, something like the map below! This is a map every bit of of mouse movement, every mouse click and every keyboard press I have done on my home and work computer over every day of a whole year.
2013-2014 [click for a bigger view]
To make it I wrote a little python script using pyHook to grab inputs in Windows, which I compiled to an .exe using py2exe. I set this up so that it starts recording the mouse movement, clicks, and keyboard presses after I log into my home or work computer. After 2 years it had collected nearly 10 Gb of data! This was far too much to look through by hand, so I wrote a second set of scripts to plot it to an image.
So what does it all mean? Well the map breaks down a bit like a normal calendar, with days of the week running from the top to the bottom of the map, and successive weeks running from left to right. The years and months are marked at the top of the map.
Within each day my computer activity is broken down by time. Time runs from the top to the bottom of each day, from midnight to midnight. Coloured speckles on the dark background indicate computer activity. It is easy to see that I use computers a lot, with a chunk of time from around midnight to 7 am when I am normally asleep, then smatterings of activity from around 8 am to midnight when I am at work or awake at home.
Different types of computer activity are shown in different colours.
The structure within each of the colours also contains information; distance in the horizontal direction corresponds to horizontal mouse position across my two screens (for mouse movement) which mouse button was clicked (for mouse clicks) and which key was pressed (for keyboard presses).
2012-2013 [click for a bigger view]
In these maps of usage some interesting structures jump out; you can spot the type of work I was doing with my computer based on the type of mouse and keyboard activity:
This is usage on a day where I was writing my PhD thesis. The keyboard (cyan) has loads of activity, while the mouse (magenta) did relatively little.
This is a day where I was mainly using Blender for 3D graphics. The mouse (magenta) has huge levels of activity, centred on just the left hand screen). The keyboard is hardly active except for the control and shift keys, which light up as a single column of bright cyan pixels.
It is quite scary how much information can be gleaned from these maps of computer activity. Without knowing which programs were open or which keyboard keys were being pressed it is still easy to work out where I have been, when I have been working, and the kind of things I was doing on my computer. Similar data can be collected remotely; particularly if an internet company tracks when and where you use the internet.
Stop for a second and think about the companies you interact with, and the data mining they can do. Think how much they can learn about you and your habits; Google and the websites you visit, your phone company and when and who you text and call, the supermarket you shop in and what you buy. These companies can work out what you are interested in, what you like and dislike, when you are awake and when you are asleep. This is big data, and it is valuable and it is powerful. Big data is how Target knew a man's teenage daughter was pregnant before he did!
ImageJ: Data plotting.
Inkscape: Plot annotation.