Skip to content

Information Management and the Theory of Everything

February 18, 2011

Information Theory was fundamental to NTL's work on digital compression in the early 1990s and also key to me being able to design and produce this industry guide using the early versions of Photoshop and Quark

The first time I became aware of the work of Claude E. Shannon and the landmark paper on Information Theory he published while working at Bell Labs in the 1940s was when I worked for NTL’s Advanced Products Division and had to try to understand the principles of digital video compression to promote the company’s innovations in digital broadcast technologies.

Shannon’s Information Theory was absolutely fundamental to the encoding, transmission and decoding processes used to make digital broadcasting a reality. It also became clearer to me at that time that Information Theory sat at the heart of everything I was involved in following the transition from analogue content creation and publishing processes to digital processes, that had begun, for me, with the desktop publishing revolution in the late 80s/early 90s and continued with the arrival and growth of the web.

Information Theory however is concerned with the mechanics of communication and the quantity and readability of the information transmitted. It is not concerned with the quality of that information, its meaning or its importance. For those processes we have what has become known as Information Management – as defined here by AIIM – a practice that has been going on in shifting forms for many decades now.

Other posts on this blog highlight my dislike of unnecessary acronyms and reinvention of basic principles that are designed to create the perception that something is new and revolutionary. From the moment Information Theory was applied to compressing, storing and communicating information, there was a need to manage the quality, meaning and importance of that information.

The volume of that information, the directions in which it flows and the ways in which it is accessed have changed continually since Information Theory was first defined but that doesn’t mean that the basic requirements for managing that information have to be re-invented continually or given new descriptions. Right now, the explosion of social media driven information means the ‘signal-to-noise’ ratio, as defined by Shannon, is low. Again, this situation was foreseen back in the 1940s when Vannevar Bush published his article ‘As We May Think’ which predicted the arrival of the Internet and the web in his concept for memex, a collective memory machine. When evaluating the downsides of such a concept, Bush raised the concern that…

“truly significant attainments become lost in the mass of the inconsequential.”

Some commentators reckon 90% of web information is ‘inconsequential’. I’m inclined to be a bit more generous and apply the classic 80/20 split to estimate that 80% of this information is ‘inconsequential’. However, a friend sent me a link this morning that highlights the growth of a practice that has been known for a while and one that underlies some of my skepticism of social media and why things are not always as they might seem on the face of it. This practice takes information beyond the level of ‘inconsequential’ to being dangerously misleading.

So, a logical conclusion from this therefore is that the practice of Information Management and the application of ‘context’ and ‘conversation’ (which has happened since humans first communicated) to determine the quality, meaning and importance of that information is more important than ever right now in the evolution of human communications. Likewise, if  information seekers are to be more ‘engaged’ in this process, then it follows that the information exchanged must be of ‘significant’ value and not – by Vannevar Bush’s definition – ‘inconsequential’. As far as I can see, the only way to ensure information is of ‘significant’ value is through the application of long-established Information Management practices – anything less than this would be a superficial veneer and a recipe for creating even greater volumes of ‘inconsequential’ information that simply wastes time and effort.

As we progress through the 2nd decade of 21st century, Information Theory is no longer confined to the realms of broadcasting and communications but acknowledged to be central to our understanding of genetics and believed, by an increasing number of scientists, to be central to the holy grail of scientific knowledge – the Theory of Everything.

In a fascinating article in this month’s Knowledge Magazine, which starts by recounting the amazing story of how Information Theory was used by NASA scientists last year to fix the Voyager 2 spacecraft that is now a staggering 17 billion kilometres from Earth and ends by describing  the Holographic Principle, something that is being used to explore the laws of the cosmos – the depth and breadth of Information Theory and the answers it continues to deliver is enlightening and exciting.

It also begs the question for me that if the world’s eminent scientists are returning increasingly to ‘first principles’ in their bid to answer the mysteries of life and the universe, perhaps we should return to the first principles of Information Management in our bid to prevent ‘significant’ information from being lost in the mass of the ‘inconsequential’.

Photoshop montage produced by John Goode ( for NTL's digital broadcasting promotions circa 1995


3 Comments leave one →
  1. March 2, 2011 8:24 am

    You make some great points in this post James. And hey, I’d forgotten all about that image!

    The problem though in discussing signal : noise ratios in the context of many-to-many communications is there’s no single version of the truth. One man’s noise is another man’s signal. Hence social listening/monitoring alone in the field of normal marketing activity isn’t altogether useful, or as you’ve put it, an unnecessary and unwelcome distraction. However, dialogue with audiences via social media does work for many organisations. Tell an organisation like Sony Entertainment they can no longer use social media to promote their latest releases and they might be less than pleased!

    The web doesn’t have filters, that’s it’s strength. We the users pick up the tab though, we have to individually decide what’s good and worthy and what’s noise (to us) and allow others the same freedom.

  2. March 2, 2011 2:00 pm

    Thanks John – that image still looks good today – 15 years on 🙂

    I agree that dialog with audiences via social media does work and – as per previous points – I would say that such dialog was happenning via early ‘social media’ channels as far back as when you produced that image and we first started talking about NTL’s web challenges.

    Unfortunately, I think that it’s not just the users who have to pick up the tab for an unfiltered environment, it is ultimately society as a whole that pays the price, through the significant being drowned by the inconsequential and downright dangerous – the fraud, the hate speech, the violence and the pornography.

    My latest post expands on previous comments that this is not about the technology itself (as I maintain that much of what we have today has existed in basic forms for many years) but how we use it or abuse it that is the real and growing issue…

  3. March 2, 2011 9:26 pm

    I couldn’t agree more on the *downright dangerous* categories you list. Very undesirable.

    To qualify a point I made above I do think listening to social media back channels is useful to marketers but needs to be coupled with a strategy and technology that enables dialogue.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: