"The pub has got lots of beer"
"The pub has not got any beer"

These two messages have the same amount of characters taken from the same character set. To transmit either of them between the pub and your home takes the same amount of time and energy no matter how the pub landlord sends them, for example by running flags up a flagpole or banging a big drum, or making a phone call. (Or, if you are incredibly rich, installing a satellite link.)

This messaging system is pretty robust and the message remains intelligible even if the signal to noise ratio worsens, so if you get the message: "The pub h*s got lo%@s of bear", you still can guess what the original message meant.

In fact, because you have chosen a robust coding system, called the English language, to encode the message, the noise level would have to get pretty high for it to become unintelligible.

After days of receiving these messages, you both make a discovery that was staring you in the face all along, - you can code your messages more effectively! You decide to send the message "1" if the pub has beer, and "0" if it hasn't any beer. The messages take much less time and energy to send now, hooray for the new binary digital coding scheme!

But what happens when the signal to noise ratio gets worse, if the weather gets very foggy or thundery, or other people nearby start raising flags or banging drums too?

When you receive the message "1" now, what does it mean?

Well it all depends on how bad the signal to noise ratio was. If the noise level was very low you can be pretty sure that a when you received a "1" then a "1" was transmitted, but as the noise level gets worse you become less and less sure.

What you have done is traded off the ability to transfer information in the presence of noise for economy of transmission. After some thought, you decide to add some extra data to your message and send "111" if the pub has beer, and "000" if it hasn't. Now when you receive the message "111" you can be very sure that the pub has beer, and still be pretty certain even if you receive "101" instead.

Congratulations! You have just invented forward error correction.

Thinking about it further you realise that you can buy more immunity to noise by adding more redundant information to your message, so that the message "111111111111111111111111111111111111111" could survive an even worse signal to noise ratio.

The cost effectiveness of any messaging system you decide to adopt will be governed by:
· The coding scheme and the amount of error checking/correcting data you incorporate in your message.
· The speed of transmission you require (in our case one message an hour - not very fast).
· How many errors you are willing to tolerate in the received data - that is, how frequently you are willing to go to the pub and find that they've got no beer.
· The signal to noise ratio in your transmission channel.

But is this all we need to know?

More days go by, and the landlord has been thinking about the messages that he has been sending. He's been getting pretty fed up with running out the door and hauling flags up a flagpole, or banging an enormous drum in the car park, and is wondering if there might be a better way of doing things.

He finally realises something else: almost every message he has ever sent has been "The pub has got lots of beer"!

After some negotiation over a pint or two, you agree that when the pub first opens the landlord will send the message, "The pub has got lots of beer". After that he will not send you this message again, as it will not tell you anything that you could not already be pretty certain about. If disaster strikes, and the pub does run out of beer, then the landlord will send the very important message "The pub has not got any beer".

You go home that night having learned another valuable lesson, that the information content of a message is inversely related to the probability of receiving it and that consequently, some messages are almost a complete and utter waste of time and need not be sent at all.

So you now know that effectiveness of any messaging system can be measured by looking at
· The coding scheme you use.
· The data transmission rate.
· The amount of uncertainty about the received message you are prepared to tolerate.
· The signal to noise ratio in your transmission channel.
· The information content of the messages you transmit across the channel.

With a slight hangover, you start to wonder how all of this could possibly apply to digital broadcasting.

Note: To the best of my knowledge Claude Shannon rarely went down the pub, but he did spend a lot of time riding a unicycle.

Could this have been the start of an article about mpeg coding?

The Professor