Most people have heard that computers only operate in terms of "ones and zeroes". This is what is called "binary" information. The reason they have to use ones and zeroes for everything is because computers transfer information using mediums like electricity, light, and radio waves, where usually the only useful method of communication is to turn the signal on, or turn it off. Scattered all through all those silicon chips on your motherboard are thousands (millions?) of little gates that open and close, admitting or denying access to electricity, sending a one or a zero.
However, just a one or a zero (one "bit" of information) isn't very useful by itself. That's why they are combined into larger units, most commonly eight bits at a time, which form a "byte". (If you've never heard these terms, get used to them, they will soon become about as familiar to you as television and Wonder bread.)
Binary Arithmetic
Now, if you carry that process out to a whole byte, with eight bits, then you get all the way up to a "128s column" on the far left, and down through 64s, 32s, 16s, 8s, 4s, 2s, and back to the ones column. All these columns together can add up to 256 different possible combinations.
These combinations can be used in a number of different ways. They can simply represent a number from 0 to 255. By allowing negatives, they can represent a number from -128 to +127 (or something like that.) They can be combined with another byte so that together they represent a number from 0 to 65535, or combined with more bytes to represent even bigger numbers. Or, the byte can be interpreted as a key into a table (ASCII code) which contains symbols for each letter in the English alphabet, upper case or lower case, spaces, tabs, carriage returns, plus all the punctuation symbols.
When this last option is used, we see how computers can represent plain text, which is what most people use them for most of the time. This is how your word processor documents, emails, newsgroups, and online forums are represented electronically. All of the greatest literary works ever produced can thus be reduced to strings of ones and zeros. (You can bust that out next time somebody tells you computers "only" understand zeroes and ones.)
So, one bit can only be a one or a zero. However, if you put two bits side by side, then you can represent a number from zero to three. You do this by making the second bit represent the "twos column", just as you would have a "tens column" and a "hundreds column" in traditional (base ten) school arithmetic. If the bit in the "twos column" is turned on, then you add two to whatever is in the "ones column", either a one or a zero. Therefore, if the bit on the left is on, and the bit on the right is off, the number you are looking at is two. If the bit on the right is on and the bit on the left is off, the number is one, if they are both on it is three, if neither is on the number is zero. Make sense?