Single Sign On, as the name implies, allows a user to authenticate against a central authority and then use that authentication to access multiple systems.
From a user point of view, the authentication process is vastly simplified. You remember *one* set of credentials, you go through *one* log-in process. Time and mental effort is saved, allowing you to be a happier user, more efficient employee, etc.
From a support / admin point of view, the time spent dealing with forgotten passwords, training users against multiple systems is also reduced. Potentially, this may reduce hardware requirements too as you've reduced duplication of functionality - however that's probably open to debate.
The obvious issue is the "all your eggs in one basket" problem - if Bob gains Alice's credentials, then he has access to all of the systems using the SSO. However, this can be mitigated by balancing out a stronger password policy (since your user's only have one to remember) and complementing the usual user/pass combination with other authentication methods. This may well be something like the two factor authentication provided by systems like CRYPTOCard. Of course, these systems are often quite expensive and require a large investment but that needs to be balanced against level of security required.
There are a number of standards out there for SSO with a variety of implementations. The first bullets are the points Oliver made and then I (or some of the other keen note-takers) may add additional discussion.