I have an idea to make spectator mode even better, but I'm afraid that it will be lost with the posts of this thread...
Anyways.. Allow us to view whatever game our friends our watching!
Here's a scenario:
Your friend calls you "Dude you gotta check out this game I'm spectating!" but you can't because the person he is spectating isn't on your friends list. Therefore, the only way to watch the game is to go over to your friends house or have him stream it on some live feed websites.
Also another thing I just thought of, show how many people are currently spectating a game for the players playing or at the very least, for those also spectating. Maybe allow us to talk to each other?