You aren't screwing yourself out of 5 IP, you're doing the right thing
I agree though, leaving anonymous notes would be great, there are a few that are in such a grey area that it would be nice to see other's reasoning on their choice.
I've happily pardoned 2 today, because although one died a lot, they certainly seemed like they were trying and just sucked in general, which is not fair to punish them for.
I punish people for offensive language if and only if it occurs often. If they are offensive to their own team for now reason, not only does it ruin the game for their team members, but it also breaks down communication which greatly increases the chance of a loss, which ruins the game more for their team. Harassing the enemy team, its not hard to mute them.