Veigar has one of the weakest laning phases in the game. He essentially has to dedicate all of hi mana to Q farming, and has nothing to really return fire with. A lane Veigar is not going to bother trying to harass you, because it's not worth his time and mana.
Second, once he comes back with boots and such, he's actually capable of some pretty impressive bursting. Against an enemy AP champ, he may even be able to get a kill, but he's still reliant on his full combo which suffers from a rather weak stun at this point.
Late game, Veigar is still a single target burst mage with relatively low range, and unreliable Dark Matters. He's also insanely squishy.
Veigar has burst in place of survivability. There is absolutely nothing OP about him, and if you're losing to him, then he deserves to win more than you because HE'S BETTER THAN YOU AT THIS GAME.
I'm sorry you suck. But deal with it.