Veigar generally has a very passive early game.
As Veigar the only thing I do pretty much up until level 8 is farm my Q.
You need to allot every drop of his mana towards Q farm or else you will not be hitting as hard later.
However, Veigar gets to both harass and farm if his jungle is smart enough and independent after first blue. A few good junglers that I've seen give me blue after a while to give me lane advantage. You don't need your W to farm, and you absolutely need your Q to farm, so it works out nicely. Use your W to harass your opponent and use your Q to farm. Your W doesn't even have to hit; you just use it because you have the extra mana to force your opponent to stay on his toes.
Once mid-game starts, you should still farm you Q as veigar whenever possible, but it's more important to score insta-kills mid-late-game, because those give you 5 AP for every kill, and it can stack up fast.
BTW, if you play AP Ez I will destroy you. Better to go AD.