Rank 1 of Spirit Fire does 110 damage to everything that stands in it. This increases by 80 for every additional rank. Damage is not the problem with Spirit Fire.
It is true that enemy champions won't generally stand in it for the full duration, but the same is also true of similar abilities like Tormented Soil, Miasma, etc. Unlike those abilities, Spirit Fire will do at least half it's damage (55+40/rank, a little low base but actually still competitive growth with other AoEs 66+48 though with the extra tick), instead of the 1-2 fifths of damage that Tormented Soil might get (50+30/rank for 2 hits but also has MR shred), or the 2-3 ninths that Miasma might get (but Miasma kicks butt, with 75+30/rank at 3 hits).
It does of course lack range compared to those abilities, but also covers a larger area, and has equal or better AP ratios. Generally, the main reason that people don't max E first is because of the importance of farming with Q, not because Spirit Fire itself is in any way bad.
That all said, the fact it doesn't jell with his laning makes it feel out of place... but I don't think making it a single-target nuke would make it any better for that. Probably the only way to make his E attractive to people would be to allow it to generate (a reduced amount of) damage for Siphoning Strike, say 1-2 per kill.
The problem with that is that it might be too much of a buff to his early game considering his late game potential, even with the loss of 2-4 Q damage per kill.