Yeah, started playing Ezreal.
My early game is pretty awesome, provided my enemy isn't smart and stands behind minions. Mid game is pretty generally.
But endgame my damage just drops rapidly.
My build is Boots->SotC->Brutalizer->Sheen->Bloodthirsters->Bloodthirster. After that I just upgrade Brut into Ghostblade and Sheen into Trinity force.

I just feel like I like I'm doing something wrong. Can anyone give tips/builds?

Black Cleaver helps tons. With your passive aspd bonus, you can chip away at their armor very quickly, and you'll do more damage than a fully stacked bloodthirster. Oh and you won't have to worry about losing stacks if you die.

So, replace Bloodthirster with Black Cleaver.
I tend to spam my Q most of the time instead of auto-attacking to stay out of harms way.
Should I do this, or jump into the fray?

That's ezreal for ye, his lategame sucks. Midgame is meh, early game he rocks. If you use that to your advantage and get fed, your mid and lategame are helluva lot better

Then you destroy those minions fast and show him nothing will keep you from mystic shotting his ass.

People's reactions are sometimes delayed, so with characters that have a skill shot easily blocked by minions will feel safe. If the enemy isn't moving behind the minion that is at very low health and you're confident of your aim and timing, then the second you throw your attack that will kill the minion pop your mystic shot/whatever. Mystic shot travels faster than your normal attack so it's bound to hit more often than moving around trying to get a better aim and making them adjust to your moving as well.