Yes, Ezreal has a steep learning curve. No, most people do not realize this. People rarely distinguish between an "easy" champion and a "safe" champion. His safety and map presence are what keep him in the ADC Trinity (granted, his early-mid game damage is great with little itemization if built correctly).
Many players understand how to use him, but they do not understand how to use him effectively. They have a rough idea of his safety and map presence, given his kit. But really understanding how to use him goes well beyond that.
Ezreal thrives off of a lot of poke, and his strength shines mid-game because there are so many situations you can address in his itemization. I'll give you two main suggestions with your build order...
BT is nice if you can save up for it and get it early (assuming you are fed). Other build routes you can consider are getting a Sheen early. Helps with mana and your poking power. Ezreal is one of the poker grandmasters of this game.
I'd say drop the PD. No need for it really, unless you want the movement speed. Ezreal already has an innate AS steroid that is easy to build up thanks to the low mana cost and low CD on his Q. And if you're in a team fight, just hit a quick well-aimed W to get your stacks up (or a well-aimed ult?).
If you can safely get BT first, consider Sheen next cause you'll be hitting like a truck relatively quickly. It's all situational though. Remember that.
EDIT: And if you're completing a tri-force on Ez, instead of the Iceborn gauntlet (using the Sheen), you should consider building Zeal as your last completion item, since Phage is much more beneficial.