I've played against numerous AP Yis and I also play him myself. His laning is safe but he's unable to get early game kills because his Q will bring the enemy very low but the chances of landing a Q that kills them is pretty low as they'll probably retreat and B. Also, as has been stated, AP Yi has more cleanup potential than carrying potential. Even with 5 points in your Q, pretty much all you can do in teamfights is use it to take out some chunks of the enemy team's HP, then sit back for 10 seconds as the rest of your team goes in to fight, then use it again as cleanup. Also, hard CC shuts him down, not even his ultimate can get him out of that.
Also, AP Yi depends a lot on his team to do well because of his lack of carrying potential. Like OP said, AP Yi always comes in last during teamfights to clean up. How can he clean up if the rest of his team doesn't win the fight for him first? If his team doesn't do well, even if he wins his lane, in the end he'll lose the game. You should be more afraid of an AP Yi's team than of AP Yi himself.