I go for mana crystal into catalyst, which eventually either goes into banshee's or rod of ages. Beyond that, it really depends on their team comp; if they're heavy physical, I go for frozen heart and stack sunfires. If they go for magic, I get warmog's and force of nature.
Your main role in a teamfight is to nuke their squishy with ult, E, Q, and autoattacks, and then just tank from there. You're never going to be an excellent tank, but you can do a pretty good impression. The main problem with malphite is that, after you nuke their squishy, you still need to be relevant enough for them to not be able to ignore you, which is where sunfire comes in. Malphite is far better against physical damage than magical, so he's more of a late pick against physical teams than an all-around tank. Frozen heart combined with his E reduces enemy attack speed to a crawl, so if they're relying on physical DPS, you can easily negate most of their damage just be focusing on their DPS. Hug their trist/ashe and don't let go.