not necesary in order
1.Mercury Threads ( alacrity or homeguard)
2.Rod of the Ages
3.Rilaays Cristal Scepter
4.Abyssal Scepter
next two items can't decide between Banshee's Veil, Randuin's Omen, Frozen Heart, Runic Bulwark( if support doesn't build it), Sunfire cape. What do you think?

I like sunfire cape for armor; but what you're really missing is CDR.

I think Spirit Visage is a good option; Warmog's and Randuin's are rarely bad choices for tankiness. Void Staff, Death cap or Liandry's Torment for damage depending on needs.

I'd consider swapping out Merc Treads for Lucidity boots if you go Spirit Visage, b/c between that and Abyssal, you really don't need the MR.

I also don't like Rylai's on Maokai. His kit really can't take great advantage of the slow, and there are better ways to get HP + AP (a NLR + Giant's Belt will put you 100 health behind and 300 gold ahead, but there are very good things you can do with those items).

I played Tanky AP Mao a lot in S2 and I don't think it's an optimal pick. While he does bring a relatively unique combination of strong initiation, protection and AoE poke, he doesn't do well in lane against a lot of 1v1 match ups, and is generally weak without a large gold advantage; you're either not damaging or not tanky enough to do what you need to do.

I'm not saying don't play Mao that way, but I would advise you to consider what it is you like about that build and trying out alternatives that might fill that particular roll (Brand, Ziggs, Zyra for AoE poke/inititate; Alistar, Tarric, CDR-tank Mao for team protection). You may find them more rewarding.