For the most part, anything that scales with attack damage or states that it does physical damage will benefit from armor penetration, as well as any auto attacks. Such skills include Pantheon's Spear Shot and Heartseeker Strike, Ashe's Volley, Caitlyn's Piltover Peacemaker, Gangplank's Parrrley, Nasus's Siphoning Strike, and a few others. The majority of damaging abilities however deal magic damage and are best used with magic penetration.
For Kog'Maw, I believe his Bio-Arcane Barrage does magic damage. For a short while it was actually doing physical damage; I assume they reverted it back, but I could be wrong. If you wish to test, you could just get a friend to 1v1 you with teleport. Take the skill at level 1 and have them run mid with no items. Activate it and fire off a shot, and check the damage. Have them go back and buy some magic resist, and see if the damage done is altered.
Also, these penetration runes apply to items, too. If you have a Bloodrazor, you can improve the damage done by using magic penetration runes, as the 4% per hit it deals is done in magic damage.