In a somewhat recent post someone said the formula to figure out damage reduction was:
75 / (75 + 100) = 43% reduction
Let's chop his armor off by 17 (just red runes):
58 / (58 + 100) = 37% reduction
Let's chop his armor off by 29 (red and quints):
46 / (46 + 100) = 32% reduction
Let's factor in Brutalizer for each value because you fail if you don't have brutalizer on most AD heroes. Going to skip writing the formula every time.
Nothing: 38% reduction
Reds: 30% reduction
Both: 24% reduction
Repeat the same thing with last whisper because a number of carry-type AD heroes will have last whisper. This is brutalizer and last whisper combined.
Nothing: 26% reduction
Reds: 21% reduction
Both: 16% reduction
The curve is not difficult to understand with different armor values. The less armor you have, the more effective armor penetration is. The more armor you have, the less effective it is. This is stuff that's well known already.
I think we can safely say that critical chance runes completely beat armor penetration if the above is all true because just reds and quints alone with crit will give you 14%.
Btw I think you can use 45 or so armor for level 10ish heroes in general (aka. no one who is stacking armor).
Quick version with brutalizer:
Nothing: 23% reduction
Reds: 12% reduction
Both: Basically no reduction because they have pretty much 0 armor.
So with red runes you gain 11% damage, and 23% damage with both runes. This looks good for armor penetrate but it's kind of trap. On the heroes you will buy last whisper on you should have both items at level 10.
45-15 = 30
Oops...your armor penetrate runes took their armor to 0, that means you gain absolutely nothing from the 40% armor penetrate on last whisper.