On their actual functionality, the only real difference they have is how they interact with healing effects, and how many damage types ignore their EHP (effective hit points) bonuses. Resistances more or less apply a %healing effects bonus just by having them. Restoring 10 health when you have 50% damage reduction effectively restores 20 EHP, while restoring 10 health when you just stack health and have 0% damage reduction restores 10 EHP (and that applies to everything that restores health, including increasing your maximum health). Resistances have the inverse effect on life-steal, as life steal is based on actual damage dealt. If an opponent strikes you with 100 damage at 10% life steal, they will restore 10 health with each strike if you have 2000 health and 0 resistances (for 2000 EHP), but only recover 5 health with each strike if you have 1000 health and 50% damage reduction (for 2000 EHP). I should also add Thornmail reflects damage before it's mitigated by your resistances, which can make it particularly effective at shutting down life steal with this functionality.
Health tends to have the advantage (in basically any game) that more health will always increase your EHP score against every damage type. With armor you generally have armor penetration, or damage types (such as magical, true, and abilities that cost health) that ignore armor entirely. For magic resistance there is physical damage, true damage, and ability health costs. Which makes 50% physical damage reduction less useful on average than +100% HP (at least in relatively short fights were healing is not going to happen). And while certain champions do have %max health damage capabilities, such champions apply plenty of flat damage in their rotation or on the attacks it's applied with to make health always useful even against them.
For the game costs health is much more effective as the first pick up in LoL now. Health tends to be the most turbulent attribute gained from leveling up, were your average champion has 500 health at level one and quadruples that to around 2000 upon reaching level 18. As such health tends to be balanced around levels 9-10, so any HP bonus grabbed at level 1 will be roughly twice as effective as it's costs otherwise indicate.
Considering the average champion has somewhere around 500 health, 18 armor, and 30 magic resistance at level 1; to double your EHP score with health or resistances you need:
500 health, or (~1250 gold) worth of health.
With resistances, you need 136 armor (~2448 gold) and 160 magic resistance (~2880 gold) for a total of (~5328 gold).
On the average level 18 (melee) champion with 2000 health, 77 armor, and 51 magic resistance: to double your EHP you'd need:
2000 health, or (5000~gold) worth of health
254 armor (~4572 gold) and 202 magic resistance (~3636 gold) for a total of (8208 gold).
It's worth noting that health and resistances stack multiplicativly with each other while they stack linearly with themselves though - so even though resistances are less cost-effective in the short run you can eventually reach a point were stacking health ceases to be as cost effective as picking up some level of resistances. Which I'm fairly certain works out to were every 100 gold you spend on EHP ~73% should be on health and ~27% should be on resistances concerning average level 18 values.
You should certainly get health first, basically.