I was 10 when it came out -- I received it as a gift when I was 12 or so -- and that impacts how I feel about it. My favorites are mostly among the early/mid 70's albums, though 1983's Tougher Than Leather probably makes a personal Willie Top 5, too.
Yea, these things are mostly generational (I've got a friend about 10 years younger and you can draw a line across Rod Stewart's discography - I like the older, he likes the newer; very little crossover). I became a Willie Nelson fan shortly after Red-Headed Stranger came out. Coupled with his previous songwriting, it was just hard for me to embrace Stardust.