Good inexpensive place to stay? Don't go near Tottenham or the surrounding area's then. Try Muswell Hill. It has hotels and is quite a nice area (look up hotels on google).
Places to hang out with Spurs fans are any pub around the Ground on matchday. MMMmmmmmm, beer.
Best seats (for atmosphere) are in the South stand where all the hard-core Spurs fans are. They are the loudest and have the most offensive/funny chants. Best view would be from the middle of the West stand, but you'll pay for it (really pay!). Avoid the East stand as it has 2 dirty great pillars that might obscure your view. (A friend of mine once bought seats that obscured both goals...D'oh!)
So, stay in Muswell Hill (the W3 bus from Alexander Palace (AKA Ally Pally ) which is just down the road will take you direct to White Hart Lane), and buy South Stand tickets (I'd go for Upper Tier, it's a better view and you can still hear the chants).
Hope you have fun!