I finally found a cheap ($20) wireless router that has built-in QoS (quality of service) support, it's the Airlink AR430W. With QoS, you can set up rules based on service type (http, etc.) or by IP address, so you can say "traffic from computer X should always get priority over traffic from computer Y", or "game traffic should get priority over web traffic", etc.

I just set it up over the weekend, very straighforward, great features, good web admin tool, signal strength is better than my previous D-Link.

Fry's does special pricing on the AR430W from time to time, lowest I've seen is $20. I know people out there spend $50 or $60 on the Linksys WR54GT (or whatever) but I don't want to learn nerdy details about what patch, version, blah.. is required to enable this or disable that, and the Linksys seems to require this. I might be totally wrong, but the small amount of info I found suggested the Linksys does not offer plug-and-play QoS, whereas the Airlink does.