It's also possible you have a old piece of galvanized or iron pipe somewhere that's breaking down and introducing rust.
That's probably not the case, since the previous owners had the same problem with the water, and we've replaced pretty much every part of our system, including part of the well.
The buildup of brown - whatever it is - can be frustrating, and from replacing toilet plumbing, it appears to be like dirt. But the crap that builds up in the tub and faucets is murder to scrub off.
I suspect our old softener has finally gone kaput - if it ever did anything at all (it was here when we bought the house), but I am not sure what we will replace it with.
Still just hoping someone will know about St Mary's county well water and say "OH! What YOU need is..." and give an idea.
I'm going to check the chlorination idea - I guess it hadn't occurred to me that municipal water is usually chlorinated before it reaches the house, but well water -- isn't.
I also know that our pool is heavily chlorinated with spigot water - and this never happens.