I work with a couple of people who by their own admission, are "house poor". They make good money but the mortgage and all of the house costs keep them from really having all that much.
I also have an old friend from college whom I reconnected with on Facebook. He's a database administrator in Cleveland, and his wife is an engineer.
Basically - they downsized. They had the money - they just decided it would be better to live on more meager means, in a smaller house, for the freedom of longer, better vacations and more family time and events.
He and his three kids live in a home not much bigger than my first starter home, but they always have time and money to do whatever they want. Sometimes I wish I'd had this foresight.