The Postal Service is at best a quasi Government agency. At best.
As for the rest, I'm not sure what he means by "the basics of a civilized society."
If he means the common defense, roads, things like that, he's absolutely correct. That pesky Constitution thing again.
The Government is spending tax-payer dollars (including, presumably, their own), and therefore have a duty to be good stewards of that money; to get the best bang for the buck, for whatever they spend the money on.
The problem with the general public talking out their asses, as they are wont to do, is that they have no conception of the processes, procedures, standards, and guidelines - let's face it, miles and miles of red tape regulations - that everyone that works for the government is theoretically constrained by.
The BLUF is that EVERY GD THING has a cost to it. Want to move your desk over there? There's a cost to that. Want to buy supplies? There's a cost to that. With few exceptions, you can't just go down to Wally World and buy sh*t. Want to have a program on your computer that EVERYBODY doesn't have (such as Project)? There's a cost for that. New laptops cost money. Monitors cost money. If you can imagine it, it cost money.
Not only that, but everything costs MORE money than a similar thing at Office Depot, or where ever. Everything has to be modified to meet Government criteria. You can't just take a box of Windows 10, or 11 and stick it on people's computers. It has to be modified to suit the specific needs of the Agency. Much of it, probably most of it has to do with security. They always talk about COTS, which is Commercial Off-the-Shelf items, but for Government agencies, there is no such thing. It all has to be modified for government use.
Even the credit card buys for things like office supplies (that don't have to be modified), at least in the DoD have very specific and narrow guidelines. Remember those crappy ball point pens that we used in the military? The ones that say "U.S. Government" on the barrels? The ones that didn't work half the time? Guess what? Those are made by industry for the blind, or whatever it's called, and government agencies are REQUIRED to buy a certain amount of them before they're allowed to go to Staples and credit card pens that actually work. It's so much fun running out of pens (because people tend to hoard them) and having to use those old things.