I think you may have selective memory. I have never, ever, been "randomly selected" by TSA and neither have any of my family (parents/in-laws) or co-workers with whom I travel with quite regularly (10-15 trips per year total).
The only exception is two engineers of Indian descent in my group. It seems like a 50/50 chance that one or both will get additional screening and I know they take the utmost care to prevent it (don't bring chargers/laptops/etc. in carry-on, are professionally dressed and clean shaven, etc).
I was once selected for additional screening along with the guy that was behind me. I am white and the guy behind me, I am guessing, was from middle eastern decent. I always thought I just got caught up as the person the TSA guys could say "we screened a white one too" if the other guy complained. He actually was finished before me because he had less stuff in his carryon but it didn't take me long either. It was basically a minor inconvenience that really wasn't that big of a deal to me.