I grew up there (elementary - high school, etc.) As a kid, it wasn't bad at all. We had a nice place in a nice neighborhood with good neighbors (Pinefield). Moved out around 91 (in high school) to a place a little further south (listed as Waldorf but not really). Anyhow, it was a decent place for the most part and I wouldn't think I'd say it started going downhill until the late 80's. Schools were decent, safe, etc. The mall really changed things for the worse as did all the pop up communities surrounding it. I vividly remember Mattawoman-Beantown Road being 1 lane in each direction. I remember the area at the far north end of Waldorf before it was all developed. It was a good place. The mall became an issue in the mid/late 90's where people would drop their hoodrat kids off and use it as a pseudo-babysitter.
The only place we frequent in Waldorf now-a-days (food wise) is a little restaurant on 925 (parallel to 301) called Silver Skewers. That place is fantastic. The owner is such a nice, hardworking lady. Each time I'm in there, I tell her she'd stand to do far better if she moved away. Prince Frederick, Huntington, Edgewater, etc. The clientele is there to support her. Not so much in Waldorf.
Yes, we did go there and it was not stellar. Another one for the 'never to return to' list.