Eh, I've lived here ~20 years, and by and large, I think it largely depends on where you live that determines whether it gets the "ghetto" moniker or not. I'm in one of the "old" neighborhoods (Wakefield, Sentry, Bannister, etc), and in them, things can change largely court-to-court. It seems if you live in one of the "inner" courts (those that are inside of the circle), and have a bike path running nearby or exiting into the court, then you can get more occurrences of seedier folk passing through. If you live in one of the "outside" courts, things can be entirely different. The, of course, there are the apartments, town homes, and duplex/triplexes where things gett even stranger.
As for the whole DC migration thing, maybe my area has just been lucky. Most of the african-american families moving in are very well-to-do and hold stable jobs and even had kids and whatnot. With all the new growth in the area, things are getting to be very dynamic, so it'll be interesting to see how things develop over the next few years.
We've got a new baseball stadium being built, possibly another nearby power plant, US301 and MD210 road improvements (not to mention the Nice Bridge project), high-energy physics laboratory etc..The next 5-10 years I think will make this county and town vastly different than what it is today.
Just as long as we don't turn into Loudon county, I think things will be alright