Inner Harbor's a #### hole and so is Fell's Point. You can't sit on any of the benches because homeless people are sleeping on them. Then at dusk the druggies rise and roam.
It's generally safe at Power Plant on a weekend night because there are so many people, but even then you have to be careful and use the multiple buddy system. It really is a shame what the politicians have done to that city.
Inner Harbor and Fells Point are not ####holes. Granted the Inner Harbor isn't as "nice" as it used to be but there is plenty to do, is maintained and offers a variety of decent restaurants and shopping (let alone places to visit).
Part of the problem may be age and I mean no offense. But based on what I know of you (very limited of course and solely from the posts on this board), I'd put you somewhere in the 40-60 range. My apologies if I've shot too high . But my point here is that any of these places are probably not for you after dark. I'm 41 and they're not for me anymore - most "busy" places aren't. But during the day, they are safe. I've taken my family to the Inner Harbor many times in the recent years.
Harbor East is "high end" now and lies directly between the Inner Harbor and Fells Point and is heavily patrolled with lots of people out at all times. Fells Point has some pretty decent day spots but is largely known for it's bar scene and a younger crowed later in the evening. Staying at the Inner Harbor until 10pm with family (even young kids) shouldn't be an issue in terms of safety. I can't imagine Fells Point has much to offer a family with young kids or even much for singles/couples my age and older later into the evening. After dark and overnight, yes, you probably do see an influx of homeless but that is well past when I would tend to be there.