Not necessarily "Urban" exploring but...
Members of the informal railway enthusiast group that I'm a member of have hiked some local sites of interest in the last couple of years:
1. Brandywine-hiked the wye where the former Washington, Brandywine & Point Lookout Railroad connected with the Pennsylvania Railroad. The concrete footers and stand pipe for the old water tower are still visible.
2. The soon-to-be Indian Head Rail Trail. The portion from Mattingly Avenue in Indian Head to Bumpy Oak Road is especially scenic once you get out of the town of Indian Head and start paralleling the Mattawoman Creek.
3. The original portion of the Pope's Creek line that ran to Pope's Creek until sometime in the late 1960's/early 1970's. Starting at Pope's Creek, we followed the old roadbed northeast to Route 301. A big surprise was a large trestle about 2 miles from our starting point. Also of interest at Pope's Creek is the original SMECO generating plant (in operation from 1938-1953), the shell of the building still stands. The adjacent pier at one time could support railroad cars. A number of ships were scrapped at Pope's Creek after WWII with the scrap steel being loaded into freight cars and taken to the Sparrow's Point steel plant. You can still find some rails near the generating station and cross ties and a couple of mile posts can be found along the old right-of-way.
The pier at Pope's Creek
Looking away from the river, the old SMECO plant in the background
The old railroad station at Pope's Creek
(All photos courtesy of Charlie Kilbourne, Upper Marlboro)
It's best to make these hikes in the early spring before the vegetation starts to reclaim lineside artifacts and snakes and insects begin acclimating to the warmer weather.
EDIT: I almost forgot to mention that the Chesapeake Beach Railway Museum also holds a yearly hike along selected portions of the old Chesapeake Beach Railway right-of-way. That should be coming up in March or April. You might try their
website for further information.