If you just want to do a day trip, Rock Creek Park in DC does guided trail rides. Here's the link...
Rock Creek Horse Center
If you're looking to enroll them in riding lessons, there are a lot of places. Depends on where you live and how far you want to drive