I wonder that too! If I remember correctly, it did well when Perkins was there, but then all the Perkins in this area were shut down. After Perkins, it was just one fail after another. Most of them started out really well too. Very strange indeed.
There are a few "cursed" restaurant buildings around here... I would include Pho Saigon; the original Golden Corral AKA Coco Cantina; Perkins AKA DB McMillans; Roy Rogers which was a few other no-name joints (Cuz's Country Kitchen) which is now Chesapeake Custom Embroidery; Willows which went thru several iterations; there are probably others I'm not remembering.
There are a few common factors from my point of view.
1) Lousy maintenance. Business owners around here seem to have a very short-sighted view of how much money they must allocate for facility renovations. Sometimes I'm astounded at how often Panera Bread changes something about its interior, but as a consequence it always looks fresh and clean and new. That keeps people coming back. By contrast, I generally avoid (for example) IHOP because the bathrooms are a stinking mess and the general interior feels like something straight out of the 80s. My guess is that Western Steer (corner of 4/235, now a WaWa) closed in the early 90's because it was unsustainably grungy inside and needed a complete gut and rebuild to clean it up. I don't even want to know what's in the kitchen and the food from places like that.
2) Too narrow a target market. Cuz's Country Kitchen, serving "southern Gospel" food on Styrofoam plates and plastic cups? Potential market share was near zero from day one. That's been true of many of the other failures.
3) People's desire for new, new, new. When a new place opens nearby, it slurps up all the traffic for a while. That can devastate a small-town, non-chain place that's riding the thin edge of financial solvency. Coco Cantina wasn't bad, but was probably done in by a couple direct competitors with bigger names taking their core customers. With Cracker Barrel open, I'm surprised IHOP is still alive.
4) Memory. I think people still associate the location (unfairly) with the mess that was there before, and write it off their "places I enjoy" mental list.
5) Drop-off in quality. New places often bring in star managers from Corporate to get things kicked off. Then a year later they're running on local talent only, who just never "got" the culture of successful restauranteering. Ever watch Restaurant:Impossible or Bar Rescue shows? Cool. But do you ever read the followup information from real customers? Usually within a month they're back to their failing ways of doing business. Something like 60-70% of rescued places fail within a year or two. The owner just can't change their stripes, and the quality doesn't stick.