(Old fashioned) Small village. Imagine a town  so small, it only has one  horse implying very, very few people live there!

“I’m from Kentucky, from as one horse town and I’m glad to live in the city now.”