Considering that Nightmare Moon's legend morphed her into a cannibal pony-eater, I can easily envision the fear that a Pony would have for a Horse. Especially an enraged Horse.
Horses are essentially the giants nowadays. But it's far more complex than that.
What I have in mind:
In the First Age, two proud Equine races ruled the lands of Equestria in harmony: Horses and Alicorns [see separate entry for Horse and Alicorn]. Alicorns, less numerous, led the hiearchy and formed the noble-class, leading their Horse-kin in founding the once-great realm of Equestria.
Horse-kind formed the working-class - tireless, strong, brave. Led by Alicorn officers and Nobility, the Horses harnessed the earth and dominated it. The Great Army of the First Age ensured that all citizens of Equestria lived, worked and slept in absolute safety.
In harmony, the two Equine races inter-married with one another. And a curious thing happened. Offspring of this perfect union formed... Ponies.
Alicorn mystics were befuddled by this matter. Tiny foals, who grew to full maturity with a mere fraction of the stature of their progenitors. Theyy were different in other ways. Some had only wings, others had only horns, and some had neither. Thus, was born the equine races of Pegasi, Unicorn, and Earth Pony.
-excerpt from Ancient Equestrian Lore
Alicorns used to be taller, so they were pretty much the same height. Might show some long-abandoned ruins from that ancient time and ponies are all "wow, they made everything big here" or something.
They inter-bred, and the result was pony-kind - earth ponies are much like their Horse parents, while Pegasi and Unicorns got wings or horn traits from their Alicorn heritage. Some were born as Alicorns, although smaller in stature (we can debate that part). As a result of inter-breeding, pure-blood Horse and Alicorn populations declined rapidly, until both were virtually extinct. Celestia and Luna are (well, could be)purebloods. Heck, Celestia could be old enough to recall pure-bred Horses.
Anyway.
Horses are thought to be long-extinct. Their closests genetic off-spring are Earth Ponies, as they're both fairly mundane. Earth Ponies have inate magic, though, as they have tracees of Alicorn blood in them. Twilight may have read about all of this at some point, whereas Horses had passed from history into legend, legend into myth. It's no longer common knowledge.
Now I just have to figure out where the heck Zebra Ponies came from.