High-level wizards are super scary in D&D and similar games. But as far as frightening antagonists go, high-level fighters and thieves (or specialists) are no slouches, either. Look at the Halloween and Friday the 13th franchises. Later entries would explicitly give the villains supernatural abilities, but in the early films these "inhuman" killing machines were simply* presented as extremely strong, durable, persistent, evil people skilled at murder. Depending on how things like stealth and stunts and multiclassing work in your OSR game of choice, I don't see why Michael and Jason couldn't just be fighters and/or thieves of, say, ninth level.
If the average person in your game is a 0-level or first level character, imagine how terrifying an opponent a determined high-level fighter would be if they just randomly went on a methodical rampage. If you manage to land a hit, they shrug off damage that would instantly kill a normal human. If you lure them into a trap, they effortlessly make their saving throw. You'd best find a way to stop them without rolling any dice, because the math favors them every time. And they're not stupid: they use tools and tactics. They use the environment to their advantage. They leave their own traps for you. And they won't be talked down or bargained with.
Imagine an invisible presence picking off your friends one by one. In a world of magic, your mind jumps to distracting speculation about monsters and spirits, while some asshole with a knife just walks up behind you. It's the most obvious, the most natural thing in the world. No one hears you die.
Sure, wizards are bad news. But don't underestimate the old man in the swamp or the jolly jester.
*I guess you could interpret the killers as ambiguously supernatural in the earlier films if your mind's eye squints a little, but I don't recall any such speculation among the characters to that effect (other than some talk of the "boogeyman" in Halloween that I took as metaphorical), and there's nothing to confirm any magical aspects to the villains until later entries in the series. If Friday the 13th ended with the fourth or fifth installment, Jason Voorhees could easily just be a scary dude in a mask instead of some kind of super-zombie. I haven't seen as many of the Halloween movies, but I hear the situation is similar. Michael Myers just struck me as a mortal man with a scary mind in his debut film. Granted, he did some pretty improbable things, but I chalked that up to movie magic rather than literal in-universe magic. My point is that Voorhees and Myers don't need to be supernatural.