How Well Do You Know English Slang?
1530s, "in a literal sense," from literal + -ly (2). Erroneously used in reference to metaphors, hyperbole, etc., even by writers like Dryden and Pope, to indicate "what follows must be taken in the strongest admissible sense" (1680s), which is opposite to the word's real meaning and a long step down the path to the modern misuse of it.
We have come to such a pass with this emphasizer that where the truth would require us to insert with a strong expression 'not literally, of course, but in a manner of speaking', we do not hesitate to insert the very word we ought to be at pains to repudiate; ... such false coin makes honest traffic in words impossible. [Fowler, 1924]