Is it farther or further?
early 13c. "millstone," from grind (v.) in sense of "sharpen" + stone (n.); meaning "revolving stone disc used for sharpening, etc." is from c.1400. Phrase nose to the grindstone in use by 1530s; originally to get control of another and treat him harshly:
This Text holdeth their noses so hard to the grindstone, that it clean disfigureth their Faces. [John Frith, "Mirror to know Thyself," 1532]The main modern (reflective) sense of "work hard" is from 1828.