"whittle away," 1728, from fritters "fragment or shred," possibly from a noun sense, but this is not recorded as early as the verb; perhaps an alteration of 16c. fitters "fragments or pieces," perhaps ultimately from Old French fraiture "a breaking," from Latin fractura. Or perhaps from a Germanic source (cf. Middle High German vetze "clothes, rags").
"fried batter," late 14c., from Old French friture "fritter, pancake, something fried" (12c.), from Late Latin frictura "a frying," from frigere "to roast, fry" (see fry (v.)).