Is it farther or further?
mid-15c., "handle clumsily," possibly from Old Norse falma "to fumble, grope." Similar words in Scandinavian and North Sea Germanic suggest onomatopoeia from a sound felt to indicate clumsiness (cf. bumble, stumble, and obsolete English famble, fimble of roughly the same meaning). Related: Fumbled; fumbling.
1640s, from fumble (v.).