Know how to use "fewer" and "less"? Find out.
1530s, scullion, kitchen knave. Perhaps once an actual military or guard unit; more likely originally a mock-military reference to scullions and kitchen-knaves of noble households, of black-liveried personal guards, and of shoeblacks. By 1736, sense had emerged of "one of the criminal class." Hence the adjectival use (1784), "of low or worthless character."