early 14c., "quilted material worn under chain mail," from Old French estoffe "quilted material, furniture, provisions" (Modern French étoffe), from estoffer "to equip or stock," which according to French sources is from Old High German stopfon "to plug, stuff," or from a related Frankish word (see stop), but OED has "strong objections" to this. Sense extended to material for working with in various trades (c.1400), then (1570s) "matter of an unspecified kind." Meaning "narcotic, dope, drug" is attested from 1929. To know (one's) stuff "have a grasp on a subject" is recorded from 1927.
mid-15c., "to cram full," from stuff (n.); earlier "to furnish a fort or army with men and stores" (c.1300). The ballot-box sense is attested from 1854, American English; in expressions of contempt and suggestive of bodily orifices, it dates from 1952. Stuffing "seasoned mixture used to stuff fowls before cooking" is from 1530s. Stuffed in reference to garments, "padded with stuffing" is from mid-15c.; hence stuffed shirt "pompous, ineffectual person" (1913).