The subtext is that each of us is responsible for all of us, and if we intermarry, we risk disappearing completely.
Everyone understood that people of different races could intermarry, in principle.
1570s, "to marry one another," from inter- + marry. Meaning "to marry across families, castes, tribes, etc." is from 1610s. Related: Intermarried; intermarrying.