rough on, be
Be harmful to or difficult for, as in The harsh winter has been rough on the highways, or Their divorce was rough on the whole family.
Treat harshly, be severe with, as in The police have been very rough on house-breakers, or Don't be too rough on Sam; he's only a child. Both colloquial usages date from the second half of the 1800s.