take its toll
Be damaging or harmful, cause loss or destruction, as in The civil war has taken its toll on both sides, or The heavy truck traffic has taken its toll on the highways. This expression transfers the taking of toll, a tribute or tax, to exacting other costs. [Late 1800s]