down on one's luck
Afflicted by misfortune, as in They've been down on their luck ever since they moved out West. [Colloquial; second half of 1800s]
