eat away at
Destroy gradually, erode; also, worry one constantly. For example, The sea has been eating away at the outer banks for years, or The fact that he failed the test is eating away at him. [Early 1800s]
