get on someone's nerves
Irritate someone, as in His fidgeting gets on the teacher's nerves, or, as T.S. Eliot put it in The Elder Statesman (1959): "How it used to get on my nerves, when I saw you always sitting there with your nose in a book." [c. 1900]