against one's better judgment
Despite serious misgivings or objections, as in Against my better judgment, I told her to come whenever she pleased.
