Deliberately, intentionally, as in He left the photo out of the story on purpose. Shakespeare's use of this idiom was among the earliest; it appears in The Comedy of Errors (4:3): "On purpose shut the doors against his way."
accidentally on purpose. Seemingly accidentally but actually deliberately, as in She stepped on his foot accidentally on purpose. This generally jocular phrase was first recorded in 1862.