all joking aside
Also, all kidding aside. Seriously, as in I know I said I'd quit, but all joking aside this job is too much for one person, or All kidding aside, I hate to lose at croquet. This phrase often accompanies a joking statement.
