You want to create some negative publicity for your contractor.
You want your contractor to correct the work he's done.
You want your money back.
1) Which State you live in: Licensing standards differ widely from state to state. In some States, Licensing Authorities do not work directly with consumer complaints. If you live in one of these States you may usually file a consumer complaint with the office of your State Attorney General.
There are States that do not require contractors to be licensed. In those States you should file complaints with the State's Attorney General's Office. Also check if your city or town has a contractor licensing department since some States allow contractors to be licensed at the local level.
More information and links to the majority of State licensing boards you can find here: State plumbing and contractor licensing departments.
No matter where you live you can complain to the BBB (even in Canada), post your complaint on the Internet, or take your case to court.