They will not physically stop you from leaving the US with the bike (since theres no one checking), but on the canadian side you will need to report to customs. When you export a vehicle from the US, you get a form (can't remember what it is called anymore) you give to the canadian side. It is possible you can get through without it, but I cannot confirm that. Haven't heard of anyone else doing it, but doesn't mean its not doable.
Don't be a road crayon, pavement does not care how cool you think you look in a t-shirt and shorts.