Returning it wont do any good. It's not like buying a tent from Wal Mart, never using it, and taking it back. Retainers are custom fitted.
As for the insurance side of things, most policies are written to exclude cosmetic work unless there is an endorsement to the declaration page. You would have paid for said endorsement. Unfortunately, you're contractually obligated to comply with the conditions and terms written on that page. You're also contractually obligated to pay your ortho bill if you signed anything, which includes a release to perform any type of preliminary procedure, such as xrays, molds, ect...
The ortho office has also created an issue for themselves due to them not confirming the policy coverage would be applicable to this situation. Keep in mind that they too have multiple policies in place in order to practice their trade. One of which in Pro Liability with an E&O Clause specifically written in for these types of mistakes.
Contact your dental insurance provider and ask to see the policy dec page along with the list of exclusions and endorsements. Should you find the devil in the details there, you'll know whether or not you have ground to stand on.
As for the collections company, you have a window of roughly 30 days in most states before they ever pursue you in a manner that creates discomfort. A second notice is mailed via US Postal, which provides an additional 5-7 days. Considering you they use a national brand entity, you'll be one of thousands and small fries such as a retainer would be low on the pecking order.
I'm in insurance so take it for what its worth. Good luck.