Changes in toileting behavior in cats are a very common sign of illness; if you have not taken your cat to a vet to have them evaluate the issue, you should do so immediately, as it sounds like you've been letting this issue go on for a very long time without properly addressing it. Male cats in particular can be very susceptible to urinary tract issues that can turn fatal.
In addition, you need to look at how you are handling cleaning of both his litter box and the other areas where he's relieved himself. Keep the litter box clean; if you aren't cleaning it at least once a day, start doing that. Additionally, while you might think your clothes are clean, regular detergent does not break down all the enzymes in cat urine, which will make them smell like an appropriate toilet to your cat until you properly remove them. Get an enzymatic laundry additive specifically for cleaning pet messes, and wash everything with that, whether or not you think it's clean. Use it every time your cat urinates or defecates on a garment. Furthermore, you'll need to clean the surfaces below those items, using a surface enzymatic cleaner for pet messes (appropriate to the surface you're cleaning up). Lastly, pick up your clothes and put them away properly. If you don't leave them where the cat can use them as a toilet, the cat won't do so.
None of the above is a substitute for a vet visit, and a vet visit is not a substitute for any of the above cleaning steps. You will need to do both to properly address the situation. In the meantime, stop taking your anger out on your cat. He isn't feeling well, and you lashing out at him isn't making him feel any better. Treat him with kindness, and make sure you address the root causes of the problem. He's not doing any of this because he's "mad at you" or "naughty," there are always underlying reasons, and unless you address those, nothing will ever change.
