So you're basically training your models on a digital hoarder's garage? That's one way to make them tough I guess. Just hope they learn to ignore the junk and not start collecting it. Next you'll be feeding them blurry pictures on purpose to fix their vision. Where does it end?
Honestly that "real-world messes" part is key. Tbh a lot of real data has hidden patterns in the noise, not just pure randomness. Adding structured junk might teach the model to ignore actual useful signals.