That has happened to me... twice. Once they sent spam to abuse@ and once to postmaster@. Both of those are "well-known" addresses that received one spam mail each. Having your own domain with a catch-all address is rare enough that spammers don't seem to try to target it. Meanwhile I set up straight-to-spam rules for a handful of companies that leaked my email address. Very useful.