Well, I pulled the fender did some more measuring and it seems to be that the wheel IS out of dish. Dropping a straight edge from various points on the frame, and measuring from rim to swingarm shows the rim to be sitting 13mm closer to the drive side. So @6mm off.
Oddly enough looking at the underside of the fender, there is a clean "stripe" that runs pretty darn close to 6mm off center. Not very scientific, but seems to confirm the other measurements.
Deon: I did check the bolts going through the bag mounts, and they were protruding a bit past the threaded nut on the fender. Looking at the tire closer though it seems that the welded nuts were hitting too.
PO said nothing has ever been done to the wheel aside from tire changes, and owned the bike from new...so who knows.
Looks like I have a trip to the dealer in my future.