With only 8100 miles on your bike, you shouldn't have a problem just changing the front sprocket. That's only about 1/3rd to 1/2 the chain's normal useful life.
So do the 05 Speedmasters have the 16 tooth sprocket? I thought I'd heard they went to 17 tooth, but I don't know if that's true. I think 17 tooth is just right for this bike, but others swear by the 18 tooth. With a 19 tooth, you're going to lose a lot of acceleration off the line, and I don't think it's worth that sacrifice. Just depends on your riding style.
Since we have Brent at newspeedmaster.com selling front sprockets for $20 each plus shipping, it's cheap enough to do some experimenting to find which size suits you. I certainly don't think cruising at 4,000 or even 4,500 rpm is harming the motor at all. These bikes are bulletproof by all accounts.