If you ever have to take the swingarm off, you might regret going to that extreme. I guess you could get the swingarm off without removing the pipes, but it is easier with them off.
I do keep mine fairly tight though...which is easy with the nylon washer. It hasn't come loose yet in over 2 years.
