I chased a guy who did that to me about 5 0r 6 miles once.....then thought to muself "Ok after you drag this guy out and beat him half to death....then what"?.....slowed down backed off and went about my bussiness.
Now i just do as you said...stay as close to the center as possible,but out of the greese spot.