i used to get that too untill i got jims spacers and adjusted my bars ever so slightly. it often dont take much in the way of moveing up or down but a slight adjustment can alter the effects of a long ride.
you might want to try moveing them just a little bit at a time up or down untill you find the sweet spot. i marked with a sharpie on the riser and the bar and worked from that starting point, just a little at a time
