Unbalance side to side will cause as much trouble as unbalance top to bottom, just in a different direction. So, yes, it can cause a wobble.
Anyone old enough to remember the old steel wheeled clamp-on sidewalk skates? Turn the skate upside down and give the wheels a spin, and nearly all would shake side to side so badly at to look out of focus, make a loud buzzing noise and stop turning right away because of this.
