Mine was the same way on the same side. Just ensure you only grip the plastic connector and work it side-to-side. It will come off and you won't have difficulty in the future.

I had the same concern my first time adjusting.