Most of the time, people run skinny tires on 21 inch rims. Because of this, the total assembly diameter comes out about the same. So, the geometry of the setup probably won't change much.
If you are given to cornering right at the traction limit, you might do well to reduce your speed around bends untill you get a good feel for what your new limits are. There are 2 opposite forces working there. A skinny tire will have a smaller patch on the ground giving you traction. At the same time, with fewer square inches of rubber on the ground, the weight per square inch increases and gives you more traction. How it balances out is anybodies guess.
You will notice a considerable difference in ride. A skinny tire will absorb a lot less road vibration. At the same time, the wheel will probably weigh a little less, reducing your unsprung weight and improving other aspects of ride quality.
The reduced weight will also tend to make your steering seem a bit quicker, so ride as if on loose sand for a few days untill you get the feel of it.
Overall, your bike probably won't handle much better or worse, just feel a bit different. So, just take it easy for a while and don't worry because nothing really wierd is likely to happen.
You might have to raise your front fender (mudguard) a little, depending on the tire size you end up with.