It's supposed to just sit on the top. The cover that you removed is supposed to hold it down & keep everything sealed.
Under the diaphram, isn't that a dimple that should match up with a slot on top of the carb body? This is to ensure you have the slide/diaphram alligned correctly. Otherwise, yes, it would leak & do strange things.