If you use the hole saw its best. I tired but my drill didn't have enough HP to drive the hole saw.
So, I just punched a bunch of holes around the baffles using a long a long bit. Then bang them in with a pipe and pull them out.
I still have the hole saw if anyone needs it.