I used a 1/8" drill bit in a right angle dremel attachment. Use what you can to get a hole in the cap and then screw in a drywall screw and it doesn't matter if you bugger it up. Then grab the screw with a pair of pliers and rip that sucker out.
I actually had to do this a couple of times as the screw kept popping out...
