I seem to recall my Dad borrowed from his retirement for a truck... and not long after he retired. He didn't think it was a big deal because he was "borrowing from himself." He didn't plan accordingly and got killed by penalties and taxes. I think he eventually declared bankruptcy over it.
