I used to feel that way, as well, but my current position is this, after all is said and done (i.e. tuition, room and board, etc. is paid) the full scholarship athlete can take out the same student loans the non revenue sport athletes have to take out to pay for their education. They just won't have to borrow nearly as much.
All three of my kids are/were Division 1 athletes. One actually made it to the NCAA Championships. But they were in non revenue sports and only got partial scholarships. My daughter competes for an Ivy League school (hurdler and heptathlete)and they don't give athletic scholarships (or any other kind, as far as we can tell).