This is a pretty interesting question. I too had always heard that the proper usage of "alumnus" was for people who had actually graduated from a particular institution, but all the dictionaries I can can find -- including my OED, which is of course as close to an ultimate authority as there is -- include the "former student" usage as well. My usage guides don't address it, as they're fixated on the misuse of the plural "alumni" as a singular noun (e.g., "I'm an alumni of UT.") The "former student" usage doesn't bother me, of course, but I'm curious -- from where, then, did the notion that it applies only to graduates come?
From Latin alere, meaning "to nourish," if any other word nerds care about that sort of thing.