I have the same thing on my jeep... a strange whirring/rubbing noise coming from under the transmission cover, only mine is very noticeable in first through third gear, and then goes away completely in fourth. Mine even makes the noise in neutral while it's idling. Is there a possibility that there is sand or something inside, or is it necessary to have it rebuilt? I've driven it with this noise for several months now without any noticeable problems except for a slight difficulty going into reverse. I guess I just assumed that this was a sign of normal wear and didn't think much of it...
'85 CJ7, 33x12.5x15 Armstrong Formula ATs, Cartered 258, T176, 2.73 rear, Pioneer CD and lots of character!