Vacuum Leak low uneven idle
So I have a 97 TJ with a 2.5L and manual tranny. I've noticed that lately my idle has been low and a bit uneven, (goes up a bit then down a bit on regular, 1-second or so intervals). I'm pretty sure it is some sort of vacuum problem, because whenever I turn on the fan, the idle goes up a little (but still isn't steady.) Also, sometimes when I'm driving at high RPMS, I lose vacuum and the blower turns off and is replaced with a whirring sound until I slow down or put in the clutch.
I don't know very much about the vacuum system, and I was wondering if anyone could, based on this, give me a good idea of what to check first? Thanks,