I would leave the caster alone. Less caster would cause less steering return-to-center tendancy, and they will only cause a pull if they are drastically different left to right(yours are fine). Your camber (nonadustable on TJ) would cause a slight pull to the left, so I doubt that's your pull. Your rear axle is side tracking to the left, so the front toe is adjusted to compensate for that.
I would normally suspect a tire related pull and do a straight track/road force check of the tires, but very difficult and inaccurate to do on mud terrain tires.
Only thing i can guess is it's sensitive to road crown. The road will have a very slight curve side to side to help rain runoff, so if you are always in the left lane or on a 2 lane road, the road crown for you is sloping to the right, and that might be the pull you are feeling. Try driving on a divided highway in the left lane and see if the pull is less.
'98 TJ has been under the knife the last 30 months getting a new frame, torque boxes and fenders.