258 or 304 is personal preference.
I had a 258 in my last CJ that I built with Clifford Performance products (390 Holley 4-BBL on a 4-BBL Clifford Water Heated Aluminum Intake, Dual Exhaust Clifford Headers mated to a 2 1/2" custom exhaust run all the way out the back, and Mild Duration Cam). It was a great performer and great daily driver, and when I went to a gas station, people used to ask me if it was a 350 in there, and they S**T when I said it was an inline 6
It was fun and different engine that many people don't see everyday, so it gets attention.
My next Jeep listed below in my signature has a 360, and I have yet to drive it around to experience it. My next frame-up build will be getting a 401.
So, basically, do whatever you feel like
and whatever you think will be most fun. a 350 swap will be more costly due to the adapters you will need, etc., and you can get plenty of performance out of a 304, if you decide to go V8.