Try turning the motor over by hand, use a long ratchet/bar with a big socket on the crank bolt. See if it is seized.
If the motor turns over by hand, try replacing your battery cables.
Just because you just bought the starter doesnt mean it isnt bad. Most of the ones from the auto store are rebuilt.
Check the bendix gear on the starter see if it has any missing teeth, or can spin freely on the shaft. You can also try just replacing it, it may be bad internally.
Hook up a volt meter to the battery, see if you have 12v. Have a friend watch it when you are cranking the starter. (or use long electrical leads). If it falls below 8v or so I would say there is an issue with the battery. I know, I know, you just replaced it. The battery should go below 9.5 volts or so when starting the motor.
Check the flywheel for missing teeth as well. I doubt this is it, because you said it enguages, but couldn't hurt.
92 YJ 4.0 AX-15 NP231 D30 D35 2"Bod 2"Shack
Soon to be
92 YJ 4.7 NP435 NP203->NP205 D60 14bolt SOA Waggy offset