From 03-06 jeep tailgates had vents in them to let air escape, and from 97-02 jeeps had vents in the hardtops. But everything should bolt together fine. I had a 98 jeep with hardtop and I replaced the body with an 2006 tub and had no issues with fitment with my 97-02 hardtop.
I think the only issue you will have will be more of an annoyance than anything, because the initial air pressure of slamming a full door shut, might cause it not to fully latch, since the air has no where to escape fast enough. But there is a possibility that you won't have an issue with this problem.
There is no "I" in jeep. Just a lot of money.