That's always a concern but like the Ruger 10/22 Takedown, I don't believe it is an issue (kind of similar in design). The barrel with breech assembly is indexed to tight tolerances so it can only fit in a very specific way. When it locks in it is solid and has zero play. Also when the bolt is forward, it locks in the assembly and you can't remove it. I haven't had a chance to test whether or not swapping barrels affects zero, but I doubt it would significantly.