What can happen is that pieces of gunk will break loose and temporarily clog up oil galleries (channels throughout the engine that the oil circulates through). Also, lifters have a black tar like build up on them and they can stick as they spin (they spin but not strictly by design). They go up and down on the lobes of the cam but spin around too.
I think the best way to limit/eliminate the tick is to switch to synthetic oil if you have not already done that. It will gradually clean things while increasing protection against wear. I changed my lifters for new ones and got rid of most of my tick. It is not a perfect solution and can cause excessive wear on the cam lobes over time. However, there was almost 3mm of wear on some of the lifters. I figured the bigger problem was to leave them in. Plus, I only put about 2k miles on my jeep a year.
1994 Wrangler YJ
31x10.5x15 BFG MT