Long crank before start

Hi everyone, i am brand new to the forum and to owning a jeep! Looking for a little help if possible.. So i just recently bought a used 2001 XJ sport with roughly 150K on it. I am having an issue with a very long crank time before the engine kicks over. Its not every time i start the car, it happens at pretty specific times. For instance, if it has been sitting over night or for at least several hours it has absolutely no problems and starts right up, even in the very cold weather. If it has been very recently started, say less than an hour it has no problem then as well.. Its only when it has been recently started/driven and i go to start it after it sits for just a couple/few hours it will crank for a good 4-5 seconds before kicking in.. Any idea what the issue is??? I just got a new crank sensor and a new gas cap put in and the issue remains.. Please help!!!