Fan ticking noise. Starter?

So yesterday i was changing my oil and notice that the 7/16 bolt was missing from my starter. So after hours of searching i finally found the bolt and installed it. Imediatly after i here this ticking noise that resembles a fan hitting something and assume it would be the starter. It seems to go away when im in park and does not happen on start up. It also will go and come depending on being in reverse, neutral, or drive but is never constant or non existant in any position. It does seem to accelerate with the engine rpms. It does sound like its coming from that part of the vehicle also(starter area). Im assuming now that the starter is in its rightful position that its making this noise because it had not before. Should i test my wallet and by a new starter and see what happens or should i just ride it out and see if it stops? Can this cause major damage besides to just end up having to replace the starter anyways? Thanks

Fan ticking noise. Starter?