In this paper, fundamental limits on time delay estimation are studied for cognitive radio systems, which facilitate opportunistic use of spectral resources. First, a generic Cramer-Rao lower bound (CRLB) expression is obtained in the case of unknown channel coefficients and carrier-frequency offsets (CFOs) for cognitive radio systems with dispersed spectrum utilization. Then, various modulation schemes are considered, and the effects of unknown channel coefficients and CFOs on the accuracy of time delay estimation are quantified. Finally, numerical studies are performed in order to verify the theoretical analysis.