It has been shown that parameter varying systems with time delays can be robustly stabilized by switching control, provided that the plant's parameter varies slowly enough such that the dwell time conditions of the switched controllers can be satisfied. In this paper, the minimization of dwell time is considered, where an iterative search algorithm is developed from the singular value perspectives. The local minimal dwell time obtained in this paper can be used to estimate the upper bound on how fast the plant's parameters can vary. Meanwhile, the switching controller synthesis with optimal dwell time is also discussed, where robust stabilizer design algorithm is presented to achieve robust stability at certain operating range, as well as the local minimal dwell time for controller switching. A numerical example is given to illustrate the proposed algorithm.