Using the k · p theory and configuration interaction method, we analyze the effect of heavy hole–light hole (HH–LH) mixing in CdTe quantum dots (QDs) with a single manganese (Mn) ion. We find that the hole-Mn exchange switches the coupling between two excitons whose Luttinger spinors have both HH and LH components. If the magnetic dopant is off-centered and the QD is subjected to a single π pulse the system periodically bounces between bright and dark mostly HH excitons with opposite HH spins. A pump-and-probe setup allows to estimate the efficiency of this HH spin unpinning from the biexciton response. The biexciton absorption spectrum is also discussed.