The electronic properties of a square lattice under an applied perpendicular magnetic field in the presence of impurities or vacancies are investigated by the tight-binding method including up to second nearest neighbor interactions. These imperfections result in new gaps and bands in the Hofstadter butterfly even when the second order interactions break the bipartite symmetry. In addition, the whole spectrum of the Hall conduction is obtained by the Kubo formula for the corresponding cases. The results are in accordance with the Thouless-Kohmoto-Nightingale-den Nijs integers when the Fermi energy lies in an energy gap. We find that the states due to the vacancies or impurities with small hopping constants are highly localized and do not contribute to the Hall conduction. However, the impurities with high hopping constants result in new Hall plateaus with constant conduction of σ xy=±e 2/h, since high hopping constants increase the probability of an electron contributing to the conduction. © 2012 IOP Publishing Ltd.