We propose and demonstrate a new type of propagation mechanism for electromagnetic waves in photonic band gap materials. Photons propagate through coupled cavities due to interaction between the highly localized neighboring cavity modes. We report a novel waveguide, which we called coupled-cavity waveguide (CCW), in three-dimensional photonic structures. By using CCWs, we demonstrate lossless and reflectionless waveguide bends, efficient power splitters, and photonic switches. We also experimentally observe the splitting of eigenmodes in coupled-cavities and formation of defect band due to interaction between the cavity modes. The tight-binding (TB) approach, which is originally develop for the electronic structures, is applied to the photonic structures, and compared to the experimental results. Our achievements open a new research area, namely physics and applications of coupled-cavities, in photonic structures. We think that our results are very important for constructing future all-optical components on a single chip.