This paper describes an online learning based method to detect flames in video by processing the data generated by an ordinary camera monitoring a scene. Our fire detection method consists of weak classifiers based on temporal and spatial modeling of flames. Markov models representing the flame and flame colored ordinary moving objects are used to distinguish temporal flame flicker process from motion of flame colored moving objects. Boundary of flames are represented in wavelet domain and high frequency nature of the boundaries of fire regions is also used as a clue to model the flame flicker spatially. Results from temporal and spatial weak classifiers based on flame flicker and irregularity of the flame region boundaries are updated online to reach a final decision. False alarms due to ordinary and periodic motion of flame colored moving objects are greatly reduced when compared to the existing video based fire detection systems.