Detection of compound structures that are comprised of different arrangements of simpler primitive objects has been a challenging problem as commonly used bag-of-words models are limited in capturing spatial information. We have developed a generic method that considers the primitive objects as random variables, builds a contextual model of their arrangements using a Markov random field, and detects new instances of compound structures through automatic selection of subsets of candidate regions from a hierarchical segmentation by maximizing the likelihood of their individual appearances and relative spatial arrangements. In this paper, we extend the model to handle different types of primitive objects that come from multiple hierarchical segmentations. Results are shown for the detection of different types of housing estates in a WorldView-2 image. © 2016 IEEE.