A scene is a view of an environment with a spatial layout one can act within. Scene perception has been studied extensively in the neuroscience literature, examining changes in neural activity across the brain and scene-selective regions (PPA, RSC, OPA), in response to various low and high-level features and tasks. The focus of the field has been mostly on outdoor scenes based on broad categorical differences (e.g. natural/man-made) or basic differences between otherwise similar indoor environments (e.g. ceiling height) and behavioral components regarding scene perception have been overlooked. Interactions with fields such as environ-mental psychology or neuroarchitecture, which could inspire a more ecologically valid study of scenes, are rare. Hereby, we investigated the perception of built environments where we spend most of our time, drew our categorization method from the architecture literature, and employed multiple tasks. The categories were elements that (i) allow our access to and circulation within environments (entrances, exits, corridors, stairs); and that (ii) do not directly aid locomotion but rather serve human needs (restrooms, eating and seating areas). FMRI scans were obtained from 23 participants as they viewed scenes from built environments and performed two tasks: a categorization task based on the main afforded action differences between the defined categories, and an approach-avoidance task where participants decided to enter the scene or not, measuring the initial action regarding an environment. Scene-selective ROIs were defined with a localizer session. Univariate analyses did not reveal strong differences between the tasks. Searchlight MVPA revealed categories, but not tasks, are classified at the whole-brain level, at the lingual and parahippocampal gyri, the SMA, and the occipital cortex. Model-based RSA at the ROI level revealed that tasks modulate activation patterns to built environments in all three ROIs, but do not entirely explain them, whereas categorical and visual models did not correlate with the activation patterns in any of these regions. We utilize an interdisciplinary perspective to scene perception to expand the ecological validity of the stimuli and task con-tent, showing that neural responses to built environments are modulated by the behavioral goals of the observer at the ROI level, and stimulus category at the whole-brain level.