Phase Contrast Magnetic Resonance Image (PC-MRI) is an emerging noninvasive technique that contains pulsatile information by measuring the parameters of cerebrospinal fluid (CSF) flow. As CSF flow quantities are measured from the selected region on the images, the accuracy in the identification of the interested region is the most essential, and the examination requires a lot of time and experience to analyze and for accurate CSF flow assessment. In this study, a three-dimensional (3D)-Unet architecture, including pulsatile flow data as the third dimension, is proposed to address the issue. The dataset contains 2176 phase and rephase images from 57 slabs of 39 3-tesla PC-MRI subjects collected from the lower thoracic levels of control and Idiopathic Scoliosis (IS) patients. The procedure starts with labeling the CSF containing spaces in the spinal canal. In the preprocessing step, unequal cardiac cycle images (i.e., frame) and the numbers of MRIs in cases are adjusted by interpolation to align the temporal dimension of the dataset to an equal size. The five-fold cross-validation procedure is used to evaluate the 3D Attention-U-Net model after training and achieved an average weighted performance of 97% precision, 95% recall, 98% F1 score, and 95% area under curve. The success of the model is also measured using the CSF flow waveform quantities as well. The mean flow rates through the labeled and predicted CSF lumens have a significant correlation coefficient of 0.96, and the peak CSF flow rates have a coefficient of 0.65. To our knowledge, this is the first fully automatic 3D deep learning architecture implementation to segment spinal CSF-containing spaces that utilizes both spatial and pulsatile information in PC-MRI data. We expect that our work will attract future research on the use of PC-MRI temporal information for training deep models.