Boosting fully convolutional networks for gland instance segmentation in histopathological images

In the current literature, fully convolutional neural networks (FCNs) are the most preferred architectures for dense prediction tasks, including gland segmentation. However, a signi cant challenge is to adequately train these networks to correctly predict pixels that are hard-to-learn. Without additional strategies developed for this purpose, networks tend to learn poor generalizations of the dataset since the loss functions of the networks during training may be dominated by the most common and easy-to-learn pixels in the dataset. A typical example of this is the border separation problem in the gland instance segmentation task. Glands can be very close to each other, and since the border regions contain relatively few pixels, it is more di cult to learn these regions and separate gland instances. As this separation is essential for the gland instance segmentation task, this situation arises major drawbacks on the results. To address this border separation problem, it has been proposed to increase the given attention to border pixels during network training either by increasing the relative loss contribution of these pixels or by adding border detection as an additional task to the architecture. Although these techniques may help better separate gland borders, there may exist other types of hard-to-learn pixels (and thus, other mistake types), mostly related to noise and artifacts in the images. Yet, explicitly adjusting the appropriate attention to train the networks against every type of mistake is not feasible. Motivated by this, as a more e ective solution, this thesis proposes an iterative attention learning model based on adaptive boosting. The proposed AttentionBoost model is a multi-stage dense segmentation network trained directly on image data without making any prior assumption. During the end-to-end training of this network, each stage adjusts the importance of each pixel-wise prediction for each image depending on the errors of the previous stages. This way, each stage learns the task with di erent attention forcing the stage to learn the mistakes of the earlier stages. With experiments on the gland instance segmentation task, we demonstrate that our model achieves better segmentation results than the approaches in the literature.