In this paper, we study anomaly detection in a distributed network of nodes and introduce a novel algorithm based on Support Vector Machines (SVMs). We first reformulate the conventional SVM optimization problem for a distributed network of nodes. We then directly train the parameters of this SVM architecture in its primal form using a gradient based algorithm in a fully distributed manner, i.e., each node in our network is allowed to communicate only with its neighboring nodes in order to train the parameters. Therefore, we not only obtain a high performing anomaly detection algorithm thanks to strong modeling capabilities of SVMs, but also achieve significantly reduced communication load and computational complexity due to our fully distributed and efficient gradient based training. Here, we provide a training algorithm in a supervised framework, however, we also provide the extensions of our implementation to an unsupervised framework. We illustrate the performance gains achieved by our algorithm via several benchmark real life and synthetic experiments.