Distributed stream-processing framework for graph-based sequence alignment

Optimized the sequence alignment pipelines are needed to minimize the time required to complete processing the short-read genomic data. Today there are many sequence alignment tools exist, yet few of them are capable of directly ingesting the streaming base-call data. The sequencing has to be entirely completed before the mainstream aligners can begin mapping the reads to the reference. The sequencing process can take days to complete. The output is then needs to be demultiplexed into individual reads and aligned to the reference, which can take several more hours. Overall time of a genomic analysis can be shortened significantly by progressively computing the alignments at the time when the reads are still being generated. It is important to have genomic analysis done as quickly as possible, especially in life critical situations. Here we introduce a distributed stream processing framework for aligning short-reads into a graph representation of the genome. The massively parallel nature of the genomic sequencing data requires a massively parallel computation architecture. Thus we have designed our pipeline called R2G2Flow to align many reads to a de Bruijn graph in parallel. Our aligning method is specialized for the sequencing technologies that are based on base-call cycles, such as produced by Illumina. The results are made available soon after the final bases from the sequencing devices has been emitted. R2G2Flow is available at https://github.com/BilkentCompGen/r2g2