SCALCE: Boosting sequence compression algorithms using locally consistent encoding
Please cite this item using this persistent URLhttp://hdl.handle.net/11693/21215
- Research Paper 
Motivation: The high throughput sequencing (HTS) platforms generate unprecedented amounts of data that introduce challenges for the computational infrastructure. Datamanagement, storage and analysis have become major logistical obstacles for those adopting the new platforms. The requirement for large investment for this purpose almost signalled the end of the Sequence Read Archive hosted at the National Center for Biotechnology Information (NCBI), which holds most of the sequence data generated world wide. Currently, most HTS data are compressed through general purpose algorithms such as gzip. These algorithms are not designed for compressing data generated by theHTSplatforms; for example, they do not take advantage of the specific nature of genomic sequence data, that is, limited alphabet size and high similarity among reads. Fast and efficient compression algorithms designed specifically forHTS data should be able to address some of the issues in data management, storage and communication. Such algorithms would also help with analysis provided they offer additional capabilities such as random access to any read and indexing for efficient sequence similarity search. Here we present SCALCE, a 'boosting' scheme based on Locally Consistent Parsing technique which reorganizes the reads in a way that results in a higher compression speed and compression rate, independent of the compression algorithm in use and without using a reference genome. Results: Our tests indicate that SCALCE can improve the compression rate achieved through gzip by a factor of 4.19-when the goal is to compress the reads alone. In fact, on SCALCE reordered reads gzip running time can improve by a factor of 15.06 on a standard PC with a single core and 6 GB memory. Interestingly even the running time of SCALCE \+ gzip improves that of gzip alone by a factor of 2.09. When compared with the recently published BEETL, which aims to sort the (inverted) reads in lexicographic order for improving bzip2 SCALCE\+gzip provides up to 2.01 times better compression while improving the running time by a factor of 5.17. SCALCE also provides the option to compress the quality scores as well as the read names in addition to the reads themselves. This is achieved by compressing the quality scores through order-3 Arithmetic Coding (AC) and the read names through gzip through the reordering SCALCE provides on the reads. This way, in comparison with gzip compression of the unordered FASTQ files (including reads, read names and quality scores), SCALCE (together with gzip and arithmetic encoding) can provide up to 3.34 improvement in the compression rate and 1.26 improvement in running time. © The Author 2012. Published by Oxford University Press. All rights reserved.
Showing items related by title, author, creator and subject.
Huddleston J.; Ranade, S.; Malig, M.; Antonacci F.; Chaisson, M.; Hon L.; Sudmant P.H.; Graves, T.A.; Alkan, C.; Dennis, M.Y.; Wilson, R.K.; Turner, S.W.; Korlach J.; Eichler, E.E. (Cold Spring Harbor Laboratory Press, 2014)Obtaining high-quality sequence continuity of complex regions of recent segmental duplication remains one of the major challenges of finishing genome assemblies. In the human and mouse genomes, this was achieved by targeting ...
Dalkic, E.; Kuscu, C.; Sucularli, C.; Aydin I.T.; Akcali, K.C.; Konu O. (2006)Robo2, a member of the robo gene family, functions as a repulsive axon guidance receptor as well as a regulator of cell migration and tissue morphogenesis in different taxa. In this study, a novel isoform of the zebrafish ...
Atalay V.; Cetin-Atalay, R. (2005)Motivation: We designed a general computational kernel for classification problems that require specific motif extraction and search from sequences. Instead of searching for explicit motifs, our approach finds the distribution ...