How to serialise SeqAn data structures

Table of Contents

• Motivation
• Storing
• Loading
• Storing & Loading in the same function

This HowTo shows how to serialise data structures with cereal. Every SeqAn data structure which is is marked as cerealisable can be used.

Difficulty	Easy
Duration	15 min
Prerequisite tutorials	No prerequisites
Recommended reading

Motivation

Storing and loading data, for example a FM-Index, is a common use case. Thanks to the cereal library doing so is incredible easy. This page will show you how to use cereal in SeqAn. As an example data structure we will use a std::vector, but as already mentioned, any SeqAn data structure that is documented as cerealisable can be used.

Storing

Storing a data structure is as easy as using the cereal::BinaryOutputArchive. In order to use it, you need to include

#include <cereal/archives/binary.hpp> // includes the cereal::BinaryOutputArchive

Note that stl data types, like std::vector, are not automatically usable with cereal. You need to include the respective header, e.g.

#include <cereal/types/vector.hpp>    // includes cerealisation support for std::vector

In SeqAn, our data structures have integrated cereal support so there is no need to include an extra header.

#include <fstream>
#include <vector>
 
#include <cereal/archives/binary.hpp> // includes the cereal::BinaryOutputArchive
#include <cereal/types/vector.hpp>    // includes cerealisation support for std::vector
 
#include <seqan3/core/debug_stream.hpp>
#include <seqan3/test/tmp_filename.hpp>
 
// Written for std::vector, other types also work.
void store(std::vector<int16_t> const & data, seqan3::test::tmp_filename & tmp_file)
{
    std::ofstream os(tmp_file.get_path(), std::ios::binary); // Where output should be stored.
    cereal::BinaryOutputArchive archive(os);                 // Create an output archive from the output stream.
    archive(data);                                           // Store data.
}
 
int main()
{
    // The following example is for a std::vector but any seqan3 data structure that is documented as serialisable
    // could be used, e.g. seqan3::fm_index.
    seqan3::test::tmp_filename tmp_file{"data.out"}; // This is a temporary file, use any other filename.
 
    std::vector<int16_t> vec{1,2,3,4};
    store(vec, tmp_file);                            // Calls store on a std::vector.
 
    return 0;
}

Loading

Loading a data structure is as easy as using the cereal::BinaryInputArchive. In order to use it, you need to include

#include <cereal/archives/binary.hpp> // includes the cereal::BinaryInputArchive and cereal::BinaryOutputArchive

#include <fstream>
#include <vector>
 
#include <cereal/archives/binary.hpp> // includes the cereal::BinaryInputArchive and cereal::BinaryOutputArchive
#include <cereal/types/vector.hpp>    // includes cerealisation support for std::vector
 
#include <seqan3/core/debug_stream.hpp>
#include <seqan3/test/tmp_filename.hpp>
 
// Written for std::vector, other types also work.
void load(std::vector<int16_t> & data, seqan3::test::tmp_filename & tmp_file)
{
    std::ifstream is(tmp_file.get_path(), std::ios::binary); // Where input can be found.
    cereal::BinaryInputArchive archive(is);                  // Create an input archive from the input stream.
    archive(data);                                           // Load data.
}
 
// Written for std::vector, other types also work.
void store(std::vector<int16_t> const & data, seqan3::test::tmp_filename & tmp_file)
{
    std::ofstream os(tmp_file.get_path(), std::ios::binary); // Where output should be stored.
    cereal::BinaryOutputArchive archive(os);                 // Create an output archive from the output stream.
    archive(data);                                           // Store data.
}
 
int main()
{
    // The following example is for a std::vector but any seqan3 data structure that is documented as serialisable
    // could be used, e.g. seqan3::fm_index.
    seqan3::test::tmp_filename tmp_file{"data.out"}; // This is a temporary file, use any other filename.
 
    std::vector<int16_t> vec{1,2,3,4};
    store(vec, tmp_file);                            // Calls store on a std::vector.
    // This vector is needed to load the information into it.
    std::vector<int16_t> vec2;
    load(vec2, tmp_file);                            // Calls load on a std::vector.
 
    seqan3::debug_stream << vec2 << '\n';             // Prints [1,2,3,4].
 
    return 0;
}

Storing & Loading in the same function

In the example above loading and storing was encapsulated in separated functions. It is possible to use cereal::BinaryInputArchive and cereal::BinaryOutputArchive in one function, but then it is necessary to encapsulate each in an individual scope (using extra braces{}). The reason for this is that the output/input stream handle of an archive is closed on deconstruction and only then is the file properly written an accessible by another filehandle.