SeqAn3 3.4.0-rc.1
The Modern C++ library for sequence analysis.
|
IO related views. More...
Classes | |
class | seqan3::detail::async_input_buffer_view< urng_t > |
The type returned by seqan3::views::async_input_buffer. More... | |
struct | seqan3::detail::istreambuf_fn |
View adaptor/factory definition for views::istream. More... | |
class | seqan3::detail::view_take_exactly< urng_t, or_throw > |
The type returned by seqan3::views::take, seqan3::detail::take_exactly and seqan3::detail::take_exactly_or_throw. More... | |
class | seqan3::detail::view_take_until< urng_t, fun_t, or_throw, and_consume > |
The type returned by seqan3::detail::take_until and seqan3::detail::take_until_or_throw. More... | |
Variables | |
constexpr auto | seqan3::views::async_input_buffer |
A view adapter that returns a concurrent-queue-like view over the underlying range. | |
constexpr auto | seqan3::detail::istreambuf |
A view factory that returns a view over the stream buffer of an input stream. | |
constexpr auto | seqan3::detail::take_exactly |
A view adaptor that returns the first size elements from the underlying range (or less if the underlying range is shorter); also provides size information. | |
constexpr auto | seqan3::detail::take_exactly_or_throw |
A view adaptor that returns the first size elements from the underlying range and also exposes size information; throws if the underlying range is smaller than size . | |
constexpr auto | seqan3::detail::take_line = detail::take_until_and_consume(is_char<'\r'> || is_char<'\n'>) |
A view adaptor that returns a single line from the underlying range or the full range if there is no newline. | |
constexpr auto | seqan3::detail::take_line_or_throw = detail::take_until_or_throw_and_consume(is_char<'\r'> || is_char<'\n'>) |
A view adaptor that returns a single line from the underlying range (throws if there is no end-of-line). | |
constexpr auto | seqan3::detail::take_until |
A view adaptor that returns elements from the underlying range until the functor evaluates to true (or the end of the underlying range is reached). | |
constexpr auto | seqan3::detail::take_until_and_consume |
A view adaptor that returns elements from the underlying range until the functor evaluates to true (or the end of the underlying range is reached and consumes the end in single-pass ranges). | |
constexpr auto | seqan3::detail::take_until_or_throw |
A view adaptor that returns elements from the underlying range until the functor evaluates to true (throws if the end of the underlying range is reached). | |
constexpr auto | seqan3::detail::take_until_or_throw_and_consume |
A view adaptor that returns elements from the underlying range until the functor evaluates to true (throws if the end of the underlying range is reached and consumes the end in single-pass ranges). | |
IO related views.
|
inlineconstexpr |
A view adapter that returns a concurrent-queue-like view over the underlying range.
urng_t | The type of the range being processed. See below for requirements. |
[in,out] | urange | The range being processed. |
[in] | buffer_size | Size of the buffer. Choose the size (> 0) depending on the expected work per element. |
Header File
#include <seqan3/io/views/async_input_buffer.hpp>
This view spawns a background thread that pre-fetches elements from the underlying range and stores them in a concurrent queue. Iterating over this view then pops elements out of the queue and returns them. This is primarily useful if dereferencing/incrementing the iterator of the underlying range is expensive, e.g. with SeqAn files which lazily perform I/O.
Another advantage of this view is that multiple iterators can be created that are safe to iterate individually, even from different threads, i.e. you can use multiple threads to iterate safely over a single-pass input view with the added benefit of background pre-fetching.
In technical terms: this view facilitates a single-producer, multi-consumer design; it's a range interface over a concurrent queue.
The buffer_size
parameter should be chosen depending on the expected work per element, e.g. if the underlying range is an input file over short reads, a buffer size of 100 or 1000 could be beneficial; if on the other hand the file contains genome-sized sequences, it would be better to buffer only a single sequence (buffering 100 sequences would result in the entire file being preloaded and likely consuming significant memory).
This view always moves elements from the underlying range into its buffer which means that the elements in the underlying range will be invalidated! For underlying ranges that are single-pass, this makes no difference, but it might be unexpected for multi-pass ranges (std::ranges::forward_range).
Typically this adaptor is used when you want to consume the entire underlying range. Destructing this view before all elements have been read will also stop the thread that moves object from the underlying range. In general, it is not safe to access the underlying range in other contexts once it has been passed to seqan3::views::async_input_buffer.
Note that in addition to the buffer of the view, every iterator has its own one-element-buffer. Dereferencing the iterator returns a reference to the element in the buffer, usually you will want to move this element out of the buffer with std::move std::ranges::iter_move. Incrementing the iterator refills the buffer from the queue inside the view (which in turn is then refilled from the underlying range).
concepts and reference type | urng_t (underlying range type) | rrng_t (returned range type) |
---|---|---|
std::ranges::input_range | required | preserved |
std::ranges::forward_range | lost | |
std::ranges::bidirectional_range | lost | |
std::ranges::random_access_range | lost | |
std::ranges::contiguous_range | lost | |
std::ranges::viewable_range | required | guaranteed |
std::ranges::view | guaranteed | |
std::ranges::sized_range | lost | |
std::ranges::common_range | lost | |
std::ranges::output_range | lost | |
seqan3::const_iterable_range | lost | |
std::ranges::range_reference_t | std::ranges::range_value_t<urng_t> & | |
std::iterator_traits ::iterator_category | none |
See the views submodule documentation for detailed descriptions of the view properties.
The following operations are thread-safe:
.begin()
and .end()
on the view returned by this adaptor;Calling operators on the same iterator object from different threads is not safe, i.e. you can pass the view to different threads by reference, and have each of those threads call begin()
on the view and then perform operations (dereference, increment...) on that iterator from the respective thread; but you cannot call begin()
in a parent thread, pass the iterator to different threads and operate on that concurrently.
Running the snippet could yield the following output:
This shows that indeed elements from the underlying range are processed non-sequentially, that there are two threads and that work is "balanced" between them (one thread processed more element than the other, because its "work" per item happened to be smaller).
Note that you might encounter jumbled output if by chance two threads write to the stream at the exact same time.
If you remove the line starting with auto f1 = ...
you will get sequential processing:
Note that even if you have a single processing thread, using this view can still improve performance measurably, because loading of the elements into the buffer (which reads input from disk) happens in a background thread.
|
inlineconstexpr |
A view factory that returns a view over the stream buffer of an input stream.
istreambuf_t | The type of the stream(buffer); must be std::basic_streambuf or model seqan3::input_stream. |
[in] | istreambuf | The stream buffer or an input stream of whom the buffer is retrieved. |
Header File
#include <seqan3/io/views/detail/istreambuf_view.hpp>
This is a source-only view adaptor, also known as a range factory; you cannot pipe anything into it.
Concepts and traits | rrng_t (returned range type) |
---|---|
std::ranges::input_range | guaranteed |
std::ranges::forward_range | |
std::ranges::bidirectional_range | |
std::ranges::random_access_range | |
std::ranges::contiguous_range | |
std::ranges::viewable_range | guaranteed |
std::ranges::view | guaranteed |
std::ranges::sized_range | |
std::ranges::common_range | |
std::ranges::output_range | |
seqan3::const_iterable_range | guaranteed |
std::ranges::range_reference_t | istream_t::char_type |
See the views submodule documentation for detailed descriptions of the view properties.
This adaptor is different from std::ranges::basic_istream_view in that it operates directly on the buffer. It further uses a custom streambuf_iterator (not std::istreambuf_iterator) that performs less virtual function calls.
|
inlineconstexpr |
A view adaptor that returns the first size
elements from the underlying range (or less if the underlying range is shorter); also provides size information.
urng_t | The type of the range being processed. See below for requirements. [template parameter is omitted in pipe notation] |
[in] | urange | The range being processed. [parameter is omitted in pipe notation] |
[in] | size | The target size of the view. |
size
elements of the underlying range. Header File
#include <seqan3/io/views/detail/take_exactly_view.hpp>
Concepts and traits | urng_t (underlying range type) | rrng_t (returned range type) |
---|---|---|
std::ranges::input_range | required | preserved |
std::ranges::forward_range | preserved | |
std::ranges::bidirectional_range | preserved | |
std::ranges::random_access_range | preserved | |
std::ranges::contiguous_range | preserved | |
std::ranges::viewable_range | required | guaranteed |
std::ranges::view | guaranteed | |
std::ranges::sized_range | guaranteed | |
std::ranges::common_range | preserved | |
std::ranges::output_range | preserved except if urng_t is std::basic_string | |
seqan3::const_iterable_range | preserved | |
std::ranges::range_reference_t | std::ranges::range_reference_t<urng_t> |
See the views submodule documentation for detailed descriptions of the view properties.
The difference to seqan3::views::take is that this view always exposes size information – even if the underlying range is not sized. You should only use this if you know that the underlying range will always be at least size
long.
For seqan3::detail::take_exactly if the underlying range is shorter than size
, the behaviour is undefined. seqan3::detail::take_exactly_or_throw is a safer alternative, because it throws an exception when an iterator before the size
-th one compares equal to the end sentinel; and it also throws on construction if it knows that the underlying range is smaller.
|
inlineconstexpr |
A view adaptor that returns the first size
elements from the underlying range and also exposes size information; throws if the underlying range is smaller than size
.
seqan3::unexpected_end_of_input | If the underlying range is smaller than size . |
urng_t | The type of the range being processed. See below for requirements. [template parameter is omitted in pipe notation] |
[in] | urange | The range being processed. [parameter is omitted in pipe notation] |
[in] | size | The target size of the view. |
size
elements of the underlying range. Header File
#include <seqan3/io/views/detail/take_exactly_view.hpp>
Concepts and traits | urng_t (underlying range type) | rrng_t (returned range type) |
---|---|---|
std::ranges::input_range | required | preserved |
std::ranges::forward_range | preserved | |
std::ranges::bidirectional_range | preserved | |
std::ranges::random_access_range | preserved | |
std::ranges::contiguous_range | preserved | |
std::ranges::viewable_range | required | guaranteed |
std::ranges::view | guaranteed | |
std::ranges::sized_range | guaranteed | |
std::ranges::common_range | preserved | |
std::ranges::output_range | preserved except if urng_t is std::basic_string | |
seqan3::const_iterable_range | preserved | |
std::ranges::range_reference_t | std::ranges::range_reference_t<urng_t> |
See the views submodule documentation for detailed descriptions of the view properties.
The difference to seqan3::views::take is that this view always exposes size information – even if the underlying range is not sized. You should only use this if you know that the underlying range will always be at least size
long.
For seqan3::detail::take_exactly if the underlying range is shorter than size
, the behaviour is undefined. seqan3::detail::take_exactly_or_throw is a safer alternative, because it throws an exception when an iterator before the size
-th one compares equal to the end sentinel; and it also throws on construction if it knows that the underlying range is smaller.
|
inlineconstexpr |
A view adaptor that returns a single line from the underlying range or the full range if there is no newline.
urng_t | The type of the range being processed. See below for requirements. [template parameter is omitted in pipe notation] |
[in] | urange | The range being processed. [parameter is omitted in pipe notation] |
\n
or \r\n
). See below for the properties of the returned range. Header File
#include <seqan3/io/views/detail/take_line_view.hpp>
This adaptor returns a single line excluding the end-line character(s), but moving the cursor behind them for single-pass ranges. I.e. for all ranges that satisfy std::ranges::forward_range this is the same as calling std::views::take_while:
but for single pass input ranges this means that any endline characters after the returned range are also consumed (this potentially includes multiple newline characters).
Concepts and traits | urng_t (underlying range type) | rrng_t (returned range type) |
---|---|---|
std::ranges::input_range | required | preserved |
std::ranges::forward_range | preserved | |
std::ranges::bidirectional_range | preserved | |
std::ranges::random_access_range | preserved | |
std::ranges::contiguous_range | preserved | |
std::ranges::viewable_range | required | guaranteed |
std::ranges::view | guaranteed | |
std::ranges::sized_range | lost | |
std::ranges::common_range | lost | |
std::ranges::output_range | preserved | |
seqan3::const_iterable_range | preserved | |
std::ranges::range_reference_t | std::common_reference_with<char> | std::ranges::range_reference_t<urng_t> |
See the views submodule documentation for detailed descriptions of the view properties.
Behaviour on std::ranges::forward_range:
On single pass std::ranges::input_range it can be used to tokenise the input stream line-wise:
|
inlineconstexpr |
A view adaptor that returns a single line from the underlying range (throws if there is no end-of-line).
seqan3::unexpected_end_of_input | If the underlying range contains no end-of-line marker. |
urng_t | The type of the range being processed. See below for requirements. [template parameter is omitted in pipe notation] |
[in] | urange | The range being processed. [parameter is omitted in pipe notation] |
\n
or \r\n
). See below for the properties of the returned range. Header File
#include <seqan3/io/views/detail/take_line_view.hpp>
This adaptor returns a single line excluding the end-line character(s), but moving the cursor behind them for single-pass ranges. I.e. for all ranges that satisfy std::ranges::forward_range this is the same as calling std::views::take_while:
but for single pass input ranges this means that any endline characters after the returned range are also consumed (this potentially includes multiple newline characters).
Concepts and traits | urng_t (underlying range type) | rrng_t (returned range type) |
---|---|---|
std::ranges::input_range | required | preserved |
std::ranges::forward_range | preserved | |
std::ranges::bidirectional_range | preserved | |
std::ranges::random_access_range | preserved | |
std::ranges::contiguous_range | preserved | |
std::ranges::viewable_range | required | guaranteed |
std::ranges::view | guaranteed | |
std::ranges::sized_range | lost | |
std::ranges::common_range | lost | |
std::ranges::output_range | preserved | |
seqan3::const_iterable_range | preserved | |
std::ranges::range_reference_t | std::common_reference_with<char> | std::ranges::range_reference_t<urng_t> |
See the views submodule documentation for detailed descriptions of the view properties.
Behaviour on std::ranges::forward_range:
On single pass std::ranges::input_range it can be used to tokenise the input stream line-wise:
|
inlineconstexpr |
A view adaptor that returns elements from the underlying range until the functor evaluates to true (or the end of the underlying range is reached).
urng_t | The type of the range being processed. See below for requirements. [template parameter is omitted in pipe notation] |
fun_t | The type of the functor; must model std::invocable with std::ranges::range_reference_t<urng_t> and return a type convertible to bool . |
[in] | urange | The range being processed. [parameter is omitted in pipe notation] |
[in] | fun | The functor. |
Header File
#include <seqan3/io/views/detail/take_until_view.hpp>
Concepts and traits | urng_t (underlying range type) | rrng_t (returned range type) |
---|---|---|
std::ranges::input_range | required | preserved |
std::ranges::forward_range | preserved¹ | |
std::ranges::bidirectional_range | preserved¹ | |
std::ranges::random_access_range | preserved¹ | |
std::ranges::contiguous_range | preserved¹ | |
std::ranges::viewable_range | required | guaranteed |
std::ranges::view | guaranteed | |
std::ranges::sized_range | lost | |
std::ranges::common_range | lost | |
std::ranges::output_range | preserved | |
seqan3::const_iterable_range | preserved¹ | |
std::ranges::range_reference_t | std::ranges::range_reference_t<urng_t> |
See the views submodule documentation for detailed descriptions of the view properties.
¹ The marked properties are only preserved if the specified functor models std::regular_invocable<fun_t, std::ranges::range_reference_t<urng_t>
, i.e. applying the functor doesn't change the functor. If the functor only models std::invocable
and not std::regular_invocable
these concepts are lost.
Throwing: seqan3::detail::take_until_or_throw
and seqan3::detail::take_until_or_throw_and_consume
throw an exception if the end of the underlying range is reached before their own termination criterium is met. This is useful if you want a "strict" evaluation of the functor.
Consuming: seqan3::detail::take_until_and_consume
and seqan3::detail::take_until_or_throw_and_consume
behave the same as their non-consuming counter-parts if the underlying range models at least std::ranges::forward_range
. If, however, the underlying range is a pure std::ranges::input_range
, the view will keep moving the underlying iterator forward as long as the termination criterium holds and the underlying range is not at end. This is useful for string tokenisation among other things.
|
inlineconstexpr |
A view adaptor that returns elements from the underlying range until the functor evaluates to true (or the end of the underlying range is reached and consumes the end in single-pass ranges).
seqan3::unexpected_end_of_input | If the underlying range contains no element that satisfies the functor. |
urng_t | The type of the range being processed. See below for requirements. [template parameter is omitted in pipe notation] |
fun_t | The type of the functor; must model std::invocable with std::ranges::range_reference_t<urng_t> and return a type convertible to bool . |
[in] | urange | The range being processed. [parameter is omitted in pipe notation] |
[in] | fun | The functor. |
Header File
#include <seqan3/io/views/detail/take_until_view.hpp>
Concepts and traits | urng_t (underlying range type) | rrng_t (returned range type) |
---|---|---|
std::ranges::input_range | required | preserved |
std::ranges::forward_range | preserved¹ | |
std::ranges::bidirectional_range | preserved¹ | |
std::ranges::random_access_range | preserved¹ | |
std::ranges::contiguous_range | preserved¹ | |
std::ranges::viewable_range | required | guaranteed |
std::ranges::view | guaranteed | |
std::ranges::sized_range | lost | |
std::ranges::common_range | lost | |
std::ranges::output_range | preserved | |
seqan3::const_iterable_range | preserved¹ | |
std::ranges::range_reference_t | std::ranges::range_reference_t<urng_t> |
See the views submodule documentation for detailed descriptions of the view properties.
¹ The marked properties are only preserved if the specified functor models std::regular_invocable<fun_t, std::ranges::range_reference_t<urng_t>
, i.e. applying the functor doesn't change the functor. If the functor only models std::invocable
and not std::regular_invocable
these concepts are lost.
Throwing: seqan3::detail::take_until_or_throw
and seqan3::detail::take_until_or_throw_and_consume
throw an exception if the end of the underlying range is reached before their own termination criterium is met. This is useful if you want a "strict" evaluation of the functor.
Consuming: seqan3::detail::take_until_and_consume
and seqan3::detail::take_until_or_throw_and_consume
behave the same as their non-consuming counter-parts if the underlying range models at least std::ranges::forward_range
. If, however, the underlying range is a pure std::ranges::input_range
, the view will keep moving the underlying iterator forward as long as the termination criterium holds and the underlying range is not at end. This is useful for string tokenisation among other things.
|
inlineconstexpr |
A view adaptor that returns elements from the underlying range until the functor evaluates to true (throws if the end of the underlying range is reached).
seqan3::unexpected_end_of_input | If the underlying range contains no element that satisfies the functor. |
urng_t | The type of the range being processed. See below for requirements. [template parameter is omitted in pipe notation] |
fun_t | The type of the functor; must model std::invocable with std::ranges::range_reference_t<urng_t> and return a type convertible to bool . |
[in] | urange | The range being processed. [parameter is omitted in pipe notation] |
[in] | fun | The functor. |
Header File
#include <seqan3/io/views/detail/take_until_view.hpp>
Concepts and traits | urng_t (underlying range type) | rrng_t (returned range type) |
---|---|---|
std::ranges::input_range | required | preserved |
std::ranges::forward_range | preserved¹ | |
std::ranges::bidirectional_range | preserved¹ | |
std::ranges::random_access_range | preserved¹ | |
std::ranges::contiguous_range | preserved¹ | |
std::ranges::viewable_range | required | guaranteed |
std::ranges::view | guaranteed | |
std::ranges::sized_range | lost | |
std::ranges::common_range | lost | |
std::ranges::output_range | preserved | |
seqan3::const_iterable_range | preserved¹ | |
std::ranges::range_reference_t | std::ranges::range_reference_t<urng_t> |
See the views submodule documentation for detailed descriptions of the view properties.
¹ The marked properties are only preserved if the specified functor models std::regular_invocable<fun_t, std::ranges::range_reference_t<urng_t>
, i.e. applying the functor doesn't change the functor. If the functor only models std::invocable
and not std::regular_invocable
these concepts are lost.
Throwing: seqan3::detail::take_until_or_throw
and seqan3::detail::take_until_or_throw_and_consume
throw an exception if the end of the underlying range is reached before their own termination criterium is met. This is useful if you want a "strict" evaluation of the functor.
Consuming: seqan3::detail::take_until_and_consume
and seqan3::detail::take_until_or_throw_and_consume
behave the same as their non-consuming counter-parts if the underlying range models at least std::ranges::forward_range
. If, however, the underlying range is a pure std::ranges::input_range
, the view will keep moving the underlying iterator forward as long as the termination criterium holds and the underlying range is not at end. This is useful for string tokenisation among other things.
|
inlineconstexpr |
A view adaptor that returns elements from the underlying range until the functor evaluates to true (throws if the end of the underlying range is reached and consumes the end in single-pass ranges).
seqan3::unexpected_end_of_input | If the underlying range contains no element that satisfies the functor. |
seqan3::unexpected_end_of_input | If the underlying range contains no element that satisfies the functor. |
urng_t | The type of the range being processed. See below for requirements. [template parameter is omitted in pipe notation] |
fun_t | The type of the functor; must model std::invocable with std::ranges::range_reference_t<urng_t> and return a type convertible to bool . |
[in] | urange | The range being processed. [parameter is omitted in pipe notation] |
[in] | fun | The functor. |
Header File
#include <seqan3/io/views/detail/take_until_view.hpp>
Concepts and traits | urng_t (underlying range type) | rrng_t (returned range type) |
---|---|---|
std::ranges::input_range | required | preserved |
std::ranges::forward_range | preserved¹ | |
std::ranges::bidirectional_range | preserved¹ | |
std::ranges::random_access_range | preserved¹ | |
std::ranges::contiguous_range | preserved¹ | |
std::ranges::viewable_range | required | guaranteed |
std::ranges::view | guaranteed | |
std::ranges::sized_range | lost | |
std::ranges::common_range | lost | |
std::ranges::output_range | preserved | |
seqan3::const_iterable_range | preserved¹ | |
std::ranges::range_reference_t | std::ranges::range_reference_t<urng_t> |
See the views submodule documentation for detailed descriptions of the view properties.
¹ The marked properties are only preserved if the specified functor models std::regular_invocable<fun_t, std::ranges::range_reference_t<urng_t>
, i.e. applying the functor doesn't change the functor. If the functor only models std::invocable
and not std::regular_invocable
these concepts are lost.
Throwing: seqan3::detail::take_until_or_throw
and seqan3::detail::take_until_or_throw_and_consume
throw an exception if the end of the underlying range is reached before their own termination criterium is met. This is useful if you want a "strict" evaluation of the functor.
Consuming: seqan3::detail::take_until_and_consume
and seqan3::detail::take_until_or_throw_and_consume
behave the same as their non-consuming counter-parts if the underlying range models at least std::ranges::forward_range
. If, however, the underlying range is a pure std::ranges::input_range
, the view will keep moving the underlying iterator forward as long as the termination criterium holds and the underlying range is not at end. This is useful for string tokenisation among other things.