A Pipeline is an analysis written down as steps, to be run on every
recording in a RecordingSet. Each step is written as an
ordinary call to a function with the wave left out, and the wave is put in as
its first argument when the step runs, as tagApply() does.
Arguments
- ...
For
pipeline(), the steps: calls written with the wave left out, pipelines, or lists of pipelines. Fortask(), the arguments of.funother than the wave.- .fun
A function, or the name of one, taking the wave as its first argument.
- then
Optional. A pipeline, or a single step, to give the wave the step gives back, as described in Details.
Writing a pipeline
pipeline() keeps each call as it was written, without running it, so that
printing a pipeline shows what was typed. The arguments of each call are
evaluated once, when the pipeline is made: bandpass(2000, 11000) becomes
its WaveFilter then, so a pipeline is fixed and can be identified, and a
variable changed afterwards does not change it.
A pipeline given as a step is run in its place, so pipeline(p1, p2) runs
the steps of both in turn, and is the same pipeline, with the same id, as one
written out with all of their steps. A pipeline is therefore a step, and a
step on its own is a pipeline of one: either can go wherever the other can.
task() makes the same step as a call written in pipeline() would, as a
value, so that a pipeline can be built in code. A call written inside
lapply() would run there and then, where task() keeps it to run later:
pipeline(lapply(c(1000, 4000), function(f) task(filterWave, bandpass(f, 2 * f)))). Its function is named .fun so that an argument
called f, as seewave calls a sampling frequency, reaches the function
rather than being taken for it, as with tagApply().
An argument of pipeline() that is a call is a step, unless it is a call to
pipeline() or task(), or to one of the functions of base R that build a
list, such as list() and lapply(), which take no wave. A pipeline held in
a variable, or a list of them, is given by its name, and NULL gives no
steps.
A pipeline does not carry the recordings it is run on. They are given to
runPipeline() when it is run, so that the same pipeline can be run on any
number of sets, and its id is the same whichever set it runs on.
How waves and results pass between steps
What a step does to the pipeline depends on what it gives back, which is known only once it has run:
- A wave
The steps after it get the new wave, so a chain of transformations stays flat:
pipeline(normalise(), dutyCycle())measures the normalised wave.- Several waves
An error, as there is no single wave to carry on with, unless the step is a windowing given
then=, as below.- Anything else
The step is an analysis, and the wave it was given goes on to the next step as it was.
then= gives a step a pipeline, or a single step, of its own. The wave the
step gives goes to that pipeline, and the steps after it get the wave as it
was before it, so filterWave(bandpass(2000, 11000), then = dutyCycle())
measures the band and leaves the rest of the pipeline the whole recording.
then= is taken out of the call before the function is called, so a function
with an argument of its own called then cannot be given it through a
pipeline. A step given then= that turns out not to give a wave is an error,
raised when it runs, and so is one that gives several.
windowing() is the step that gives several waves, and takes its then=
itself as its FUN: windowing(60, time.unit = "seconds", then = pipeline(dutyCycle(), seewave::M())) runs both steps on every minute of the
recording, and their tables have a row for each window, keyed by where it
starts and how long it is. A windowing in a pipeline needs then=, and
cannot be given both then= and FUN.
A step of cutws() or seewave's cutw() is a cut, and is placed in the
recording. Every row measured on what it cuts, by the steps of its then=
or by the steps after it, has cut_start and cut_end, in seconds from the
start of the recording; a cut of a cut gives the span that was measured. A
window inside a cut is counted from the start of the recording too, so
cutws(160001, 240000, then = windowing(8000, then = dutyCycle())) on audio
sampled at 8 kHz keys its rows with a cut from 20 to 30 seconds and windows
starting at 20, 21, 22 seconds and so on. A function that cuts in some other
way, one of the user's among
them, is not recognised, and what it gives is taken to start where the wave
it was given did.
The results of a pipeline are a table for each analysis step, its result
turned into rows by flattenOutput(), named by the step's path through the
pipeline: its own call after the calls of the steps whose then= it is in,
joined by a slash, as filterWave(bandpass(2000, 11000)) / dutyCycle(). Two
steps can have the same path, as the same analysis before and after a
transformation do, and each is then followed by its number in the outline,
as dutyCycle() [3].
A function of this package that records itself, as filterWave() and
normalise() do, leaves the record it makes when called directly in the
history of the wave, and any other step is recorded as tagApply() records
it. For an analysis, Pipeline puts the result in the output of its record,
sets the record's output_hash to its input_hash, as the wave carried on is
the wave that went in, and adds the record to the history of the wave, so
that every result traces back to what was done to the wave before it.
Identity
A pipeline is identified by a hash of its steps in order, so two pipelines
that would compute the same thing share an id, and changing anything that
could change a number gives a new one. A step's part of it covers its
function, its evaluated arguments, named as the function names them so that
one given by position counts the same as one given by name, the defaults it
was not given, and the code of every package it uses. A step with then=
adds the id of its own pipeline, which is the same whether that pipeline is
run on its own or inside another.
Functions are identified by their code, never as R objects. A function object changes when R byte-compiles it, and carries its environment and the source it was read from, so it would give the same pipeline a new id in every session. A function from a package is identified by its package and name, with the code of that package. Any other function, written by the user or inline, is identified by its own code, so a change to another function it calls is not seen, and nor are values a function made by another function holds from where it was made: give those as arguments instead.
The packages a step uses are found from its code. From its function, and any
function among its arguments such as the one in a WaveFilter, calls are
followed through the functions of this package and of the user, and every
other package they reach is counted, though calls within it are not followed.
This package is always counted, as its own code runs around every step, and
scikit-maad is counted wherever a step reaches getMaad(). A package is
hashed from the deparsed code of every function in its namespace, and
scikit-maad from its installed source files, so any change to a package a
step uses gives it a new id, even a change to a function the step never
calls, and so does a version that was never released. Packages that come
with R are covered by R's version, which is part of every id. As the packages
are found from the code rather than from what ran, a package reached only on
a path a step does not take, such as one used to install another, still
counts, and one that is not installed is counted as not installed.
Code is hashed each time an id is worked out rather than remembered, so a package changed during a session is seen the next time one is. An argument holding an environment, such as a Python module, cannot be identified, and gives an error.
Printing
Printing a pipeline gives the start of its id, its steps as an outline numbered so that a table of results can be found in it, each call as it was written, and a last line of the code it was identified by: the start of the hash of each package, and R's version.
See also
runPipeline() to run one over a set of recordings, tagApply(),
whose records the steps leave, and flattenOutput(), which turns their
results into rows.
Examples
p <- pipeline(
normalise(),
filterWave(bandpass(2000, 11000), then = dutyCycle()),
dutyCycle()
)
p
#> <Pipeline 95f8a1e6: 4 steps>
#> 1. normalise()
#> 2. filterWave(bandpass(2000, 11000)), then
#> 2.1 dutyCycle()
#> 3. dutyCycle()
#> Code: sonicscrewdriver 8ffbd680, digest f0d5de56, seewave 76dcf15f, tuneR 3763cca6, R 4.6.2 (2026-10-29)
# Two bands, built in code
pipeline(lapply(c(1000, 4000), function(f) {
task(filterWave, bandpass(f, 2 * f), then = dutyCycle())
}))
#> <Pipeline fae0e659: 4 steps>
#> 1. filterWave(bandpass(1000, 2 * 1000)), then
#> 1.1 dutyCycle()
#> 2. filterWave(bandpass(4000, 2 * 4000)), then
#> 2.1 dutyCycle()
#> Code: sonicscrewdriver 8ffbd680, digest f0d5de56, seewave 76dcf15f, tuneR 3763cca6, R 4.6.2 (2026-10-29)
