Skip to contents

Managing labelled audio data in R with SonicScrewdriveR

Setting annotation boundaries

SonicScrewdriveR provides a set of functionality to read, generate, manage and write label data for audio files. This Annotation class is used to consistently handle annotations to audio files. Annotations are delimited in the time domain (seconds) and frequency domain (Hz).

a1 <- annotation(
  start = 0,
  end = 1
)

a2 <- annotation(
  low = 0,
  high = 1000
)

Inf can be used as a value to end or high to indicate that the annotation extends to the limits of the file.

a3 <- annotation(
  start = 0,
  end = Inf
)

a4 <- annotation(
  low = 0,
  high = Inf
)

Handling metadata

Objects of the Annotation class have inbuilt character slots for storing the file name (file) and the source, type and value of the annotation. A list slot metadata is provided for storing additional metadata.

Who made an annotation and when are held in the identifiedBy and dateIdentified slots, as Audiovisual Core gives them for a region of interest (dwc:identifiedBy and dwc:dateIdentified). identifiedBy is a person, or the software and model that made the annotation, while source is where the annotation came from. dateIdentified is ISO 8601 text, and a Date or a time given to annotation() is written as such, a time in UTC.

The file, source, type, value and identifiedBy slots are used by SonicScrewdriveR. For example, the merge_annotations function uses these slots to appropriately merge overlapping annotations, so that the annotations of two people are not merged into one unless asked.

When using tools from SonicScrewdriveR these slots may be pre-populated. The example below creates an Annotation object equivalent to one generated by the function birdNetAnalyse.

a5 <- annotation(
  start=27,
  end=30,
  file = "example.wav",
  source = "BirdNet-Analyzer",
  type = "birdnet-detection",
  value = "sparrow",
  identifiedBy = "BirdNET-Analyzer 2.4",
  dateIdentified = as.POSIXct("2026-10-30 09:15:00", tz="UTC"),
  metadata = list(
    confidence = 0.9
  )
)

Reading annotations

From audioblast database

# Fetch annotations from audioBlast as `Annotation` objects
a <- audioblast("data", "annomate", source="bio.acousti.ca", source_id=10754, output="Annotation")

From the cue points of a wave file

Audio editors save the markers and labelled regions someone has put on a recording as cue points in the wave file. wavCues reads them as Annotation objects: a labelled region runs from its position for its length, and a marker starts and ends where it is.

# Read the markers and labelled regions saved in a wave file
a <- wavCues("marked.wav")

From an AOEF file

The Whombat annotation tool exports its projects in the Acoustic Objects Exchange Format (AOEF), the JSON format of the soundevent Python package. readAOEF reads them as Annotation objects, one for each tag of each sound event. The paths of the recordings in the file are relative to the directory the audio is in, which is given as audio.dir.

# Read the annotations of a project exported from Whombat
a <- readAOEF("project.json", audio.dir="audio")

Generating annotations

Using BirdNET

# Analyse sound files using BirdNET-Analyzer
f <- system.file("extdata", "AUDIOMOTH.WAV", package="sonicscrewdriver")
annotations <- birdNetAnalyse(f, output="Annotation")

Merging annotations

The merge_annotations function can be used to merge overlapping annotations. The function will merge annotations with overlapping time extents with the same file, type, value and optionally source and identifiedBy slots.

a <- list(
  annotation(
    start=27,
    end=30,
    file = "example.wav",
    source = "BirdNet-Analyzer",
    type = "birdnet-detection",
    value = "sparrow",
    metadata = list(
      confidence = 0.9
    )
  ),
  annotation(
    start=30,
    end=33,
    file = "example.wav",
    source = "BirdNet-Analyzer",
    type = "birdnet-detection",
    value = "sparrow",
    metadata = list(
      confidence = 0.8
    )
  )
)

merge_annotations(a)
#> [[1]]
#> An object of class "Annotation"
#> Slot "file":
#> [1] "example.wav"
#> 
#> Slot "metadata":
#> $confidence
#> [1] 0.8
#> 
#> 
#> Slot "start":
#> [1] 27
#> 
#> Slot "end":
#> [1] 33
#> 
#> Slot "low":
#> [1] 0
#> 
#> Slot "high":
#> [1] Inf
#> 
#> Slot "source":
#> [1] "BirdNet-Analyzer"
#> 
#> Slot "type":
#> [1] "birdnet-detection"
#> 
#> Slot "value":
#> [1] "sparrow"
#> 
#> Slot "identifiedBy":
#> [1] ""
#> 
#> Slot "dateIdentified":
#> [1] ""

The two annotations in the list a overlap in time and have the same file, type and value slots. The merge_annotations function has merged these into a single annotation with the time extent from 27 to 33 seconds.

Writing annotations

Audacity label file

# Convert the annotations to an Audacity label file
writeAudacityLabels(annotations, "birdnet_annotations.txt")

AOEF file

writeAOEF writes annotations to an AOEF file, which soundevent reads, and which Whombat imports as an annotation project. Annotations read by readAOEF are written back as they were read.

# Write the annotations as an annotation project
writeAOEF(annotations, "birdnet_annotations.json", audio.dir="audio", collection="annotation_project")