Update a metric for the currently running pipeline

Records or overwrites a named metric for the pipeline currently being executed by run_schedule(). Call this from within a pipeline function's body - maestro automatically attributes the metric to the pipeline and run that is currently executing, so there's no need to pass the pipe name yourself.

Usage

update_metric(name, value, increment = FALSE)

Arguments

name

character scalar; the name of the metric to set (e.g. "rows_in").

value

numeric scalar to record for name.

increment

logical; if TRUE, value is added to the metric's current value (within the current pipeline run) instead of overwriting it. A negative value will therefore decrement it. Default FALSE.

Calling update_metric() outside of a pipeline run (e.g. while developing or testing a pipeline function interactively) is not an error - it emits an informational message and has no effect.

Value

invisible NULL

Details

Calling update_metric() again with the same name during the same pipeline run overwrites the previous value (last write wins), unless increment = TRUE, in which case value is added to the existing value instead (accumulating across calls). This works even if the metric hasn't been set yet in the current run - the first call just sets it, same as a regular update. Metrics from prior runs of the same pipeline are preserved and returned separately by get_metrics().

Examples

if (interactive()) {
  # Inside a pipeline function:
  my_pipeline <- function() {
    data <- get_data()
    update_metric("rows_in", nrow(data))

    out <- process(data)
    update_metric("rows_out", nrow(out))

    # Accumulate a count across a loop
    for (chunk in split_into_chunks(data)) {
      process_chunk(chunk)
      update_metric("chunks_processed", 1, increment = TRUE)
    }

    out
  }
}