Maestro tags are roxygen2 comments for configuring the scheduling and execution of pipelines.
Maestro tags follow the format: #' @maestroTagName value
Some tags may not take a value.
Maestro tags must be written above the function that is to be included as a pipeline. A typical pipeline with tags could look like this:
#' @maestroFrequency 1 hour
#' @maestroStartTime 12:30:00
#' @maestroLogLevel WARN
my_pipeline <- function() {
# Pipeline code
}Below are descriptions of all the tags currently available in maestro along with examples.
Basic Tags
maestroFrequency
How often to run the pipeline. This tag takes a time unit indicating how long to wait between subsequent runs of the pipeline. Acceptable values include an integer value followed by one of minute(s), hour(s), day(s), week(s), month(s), and year(s). Note that some combinations of integer + unit may be invalid. Adverbs like ‘hourly’, ‘daily’, ‘weekly’, etc. are also valid.
Default: 1 day
Examples:
#' @maestroFrequency 1 hour#' @maestroFrequency daily#' @maestroFrequency 2 weeks#' @maestroFrequency 3 months
maestroStartTime
Timestamp, date, or time corresponding to the start time of the pipeline. This also sets the cadence of the pipeline in some cases. For instance, if the start time is 2025-03-18 03:00:00 and the frequency is daily, the pipeline will run at 03:00 every day. A value in the future prevents the pipeline from running until that time has been reached.
Default: 2024-01-01 00:00:00
Examples:
#' @maestroStartTime 2025-02-05 12:00:00#' @maestroStartTime 2025-01-01#' @maestroStartTime 08:00:00
maestroTz
Timezone in which the maestroStartTime is to be considered. Takes any valid timezone string that can be found in OlsonNames().
Default: UTC
Examples:
#' @maestroTz Europe/Paris#' @maestroTz America/Halifax#' @maestroTz US/Pacific
maestroLogLevel
Minimum logging threshold for messages, warnings, and errors that come from the pipeline. These levels correspond to those in logger:::log_levels_supported but most commonly used are ERROR, WARN, INFO. For example, if you use WARN then any messages of lower urgency (i.e., INFO) will be suppressed, but errors will be logged.
Default: INFO
Examples:
#' @maestroLogLevel ERROR#' @maestroLogLevel WARN
Specifier Tags
maestroHours
Specific hours of the day in which to run the pipeline. This only applies for pipelines that run at an hourly or minutely frequency. Acceptable values are integers from 0-23 separated by spaces. If empty, pipeline runs at all hours. This tag uses the timezone specified by maestroTz (will be UTC if empty).
Default:
Examples:
#' @maestroHours 1 4 7 10#' @maestroHours 0 5 20
maestroDays
Specific days of week or days of month on which to run the pipeline. This only applies for pipelines that run at a minutely, hourly, or daily frequency. Acceptable values are either integers from 1-31 or day of week strings like Mon, Tue, Wed, etc. These two options cannot be used in combination.
Default:
Examples:
#' @maestroDays 1 11 21 31#' @maestroDays Mon Tue Wed Thu Fri
maestroMonths
Specific months of the year on which to run the pipeline. This only applies for pipelines that do run at least monthly. Acceptable values are integers (1-12) corresponding to the month of the year (e.g., 1 = January, 2 = February, etc.).
Default:
Examples:
#' @maestroMonths 3 8 12#' @maestroMonths 10
DAG Tags
maestroInputs
For a DAG style pipeline, the names of pipelines that input into the pipeline. These names must match the function name defining the inputting pipeline. Multiple pipelines can be used as inputs and the input value is used in the target pipeline via the required .input parameter. Note that this tag could be redundant if the inputting pipeline uses maestroOutputs.
Default:
Examples:
#' @maestroInputs extract verify
maestroOutputs
For a DAG style pipeline, the names of pipelines that receive the return value of this pipeline as input. These names must match the function name defining the outputting pipeline. Multiple pipelines can be outputted into. The return value of the pipeline will be passed into the receiving pipeline. Note that this tag could be redundant if pipeline to be inputted into uses maestroInputs.
Default:
Examples:
#' @maestroOutputs transform
Advanced Tags
maestroSkip
Flags a pipeline to never be executed even if it is scheduled to run. This can be useful when developing or testing a pipeline. This tag takes no value, instead the presence of the tag indicates whether it is skipped. This tag is ignored when run_schedule(..., run_all = TRUE) or when using invoke().
Default:
Examples:
#' @maestroSkip
maestroPriority
Determines the order in which pipelines that run at the same scheduled instance are executed. Values are positive integers from 1 to N. Order is determined in descending order such that 1 indicates the highest priority. Pipelines with the same priority run in the order in which build_schedule() parses the pipeline (usually alphabetical according to file name and then line number within file). By default, all pipelines are given equal priority.
Default:
Examples:
#' @maestroPriority 1#' @maestroPriority 3
maestro
Generic tag for identifying a maestro pipeline with all the defaults. Useful when you want a pipeline to be scheduled via maestro that accepts all default tag values. Only use this tag if you have no other maestro tags. The tag takes no value.
Default:
Examples:
#' @maestro