Title: | Archetypes for Targets and Fish Telemetry |
---|---|
Description: | What the package does (one paragraph). |
Authors: | Michael O'Brien [aut, cre] |
Maintainer: | Michael O'Brien <[email protected]> |
License: | MIT + file LICENSE |
Version: | 0.0.0.9000 |
Built: | 2024-11-13 02:58:26 UTC |
Source: | https://github.com/mhpob/telemetar |
Dynamic branching over VDAT files
tar_vdat_read( name, vdat_dirs, csv_outdir, batch_size = 10, batches = NULL, format = c("file", "file_fast", "url", "aws_file"), repository = targets::tar_option_get("repository"), iteration = targets::tar_option_get("iteration"), error = targets::tar_option_get("error"), memory = targets::tar_option_get("memory"), garbage_collection = targets::tar_option_get("garbage_collection"), priority = targets::tar_option_get("priority"), resources = targets::tar_option_get("resources"), cue = targets::tar_option_get("cue") )
tar_vdat_read( name, vdat_dirs, csv_outdir, batch_size = 10, batches = NULL, format = c("file", "file_fast", "url", "aws_file"), repository = targets::tar_option_get("repository"), iteration = targets::tar_option_get("iteration"), error = targets::tar_option_get("error"), memory = targets::tar_option_get("memory"), garbage_collection = targets::tar_option_get("garbage_collection"), priority = targets::tar_option_get("priority"), resources = targets::tar_option_get("resources"), cue = targets::tar_option_get("cue") )
name |
Symbol, name of the target. A target
name must be a valid name for a symbol in R, and it
must not start with a dot. Subsequent targets
can refer to this name symbolically to induce a dependency relationship:
e.g. |
vdat_dirs |
Nonempty character vector of known existing directories of VDAT files to track for changes. |
csv_outdir |
file path to the output directory |
batch_size |
Positive integer of length 1, number of files to partition into a batch. The default is ten files per batch. |
batches |
Positive integer of length 1, number of batches to partition the files. The default is one file per batch (maximum number of batches) which is simplest to handle but could cause a lot of overhead and consume a lot of computing resources. Consider reducing the number of batches below the number of files for heavy workloads. |
format |
Character, either |
repository |
Character of length 1, remote repository for target storage. Choices:
Note: if |
iteration |
Character, iteration method. Must be a method
supported by the |
error |
Character of length 1, what to do if the target stops and throws an error. Options:
|
memory |
Character of length 1, memory strategy.
If |
garbage_collection |
Logical, whether to run |
priority |
Numeric of length 1 between 0 and 1. Controls which
targets get deployed first when multiple competing targets are ready
simultaneously. Targets with priorities closer to 1 get dispatched earlier
(and polled earlier in |
resources |
Object returned by |
cue |
An optional object from |
# example code
# example code
Dynamic branching over VUE/VDAT-exported CSV detection files.
tar_vue_csvs( name, csv_dirs, pattern = "^[VH]R.*\\.csv$", batch_size = 10, format = c("file", "file_fast", "url", "aws_file"), repository = targets::tar_option_get("repository"), iteration = targets::tar_option_get("iteration"), error = targets::tar_option_get("error"), memory = targets::tar_option_get("memory"), garbage_collection = targets::tar_option_get("garbage_collection"), priority = targets::tar_option_get("priority"), resources = targets::tar_option_get("resources"), cue = targets::tar_option_get("cue") )
tar_vue_csvs( name, csv_dirs, pattern = "^[VH]R.*\\.csv$", batch_size = 10, format = c("file", "file_fast", "url", "aws_file"), repository = targets::tar_option_get("repository"), iteration = targets::tar_option_get("iteration"), error = targets::tar_option_get("error"), memory = targets::tar_option_get("memory"), garbage_collection = targets::tar_option_get("garbage_collection"), priority = targets::tar_option_get("priority"), resources = targets::tar_option_get("resources"), cue = targets::tar_option_get("cue") )
name |
Symbol, name of the target. A target
name must be a valid name for a symbol in R, and it
must not start with a dot. Subsequent targets
can refer to this name symbolically to induce a dependency relationship:
e.g. |
csv_dirs |
Nonempty character vector of known existing directories of CSV files to track for changes. |
pattern |
a regular expression to search for the applicable CSV files.
Defaults to " |
batch_size |
Positive integer of length 1, number of files to partition into a batch. The default is ten files per batch. |
format |
Character, either |
repository |
Character of length 1, remote repository for target storage. Choices:
Note: if |
iteration |
Character, iteration method. Must be a method
supported by the |
error |
Character of length 1, what to do if the target stops and throws an error. Options:
|
memory |
Character of length 1, memory strategy.
If |
garbage_collection |
Logical, whether to run |
priority |
Numeric of length 1 between 0 and 1. Controls which
targets get deployed first when multiple competing targets are ready
simultaneously. Targets with priorities closer to 1 get dispatched earlier
(and polled earlier in |
resources |
Object returned by |
cue |
An optional object from |
targets::tar_dir({ ## Download example data download.file( file.path('https://raw.githubusercontent.com/ocean-tracking-network/glatos', 'main/inst/extdata/VR2W_109924_20110718_1.csv'), 'VR2W_109924_20110718_1.csv' ) for(i in 2:12){ file.copy( 'VR2W_109924_20110718_1.csv', paste0('VR2W_109924_20110718_', i, '.csv') ) } ## Run workflow targets::tar_script({ list( telemetar::tar_vue_csvs( my_detections, getwd() ) ) }) targets::tar_make(callr_function = NULL) })
targets::tar_dir({ ## Download example data download.file( file.path('https://raw.githubusercontent.com/ocean-tracking-network/glatos', 'main/inst/extdata/VR2W_109924_20110718_1.csv'), 'VR2W_109924_20110718_1.csv' ) for(i in 2:12){ file.copy( 'VR2W_109924_20110718_1.csv', paste0('VR2W_109924_20110718_', i, '.csv') ) } ## Run workflow targets::tar_script({ list( telemetar::tar_vue_csvs( my_detections, getwd() ) ) }) targets::tar_make(callr_function = NULL) })