Introduction

library(matos)

List available files

First, list_projects returns the project page, which is useful to figure out what URLs are associated with each project. You do not need MATOS permissions in order to view this page.

all_projects <- list_projects()

head(all_projects)
#>                                   name number
#> 1                            ACK Array    168
#> 2  APG Atlantic and Shortnose Sturgeon    176
#> 3 ASI - White Shark Study, Montauk, NY    211
#> 4                   ASI Acoustic Array    100
#> 5              ASI Spinner Shark Study    227
#> 6   ASI White Shark Study, Southern NE    232
#>                                               url
#> 1 https://matos.asascience.com/project/detail/168
#> 2 https://matos.asascience.com/project/detail/176
#> 3 https://matos.asascience.com/project/detail/211
#> 4 https://matos.asascience.com/project/detail/100
#> 5 https://matos.asascience.com/project/detail/227
#> 6 https://matos.asascience.com/project/detail/232

I can also view the files that I’ve uploaded to my projects using list_project_files, but that requires logging in first. The family of list_ functions in this package will prompt you to log in before moving on. Note that I’ll be entering my MATOS username and password behind the scenes here.

project_files <- list_project_files(project = 'umces boem offshore wind energy')
#> ! Please log in.
#> ✔ Login successful!

head(project_files)
#>   project                                file_type upload_date
#> 1      87 Deployed Receivers – Deployment Metadata  2020-03-30
#> 2      87               Tag Detections - .vfl file  2020-05-28
#> 3      87               Tag Detections - .vfl file  2020-05-28
#> 4      87               Tag Detections - .vfl file  2020-05-28
#> 5      87               Tag Detections - .vfl file  2020-05-28
#> 6      87               Tag Detections - .vfl file  2020-05-28
#>                      file_name
#> 1 BOEM_metadata_deployment.xls
#> 2  VR2AR_546455_20170328_1.vrl
#> 3  VR2AR_546456_20170328_1.vrl
#> 4  VR2AR_546457_20170329_1.vrl
#> 5  VR2AR_546458_20170329_1.vrl
#> 6  VR2AR_546459_20170328_1.vrl
#>                                                      url
#> 1  https://matos.asascience.com/projectfile/download/375
#> 2 https://matos.asascience.com/projectfile/download/1810
#> 3 https://matos.asascience.com/projectfile/download/1811
#> 4 https://matos.asascience.com/projectfile/download/1812
#> 5 https://matos.asascience.com/projectfile/download/1813
#> 6 https://matos.asascience.com/projectfile/download/1814

User credentials

A side note on your MATOS username and password: matos defaults to asking you for your login credentials every time you start a new session. To skirt around this you can use set_matos_credentials(), which installs your username and password in your .Renviron file. You will be automatically logged in every time you use your current computer after doing this, but beware: someone else could theoretically access your username and password if they gain access to your computer.

Back to the regularly-scheduled programming

I can also list any of my OTN node Data Extraction Files.

ACT_MATOS_files <- list_extract_files(project = 'umces boem offshore wind energy',
                                      detection_type = 'all')

head(ACT_MATOS_files)
#>   project            file_type detection_type detection_year
#> 1      87 Data Extraction File        matched           2017
#> 2      87 Data Extraction File        matched           2018
#> 3      87 Data Extraction File        matched           2019
#> 4      87 Data Extraction File        matched           2020
#> 5      87 Data Extraction File        matched           2021
#> 6      87 Data Extraction File        matched           2022
#>   upload_date                         file_name
#> 1  2023-07-06 mdwea_matched_detections_2017.zip
#> 2  2023-07-06 mdwea_matched_detections_2018.zip
#> 3  2023-07-06 mdwea_matched_detections_2019.zip
#> 4  2023-07-06 mdwea_matched_detections_2020.zip
#> 5  2023-07-06 mdwea_matched_detections_2021.zip
#> 6  2023-07-06 mdwea_matched_detections_2022.zip
#>                                                                url
#> 1 https://matos.asascience.com/projectfile/downloadExtraction/87_1
#> 2 https://matos.asascience.com/projectfile/downloadExtraction/87_2
#> 3 https://matos.asascience.com/projectfile/downloadExtraction/87_3
#> 4 https://matos.asascience.com/projectfile/downloadExtraction/87_4
#> 5 https://matos.asascience.com/projectfile/downloadExtraction/87_5
#> 6 https://matos.asascience.com/projectfile/downloadExtraction/87_6

Download project or data extraction files

There are a few ways to download the different types of files held by MATOS. I can download directly if I know the URL of the file:

project_files$url[1]
#> [1] "https://matos.asascience.com/projectfile/download/375"

get_project_file(url = project_files$url[1])
#> 
#> ── Downloading files ──────────────────────────────────────────────
#> ✔ File(s) saved to:
#>    C:\Users\darpa2\Analysis\matos\vignettes\BOEM_metadata_deployment.xls
#> 
#> ── Unzipping files ────────────────────────────────────────────────
#> [1] "C:\\Users\\darpa2\\Analysis\\matos\\vignettes\\BOEM_metadata_deployment.xls"

I can download by using an index from the ACT_MATOS_files table above, here the file on the second row.

get_extract_file(file = 2, project = 'umces boem offshore wind energy')
#> 
#> ── Downloading files ──────────────────────────────────────────────
#> ✔ File(s) saved to:
#>    C:\Users\darpa2\Analysis\matos\vignettes\mdwea_matched_detections_2018.zip
#> 
#> ── Unzipping files ────────────────────────────────────────────────
#> ✔ File(s) unzipped to:
#>    C:/Users/darpa2/Analysis/matos/vignettes/mdwea_matched_detections_2018.csv
#>    C:/Users/darpa2/Analysis/matos/vignettes/data_description.txt
#> [1] "C:/Users/darpa2/Analysis/matos/vignettes/mdwea_matched_detections_2018.csv"
#> [2] "C:/Users/darpa2/Analysis/matos/vignettes/data_description.txt"

Search and download tag detections

Using the tag_search function, I can interface with MATOS’ tag search page. Be very careful with this function – it can take a very, VERY long time to return your files. This function downloads the requested CSV into your working directory, and, if import = T is used, reads it into your R session.

my_detections <- tag_search(tags = paste0('A69-1601-254', seq(60, 90, 1)),
                            start_date = '03/01/2016',
                            end_date = '04/01/2016', 
                            import = T)

Upload files to MATOS

There are times when you want to upload new data to MATOS. The currently accepted data types and formats are:

  • newly-deployed transmitters (CSV/XLS(X))
  • detection logs (CSV/VRL)
  • receiver and glider deployment metadata (CSV/XLS(X))
  • receiver events like temperature, tilt, etc. (CSV)
  • glider GPS tracks (CSV)

A few data types use designated Ocean Tracking Network templates:

  • tag metadata
  • receiver deployment metadata
  • glider deployment metadata

If you don’t have one of these templates downloaded, you can download it through the package. For example:

get_otn_template('glider')

Then, get to uploading!

upload_file(project = 'umces boem offshore wind energy',
            file = c('this_is_a_dummy_file.csv', 'so_is_this.csv'),
            data_type = 'new_tags')