Package 's3'

Title: Download Files from 'AWS S3'
Description: Download files hosted on 'AWS S3' (Amazon Web Services Simple Storage Service; <https://aws.amazon.com/s3/>) to a local directory based on their URI. Avoid downloading files that are already present locally. Allow for customization of where to store downloaded files.
Authors: Cole Brokamp [aut, cre] , Erika Rasnick [aut], Robrecht Cannoodt [aut]
Maintainer: Cole Brokamp <[email protected]>
License: MIT + file LICENSE
Version: 1.1.0
Built: 2024-11-03 05:05:57 UTC
Source: https://github.com/brokamp-group/s3

Help Index

download s3 file

Description

Files are downloaded to the R user data directory (i.e., tools::R_user_dir("s3", "data")) so they can be cached across all of an R user's sessions and projects. Specify an alternative download location by setting the R_USER_DATA_DIR environment variable (see ?tools::R_user_dir) or by using the data_dir argument.

Usage

s3_get(
  s3_uri,
  region = "us-east-2",
  quiet = FALSE,
  progress = FALSE,
  force = FALSE,
  public = FALSE,
  data_dir = tools::R_user_dir("s3", "data")
)

Arguments

s3_uri

URI for an S3 object
region

AWS region for bucket containing the file (defaults to "us-east-2", but only required for private files)
quiet

suppress messages?
progress

show download progress? (currently only for public objects)
force

force download to overwrite existing S3 object
public

defaults to FALSE; if TRUE, ignore any environment variables specifying AWS credentials and attempt to download the file as publicly available
data_dir

root directory for downloaded files (defaults to tools::R_user_dir("s3", "data"))

Value

a character string that is the file path to the downloaded file (invisibly)

Examples

Sys.setenv("R_USER_DATA_DIR" = tempdir())
the_file <- s3_get(s3_uri = "s3://geomarker/testing_downloads/mtcars.rds")
s3_get("s3://geomarker/testing_downloads/mtcars.rds") |>
    readRDS()
unlink(the_file)

download several s3 files

Description

Progress messages for downloading several S3 objects at once cannot be silenced. Like s3_get, S3 objects that have already been downloaded will not be re downloaded

Usage

s3_get_files(
  s3_uri,
  region = "us-east-2",
  progress = FALSE,
  force = FALSE,
  confirm = TRUE,
  public = FALSE,
  data_dir = tools::R_user_dir("s3", "data")
)

Arguments

s3_uri

vector of S3 object URIs
region

AWS region for bucket containing the file (defaults to "us-east-2", but only required for private files)
progress

show download progress for each individual file? (currently only for public objects)
force

force download to overwrite existing S3 objects
confirm

ask user to interactively confirm downloads? (only possible when session is interactive)
public

defaults to FALSE; if TRUE, ignore any environment variables specifying AWS credentials and attempt to download the file as publicly available
data_dir

root directory for downloaded files (defaults to tools::R_user_dir("s3", "data"))

Value

data.frame (or tibble) with s3_uris and corresponding file paths to downloaded files (invisibly)

Examples

Sys.setenv("R_USER_DATA_DIR" = tempdir())
the_files <- s3_get_files(c(
    "s3://geomarker/testing_downloads/mtcars.rds",
    "s3://geomarker/testing_downloads/mtcars.fst"
))
unlink(the_files$file_path)