set expiration date for web resource

[LiNk-NY]

Hi Lori, @lshep

Is it possible to set an expiration date for a URL resource within the bfcadd function or any other function?
I tried this and it doesn't work.

library(BiocFileCache)
#> Loading required package: dbplyr
bfc <- BiocFileCache()
tfile <- "https://bioconductor.org/checkResults/3.15/bioc-LATEST/report.tgz"
treport <- bfcadd(
    bfc, rname = tfile, fpath = tfile, expires = Sys.Date() + 2
)
res <- bfcquery(bfc, tfile, exact = TRUE)
res
#>   id  rid                                                             rname
#> 1  9 BFC9 https://bioconductor.org/checkResults/3.15/bioc-LATEST/report.tgz
#>           create_time         access_time                    rpath rtype
#> 1 2022-01-11 23:04:43 2022-01-11 23:04:48 179285b1b6faf_report.tgz   web
#>                                                               fpath
#> 1 https://bioconductor.org/checkResults/3.15/bioc-LATEST/report.tgz
#>    last_modified_time                  etag expires
#> 1 2022-01-10 16:08:46 77b841f-5d53c8d250fe7      NA

^{Created on 2022-01-11 by the reprex package (v2.0.1.9000)}

Thanks!
-Marcel

[lshep]

think we thought it should be automatic based on HEAD/certificate information and every time it checks the header may override it --- I'll think about how to remedy this

[LiNk-NY]

Thanks Lori.
I guess this method might work for the first expiration but not for subsequent ones of the resource.
Perhaps there could be an argument expires_every that takes the number of days as input?
It would override the automatic expiration if any.

[omsai]

I need to force an expiration date for a web resource because the web server does not provide any expiration for the file. So bfcneedsupdate() always returns NA. My use case is checking the web API schema YAML without hammering the server for the unit tests and for generating a sensible message for users (Bioconductor/Contributions#3986).

There is currently no way to override the resource expiration using curl::curl_options(), etc.

Maybe allow for call to .sql_set_expires() from the public API functions bfcadd() and bfcupdate()?