BiocManager::repositories() Reports Missing `books` for 3.15/3.16

[jonyoder]

When using Bioconductor 3.15 or 3.16, the command BiocManager::repositories() reports that the book repository is available, but the repository does not appear to exist on bioconductor.org.

[lshep]

Do you get an ERROR when trying to access a book? The books are handle a bit differently and do not appear on the biocViews page . You can access their landing page, which is a rendering of the book, through the build report page. Example devel books report which has "landing page" links

[jonyoder]

Thanks for that clarification! I'll close this issue.

[jonyoder]

Oh, I see what was tripping us up. The books repo is missing from devel_repos in config.yaml. We parse config.yaml to figure out what repos are available.

## CHANGE THIS (i.e., uncomment) as various parts of the new devel version 
## become available. set to "[]" if none are available.
devel_repos:
- "bioc"
- "data/experiment"
- "workflows"
- "data/annotation"

[jonyoder]

Would it be reasonable to have BiocManager::repositories() respect what is currently enabled in config.yaml? I see it's currently hard-coded at https://github.com/Bioconductor/BiocManager/blob/master/R/repositories.R#L104.

[LiNk-NY]

Oh, I see what was tripping us up. The books repo is missing from devel_repos in config.yaml. We parse config.yaml to figure out what repos are available.

Hi Jonathan, @jonyoder

Lori can correct me if I'm wrong but this field in config.yaml is used during the release process. I don't recommend using it as the basis for available repositories. I would refer you to BiocManager::repositories() for that rather than parsing the config.yaml file.

Best regards,
Marcel

[lshep]

yes it is used in the release process as well as some of the BBS code. I agree with marcel's comment above you should use the provided functions rather than he config.yaml. This seems trivial request but it could actually have many foreseen consequences. We can look into it but its not as simple as it seems.

[jonyoder]

What we're concerned about, in particular, is the devel version of Bioconductor. If I understand it correctly, there are times during the development cycle for a new release when not all repositories are available. I noticed that BiocManager::repositories() appears to always enumerate all repos based on a hard-coded version: https://github.com/Bioconductor/BiocManager/blob/master/R/repositories.R#L99-L105

Is it possible that BiocManager::repositories(version='<devel_version'>) may sometimes return repositories (e.g., books) that are not yet available?

[jonyoder]

@LiNk-NY, @lshep, thanks for the replies! I realize I should have described our use case a bit more thoroughly. 😁

The use case here is for RStudio Package Manager (e.g. https://packagemanager.rstudio.com/), and we're essentially building a manifest of the available Bioconductor repos, including the current devel repo. We've noticed that during the devel lifecycle, some repos may not be available. A couple of our goals include:

• If a devel repo is not yet available, don't throw errors, but simply build a manifest with the available repos.
• If a new devel repo (like books) becomes available at some point, automatically include it in our manifest without any manual intervention.

Do you have any suggestions for how we can know deterministically exactly which devel repos are available? The options I'm aware of include:

• Parsing config.yaml. This was our current approach, but it turned out to be a bad idea since config.yaml wasn't updated with the books repo for 3.15 and 3.16.
• Using BiocManager::repositories(version=<ver>). This could work for us, but if I understand it correctly, it will report that all the repos are available for the devel version even if some of them have not yet been published.

[LiNk-NY]

Hi Jonathan, @jonyoder

Thanks for providing some background. That is helpful.
You're right, we should be using the yaml file as the basis for the BiocManager::repositories functions and others. For practical reasons, we recommend using BiocManager functions and handling warnings about missing repositories in the codebase.
We will consider the former approach in future versions of BiocManager though it will take time and effort to implement.

Best regards,
Marcel