Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

high-scaled RocksDB indices for all public metagenomes - locations on farm #3523

Open
ctb opened this issue Feb 4, 2025 · 1 comment
Open

high-scaled RocksDB indices for all public metagenomes - locations on farm #3523

ctb opened this issue Feb 4, 2025 · 1 comment
Labels
fyi Information that is interesting or useful

Comments

@ctb
Copy link
Contributor

ctb commented Feb 4, 2025

In /group/ctbrowngrp5/sra-metagenomes/, we have:

total 102768
drwxr-xr-x 2 irber ctbrowngrp       237 Jan 13 10:17 20241128-k21-s100000
drwxr-xr-x 2 irber ctbrowngrp       532 Jan  1 08:36 20241128-k31-s100000
drwxr-xr-x 2 irber ctbrowngrp       586 Jan  4 14:20 20241128-k51-s100000
-rw-r--r-- 1 irber irber      105158205 Jan 25 15:59 20241128-metadata.parquet

which are, if I understand correctly, RocksDB databases of all public metagenomes at scaled=100_000.
@ctb ctb added the fyi Information that is interesting or useful label Feb 4, 2025
@luizirber
Copy link
Member

The parquet file is the metadata for 1,161,119 SRA metagenomes as of 2024-11-28, and indexed in 20241128-k{21,31,51}-s100000

This is also mirrored externally in https://farm.cse.ucdavis.edu/~irber/branchwater/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
fyi Information that is interesting or useful
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@luizirber @ctb