Step #2: Dataset Creation (Local Upload)
To create a new dataset to upload into Base Command, let’s first create a new directory on our local system. Make note of where you are creating this directory on your local machine. To create a directory from the command line of a MacOS or Linux-based machine, run the following:
$ mkdir base-command-test
Next, let’s go ahead and create a single file inside that directory, to simulate a single test file.
$ touch base-command-test/base-command-test-data
The ngc command line tool can now be used to name the dataset, upload the data (in this case, one file) with the --source flag, and set the dataset description with the optional --desc flag. The final argument is the name being given to the dataset.
$ ngc dataset upload --source base-command-test/ --desc "Test dataset for Base Command" base-command-test
Calculating dataset size and file count.
.
Checking available space.
Attempting to create the dataset.
Uploaded 0 B, 0/1 files in 1s, Avg Upload speed: 0 B/s, Curr Upload Speed: 0 B/s
----------------------------------------------------
Dataset ID: 98775
Upload status: Completed
Uploaded local path (dataset): /Users/jhandzik/base-command-test
Total files uploaded: 1
Total transferred: 0 B
Started at: 2022-04-13 16:55:37
Completed at: 2022-04-13 16:55:39
Duration taken: 2s
----------------------------------------------------
You can verify the upload was successful through either the CLI or Web UI.
$ ngc dataset list --owned
+------------------+------------+------------------+------------------+------------------+--------+----------+-----------+--------------+-------+---------+
| Id | Integer Id | Name | Description | ACE | Shared | Size | Status | Created Date | Owned | Pre-pop |
+------------------+------------+------------------+------------------+------------------+--------+----------+-----------+--------------+-------+---------+
| 6yg_d9K-Q4i8Dfjc | 98775 | base-command- | Test dataset for | nv-launchpad-bc- | No | 0 B | COMPLETED | 2022-04-13 | Yes | No |
| SeOi_g | | test | Base Command | iad1 | | | | 21:55:36 UTC | | |
| Zsa1URO9TXu_Jg_K | 96139 | parabricks_sampl | parabricks homo | nv-launchpad-bc- | Yes | 10.34 GB | COMPLETED | 2022-02-22 | Yes | No |
| YrKF5Q | | e | sapiens dataset | iad1 | | | | 18:24:13 UTC | | |
+------------------+------------+------------------+------------------+------------------+--------+----------+-----------+--------------+-------+---------+
