Storage and mountpath management

1
$ ais storage <TAB-TAB>
2
cleanup     disk        mountpath   summary     validate

1
$ ais storage --help
2
NAME:
3
   ais storage - monitor and manage clustered storage
4
5
USAGE:
6
   ais storage command [arguments...] [command options]
7
8
COMMANDS:
9
   show       show storage usage and utilization, disks and mountpaths
10
   summary    show bucket sizes and %% of used capacity on a per-bucket basis
11
   validate   check buckets for misplaced objects and objects that have insufficient numbers of copies or EC slices
12
   mountpath  show and attach/detach target mountpaths
13
   disk       show disk utilization and read/write statistics
14
   cleanup    perform storage cleanup: remove deleted objects and old/obsolete workfiles
15
16
OPTIONS:
17
   --help, -h  show help

1
$ find . -type f -name "*.md" | xargs grep "ais.*mountpath"

1
$ ais storage cleanup --help
2
NAME:
3
   ais storage cleanup - remove deleted objects and old/obsolete workfiles; remove misplaced objects; optionally, remove zero size objects
4
5
USAGE:
6
   ais storage cleanup PROVIDER:[//BUCKET_NAME] [command options]
7
8
OPTIONS:
9
   --force, -f      disregard interrupted rebalance and possibly other conditions preventing full cleanup
10
                    (tip: check 'ais config cluster lru.dont_evict_time' as well)
11
   --rm-zero-size   remove zero-size objects (caution: advanced usage only)
12
   --wait           wait for an asynchronous operation to finish (optionally, use '--timeout' to limit the waiting time)
13
   --timeout value  maximum time to wait for a job to finish; if omitted: wait forever or until Ctrl-C;
14
                    valid time units: ns, us (or µs), ms, s (default), m, h
15
   --help, -h       show help

1
# ais storage cleanup
2
Started storage cleanup "BlpmlObF8", use 'ais job show xaction BlpmlObF8' to monitor the progress

1
$ ais scrub --help
2
3
NAME:
4
   ais scrub - (alias for "storage validate") Check in-cluster content for misplaced objects, objects that have insufficient numbers of copies, zero size, and more
5
   e.g.:
6
     * ais storage validate                 - validate all in-cluster buckets;
7
     * ais scrub                            - same as above;
8
     * ais storage validate ais             - validate (a.k.a. scrub) all ais:// buckets;
9
     * ais scrub s3                         - ditto, all s3:// buckets;
10
     * ais scrub s3 --refresh 10            - same as above while refreshing runtime counter(s) every 10s;
11
     * ais scrub gs://abc/images/           - validate part of the gcp bucket under 'images/`;
12
     * ais scrub gs://abc --prefix images/  - same as above.
13
14
USAGE:
15
   ais scrub [BUCKET[/PREFIX]] [PROVIDER] [command options]
16
17
OPTIONS:
18
   --all-columns          Show all columns, including those with only zero values
19
   --cached               Only visit in-cluster objects, i.e., objects from the respective remote bucket that are present ("cached") in the cluster
20
   --count value          Used together with '--refresh' to limit the number of generated reports, e.g.:
21
                           '--refresh 10 --count 5' - run 5 times with 10s interval (default: 0)
22
   --large-size value     Count and report all objects that are larger or equal in size  (e.g.: 4mb, 1MiB, 1048576, 128k; default: 5 GiB)
23
   --limit value          The maximum number of objects to list, get, or otherwise handle (0 - unlimited; see also '--max-pages'),
24
                          e.g.:
25
                          - 'ais ls gs://abc/dir --limit 1234 --cached --props size,custom,atime'  - list no more than 1234 objects
26
                          - 'ais get gs://abc /dev/null --prefix dir --limit 1234'                 - get --/--
27
                          - 'ais scrub gs://abc/dir --limit 1234'                                  - scrub --/-- (default: 0)
28
   --max-pages value      Maximum number of pages to display (see also '--page-size' and '--limit')
29
                          e.g.: 'ais ls az://abc --paged --page-size 123 --max-pages 7 (default: 0)
30
   --no-headers, -H       Display tables without headers
31
   --non-recursive, --nr  Non-recursive operation, e.g.:
32
                          - 'ais ls gs://bucket/prefix --nr'   - list objects and/or virtual subdirectories with names starting with the specified prefix;
33
                          - 'ais ls gs://bucket/prefix/ --nr'  - list contained objects and/or immediately nested virtual subdirectories _without_ recursing into the latter;
34
                          - 'ais prefetch s3://bck/abcd --nr'  - prefetch a single named object (see 'ais prefetch --help' for details);
35
                          - 'ais rmo gs://bucket/prefix --nr'  - remove a single object with the specified name (see 'ais rmo --help' for details)
36
   --page-size value      Maximum number of object names per page; when the flag is omitted or 0
37
                          the maximum is defined by the corresponding backend; see also '--max-pages' and '--paged' (default: 0)
38
   --prefix value         For each bucket, select only those objects (names) that start with the specified prefix, e.g.:
39
                          '--prefix a/b/c' - sum up sizes of the virtual directory a/b/c and objects from the virtual directory
40
                          a/b that have names (relative to this directory) starting with the letter c
41
   --refresh value        Time interval for continuous monitoring; can be also used to update progress bar (at a given interval);
42
                          valid time units: ns, us (or µs), ms, s (default), m, h
43
   --small-size value     Count and report all objects that are smaller or equal in size (e.g.: 4, 4b, 1k, 128kib; default: 0)
44
   --help, -h             Show help

1
$ ais scrub s3://data/my-prefix --large-size 500k
2
3
BUCKET/PREFIX        OBJECTS         NOT-CACHED      SMALL   LARGE           VER-CHANGED     DELETED
4
s3://data/my-prefix  1637 (1.6GiB)   1465 (1.4GiB)   -       172 (172.0MiB)  1 (1.0MiB)      1 (1.0MiB)
5
6
Detailed Logs
7
-------------
8
* not-cached objects:   /tmp/.ais-scrub-not-cached.204f71.log (1465 records)
9
* large objects:        /tmp/.ais-scrub-large.204f71.log (172 records)
10
* ver-changed objects:  /tmp/.ais-scrub-ver-changed.204f71.log (1 record)
11
* deleted objects:      /tmp/.ais-scrub-deleted.204f71.log (1 record)

1
$ ais scrub s3://data/my-prefix --large-size 500k --all-columns
2
3
BUCKET/PREFIX        OBJECTS         NOT-CACHED      MISPLACED(cluster)  MISPLACED(mountpath) MISSING-COPIES  SMALL  LARGE          VER-CHANGED   DELETED
4
s3://data/my-prefix  1637 (1.6GiB)   1465 (1.4GiB)   -                   -                    -               -      172 (172.0MiB) 1 (1.0MiB)    1 (1.0MiB)
5
6
Detailed Logs
7
-------------
8
* not-cached objects:   /tmp/.ais-scrub-not-cached.204f8c.log (1465 records)
9
* large objects:        /tmp/.ais-scrub-large.204f8c.log (172 records)
10
* ver-changed objects:  /tmp/.ais-scrub-ver-changed.204f8c.log (1 record)
11
* deleted objects:      /tmp/.ais-scrub-deleted.204f8c.log (1 record)

1
$ ais show storage mountpath t[TqPtghbiRw]
2
3
TqPtghbiRw
4
        Used Capacity (all disks): avg 15% max 18%
5
                                                /ais/mp1/2 /dev/nvme0n1(xfs)
6
                                                /ais/mp2/2 /dev/nvme1n1(xfs)
7
                                                /ais/mp3/2 /dev/nvme2n1(xfs)
8
                                                /ais/mp4/2 /dev/nvme3n1(xfs)

1
$ ais show storage mountpath --help
2
NAME:
3
   ais show storage mountpath - show target mountpaths
4
5
USAGE:
6
   ais show storage mountpath [TARGET_ID] [command options]
7
8
OPTIONS:
9
   --refresh value  interval for continuous monitoring;
10
                    valid time units: ns, us (or µs), ms, s (default), m, h
11
   --count value    used together with '--refresh' to limit the number of generated reports (default: 0)
12
   --json, -j       json input/output
13
   --help, -h       show help

1
$ ais storage mountpath show 12356t8085
2
247389t8085
3
        Available:
4
			/tmp/ais/5/3
5
			/tmp/ais/5/1
6
        Disabled:
7
			/tmp/ais/5/2
8
9
$ ais storage mountpath show
10
12356t8085
11
        Available:
12
			/tmp/ais/5/3
13
			/tmp/ais/5/1
14
        Disabled:
15
			/tmp/ais/5/2
16
147665t8084
17
        Available:
18
			/tmp/ais/4/3
19
			/tmp/ais/4/1
20
			/tmp/ais/4/2
21
426988t8086
22
	No mountpaths

1
$ ais storage mountpath attach 12367t8080=/data/dir

1
$ ais storage mountpath detach 12367t8080=/data/dir