Command-Line Interface

ais show command

View as Markdown

AIS CLI show command can universally be used to view summaries and details on a cluster and its nodes, buckets and objects, running and finished jobs - in short, all managed entities (see below).

The command is a “hub” for all information-viewing commands that are currently supported:

1$ ais show --help
2
3COMMANDS:
4$ ais show --help
5
6NAME:
7   ais show - Show configuration, buckets, jobs, etc. - all managed entities in the cluster, and the cluster itself
8
9USAGE:
10   ais show command [arguments...]  [command options]
11
12COMMANDS:
13   auth            Show entity in authn
14   object          Show object properties
15   bucket          Show bucket properties
16   cluster         Show cluster: health, version and build, and nodes (including capacity and memory, load averages and alerts)
17   dashboard       Show cluster at-a-glance dashboard: node counts, capacity, performance, health, software version, and more
18   performance     Show performance counters, throughput, latency, disks, used/available capacities (press <TAB-TAB> to select specific view)
19   storage         Show storage usage and utilization, disks and mountpaths
20   rebalance       show rebalance status and stats
21   config          Show CLI, cluster, or node configurations (nodes inherit cluster and have local)
22   remote-cluster  Show attached remote AIS clusters
23   remote          Show remote cluster information and statistics
24
25   job      Show running and/or finished jobs:
26            archive              blob-download  cleanup     copy-bucket   copy-objects   create-inventory
27            delete-objects       download       dsort       ec-bucket     ec-get         ec-put
28            ec-resp              elect-primary  etl-bucket  etl-inline    etl-objects    evict-objects
29            evict-remote-bucket  get-batch      list        lru-eviction  mirror         prefetch-objects
30            promote-files        put-copies     rebalance   rechunk       rename-bucket  resilver
31            summary              warm-up-metadata
32
33   tls  Show TLS certificate's version, issuer's common name, from/to validity bounds
34   etl  Show ETL(s)
35   nbi  Show bucket inventory

For easy usage, all show commands have been aliased to their respective top-level counterparts:

1$ ais show <command>

is equivalent to:

1$ ais <command> show

For instance, ais show performance is an alias for ais performance - both can be used interchangeably.

As a general rule, instead of remembering any of the above (as well as any of the below), type (e.g.) ais perf<TAB-TAB> and press Enter.

You can also hit <TAB-TAB>` maybe a few more times while typing a few more letters in between. Any combination will work.

When part-typing, part-TABing a sequence of words that (will eventually) constitute a valid AIS CLI command, type --help at any time. This will display short description, command-line options, usage examples, and other context-sensitive information.

For usage examples, another way to quickly find them would be a good-and-old keyword search. E.g., cd aistore; grep -n "ais.*cp" docs/cli/*.md, and similar. Additional useful tricks (including ais search) are also mentioned elsewhere in the documentation.

As far as ais show, the command currently extends as follows:

1$ ais show <TAB-TAB>
2
3auth             cluster          storage          remote-cluster   log              nbi
4object           dashboard        rebalance        remote           tls
5bucket           performance      config           job              etl

In other words, there are currently 12 subcommands that are briefly described in the rest of this text.

Table of Contents

ais show performance

The command has 5 subcommands:

1$ ais show performance
2counters     throughput   latency      capacity     disk

The command provides a quick single-shot overview of cluster performance. If you type ais show performance and instead of <TAB-TAB> hit Enter, the command will show performance counters, aggregated cluster throughput statistics, and the other 3 tables (latency, capacity, disk IO) - in sequence, one after another.

The command’s help screen follows below - notice the command-line options (aka flags):

1$ ais show performance --help
2NAME:
3   ais show performance - show performance counters, throughput, latency, disks, used/available capacities (press <TAB-TAB> to select specific view)
4
5USAGE:
6   ais show performance command [TARGET_ID] [command options]
7
8COMMANDS:
9   counters    show (GET, PUT, DELETE, RENAME, EVICT, APPEND) object counts, as well as:
10               - numbers of list-objects requests;
11               - (GET, PUT, etc.) cumulative and average sizes;
12               - associated error counters, if any.
13   throughput  show GET and PUT throughput, associated (cumulative, average) sizes and counters
14   latency     show GET, PUT, and APPEND latencies and average sizes
15   capacity    show target mountpaths, disks, and used/available capacity
16   disk        show disk utilization and read/write statistics
17
18OPTIONS:
19   --refresh value   interval for continuous monitoring;
20                     valid time units: ns, us (or µs), ms, s (default), m, h
21   --count value     used together with '--refresh' to limit the number of generated reports (default: 0)
22   --all             when printing tables, show all columns including those that have only zero values
23   --no-headers, -H  display tables without headers
24   --regex value     regular expression to select table columns (case-insensitive), e.g.: --regex "put|err"
25   --units value     show statistics and/or parse command-line specified sizes using one of the following _units of measurement_:
26                     iec - IEC format, e.g.: KiB, MiB, GiB (default)
27                     si  - SI (metric) format, e.g.: KB, MB, GB
28                     raw - do not convert to (or from) human-readable format
29   --average-size    show average GET, PUT, etc. request size
30   --help, -h        show help

As one would maybe expect, --refresh, --units and all the other listed flags universally work across all performance subcommands.

For instance, you could continuously run several screens to simultaneously display a variety of performance-related aspects:

1# screen #1
2
3$ ais show performance counters --refresh 5
1# screen #2
2
3$ ais show performance throughput --refresh 5

and so on - all the 5 tables with 5-seconds periodicity, or any other time interval(s) of your choosing.

As far as continuous monitoring goes, this (approach) has a chance to provide a good overall insight. A poor-man’s addition, if you will, to the popular (and also supported) tools such as Grafana and Prometheus. But available with zero setup out of the box (which is a plus).

What’s running

Use ais show performance and its variations in combination with ais show job (and variations). The latter shows what’s running in the cluster, and thus combining the two may make sense.

See also

ais show job

1$ ais show job --help
2NAME:
3   ais show job - Show running and/or finished jobs:
4     archive        blob-download  cleanup     copy-bucket       copy-objects   delete-objects
5     download       dsort          ec-bucket   ec-get            ec-put         ec-resp
6     elect-primary  etl-bucket     etl-inline  etl-objects       evict-objects  evict-remote-bucket
7     list           lru-eviction   mirror      prefetch-objects  promote-files  put-copies
8     rebalance      rename-bucket  resilver    summary           warm-up-metadata
9   (use any of these names with 'ais show job' command, or try shortcuts: "evict", "prefetch", "copy", "delete", "ec")
10   e.g.:
11     - show job prefetch-listrange         - show all running prefetch jobs;
12     - show job prefetch                   - same as above;
13     - show job prefetch --top 5           - show 5 most recent prefetch jobs;
14     - show job tco-cysbohAGL              - show a given (multi-object copy/transform) job identified by its unique ID;
15     - show job copy-listrange             - show all running multi-object copies;
16     - show job copy-objects               - same as above (using display name);
17     - show job copy                       - show all copying jobs including both bucket-to-bucket and multi-object;
18     - show job copy-objects --all         - show both running and already finished (or stopped) multi-object copies;
19     - show job copy-objects --all --top 10 - show 10 most recent multi-object copy jobs;
20     - show job ec                         - show all erasure-coding;
21     - show job list                       - show all running list-objects jobs;
22     - show job ls                         - same as above;
23     - show job ls --refresh 10            - same as above with periodic _refreshing_ every 10 seconds;
24     - show job ls --refresh 10 --count 4  - same as above but only for the first four 10-seconds intervals;
25     - show job prefetch --refresh 1m      - show all running prefetch jobs at 1 minute intervals (until Ctrl-C);
26     - show job evict                      - all running bucket and/or data evicting jobs;
27     - show job --all                      - show absolutely all jobs, running and finished.
28
29USAGE:
30   ais show job [NAME] [JOB_ID] [NODE_ID] [BUCKET] [command options]
31
32OPTIONS:
33   all           Include all jobs: running, finished, and aborted
34   count         Used together with '--refresh' to limit the number of generated reports, e.g.:
35                  '--refresh 10 --count 5' - run 5 times with 10s interval
36   date-time     Override the default hh:mm:ss (hours, minutes, seconds) time format - include calendar date as well
37   json,j        JSON input/output
38   log           Filename to log metrics (statistics)
39   no-headers,H  Display tables without headers
40   progress      Show progress bar(s) and progress of execution in real time
41   refresh       Time interval for continuous monitoring; can be also used to update progress bar (at a given interval);
42                 valid time units: ns, us (or µs), ms, s (default), m, h
43   regex         Regular expression to select jobs by name, kind, or description, e.g.: --regex "ec|mirror|elect"
44   top           Show top N most recent jobs (e.g., --top 5 to show the 5 most recent jobs)
45   units         Show statistics and/or parse command-line specified sizes using one of the following units of measurement:
46                 iec - IEC format, e.g.: KiB, MiB, GiB (default)
47                 si  - SI (metric) format, e.g.: KB, MB, GB
48                 raw - do not convert to (or from) human-readable format
49   verbose,v     Show extended statistics
50   help, h       Show help

The command has no statically defined subcommands. When you type ais show job <TAB-TAB>, the resulting set of shell completions will only include job names (aka “kinds”) that are currently running. Example:

1$ ais show job <TAB-TAB>
2move-bck    rebalance

Just maybe to reiterate the same slightly differently: ais show job <TAB-TAB> won’t produce anything useful iff the cluster currently doesn’t run any jobs

On the other hand, job-identifying selections [NAME] [JOB_ID] [NODE_ID] [BUCKET]:

  1. are all optional, and
  2. can be typed in an arbitrary order

Example:

1$ ais show job xhcTyUfJF t[Kritxhbt]

is the same as:

1$ ais show job t[Kritxhbt] xhcTyUfJF

In both cases, we are asking a specific target node (denoted as NODE_ID in the command’s help) for a specific job ID (denoted as JOB_ID).

Selection-wise, this would be the case of ultimate specificity. Accordingly, on the opposite side of the spectrum would be something like:

1$ ais show job --all

The --all flag always defines the broadest scope possible, and so the query ais show job --all includes absolutely all jobs, running and finished (succeeded and aborted).

Here’s at a glance:

1$ ais show job --help
2NAME:
3   ais show job - Show running and/or finished jobs:
4     archive        blob-download  cleanup     copy-bucket       copy-objects   delete-objects
5     download       dsort          ec-bucket   ec-get            ec-put         ec-resp
6     elect-primary  etl-bucket     etl-inline  etl-objects       evict-objects  evict-remote-bucket
7     list           lru-eviction   mirror      prefetch-objects  promote-files  put-copies
8     rebalance      rename-bucket  resilver    summary           warm-up-metadata
9   (use any of these names with 'ais show job' command, or try shortcuts: "evict", "prefetch", "copy", "delete", "ec")
10   e.g.:
11     - show job prefetch-listrange         - show all running prefetch jobs;
12     - show job prefetch                   - same as above;
13     - show job prefetch --top 5           - show 5 most recent prefetch jobs;
14     - show job tco-cysbohAGL              - show a given (multi-object copy/transform) job identified by its unique ID;
15     - show job copy-listrange             - show all running multi-object copies;
16     - show job copy-objects               - same as above (using display name);
17     - show job copy                       - show all copying jobs including both bucket-to-bucket and multi-object;
18     - show job copy-objects --all         - show both running and already finished (or stopped) multi-object copies;
19     - show job copy-objects --all --top 10 - show 10 most recent multi-object copy jobs;
20     - show job ec                         - show all erasure-coding;
21     - show job list                       - show all running list-objects jobs;
22     - show job ls                         - same as above;
23     - show job ls --refresh 10            - same as above with periodic _refreshing_ every 10 seconds;
24     - show job ls --refresh 10 --count 4  - same as above but only for the first four 10-seconds intervals;
25     - show job prefetch --refresh 1m      - show all running prefetch jobs at 1 minute intervals (until Ctrl-C);
26     - show job evict                      - all running bucket and/or data evicting jobs;
27     - show job --all                      - show absolutely all jobs, running and finished.
28
29USAGE:
30   ais show job [NAME] [JOB_ID] [NODE_ID] [BUCKET] [command options]
31
32OPTIONS:
33   all           Include all jobs: running, finished, and aborted
34   count         Used together with '--refresh' to limit the number of generated reports, e.g.:
35                  '--refresh 10 --count 5' - run 5 times with 10s interval
36   date-time     Override the default hh:mm:ss (hours, minutes, seconds) time format - include calendar date as well
37   json,j        JSON input/output
38   log           Filename to log metrics (statistics)
39   no-headers,H  Display tables without headers
40   progress      Show progress bar(s) and progress of execution in real time
41   refresh       Time interval for continuous monitoring; can be also used to update progress bar (at a given interval);
42                 valid time units: ns, us (or µs), ms, s (default), m, h
43   regex         Regular expression to select jobs by name, kind, or description, e.g.: --regex "ec|mirror|elect"
44   top           Show top N most recent jobs (e.g., --top 5 to show the 5 most recent jobs)
45   units         Show statistics and/or parse command-line specified sizes using one of the following units of measurement:
46                 iec - IEC format, e.g.: KiB, MiB, GiB (default)
47                 si  - SI (metric) format, e.g.: KB, MB, GB
48                 raw - do not convert to (or from) human-readable format
49   verbose,v     Show extended statistics
50   help, h       Show help

Example: show all currently running jobs, and narrow the selection to a given target node:

1$ ais show job t[ugoFtqUrrm]
2
3NODE          ID            KIND         BUCKET                     OBJECTS     BYTES        START           END             STATE
4ugoFtqUrrm    vOYSo5pHG     ec-get       mybucket-ec-rebalance      -           -            12-03 10:32:25  -               Running
5ugoFtqUrrm    b4Ks45pHv     ec-get       mybucket-obj-n-slice       9           42.36MiB     12-03 10:31:33  -               Running
6ugoFtqUrrm    vUYSo5pHvS    ec-put       mybucket-ec-rebalance      3           1.43MiB      12-03 10:32:25  -               Running
7ugoFtqUrrm    Kobs45pHvS    ec-put       mybucket-obj-n-slice       9           4.75MiB      12-03 10:31:33  -               Running
8ugoFtqUrrm    U8UcSo5pHv    ec-resp      mybucket-ec-rebalance      18          89.45MiB     12-03 10:32:25  -               Running
9ugoFtqUrrm    M8M6sodqmv    ec-resp      mybucket-obj-n-slice       13          64.49MiB     12-03 10:31:34  -               Idle
10ugoFtqUrrm    Ioa31VqaB     list         mybucket-ec-rebalance      5           -            12-03 10:32:32  12-03 10:32:42  Aborted
11ugoFtqUrrm    X3H381Vqau    list         mybucket-ec-rebalance      3           -            12-03 10:32:29  12-03 10:32:42  Aborted
12ugoFtqUrrm    g5            rebalance    -                          6           25.59MiB     12-03 10:32:32  12-03 10:32:39  Finished

Example: show a job by name, and include finished/aborted:

1$ ais show job resilver --all
2
3resilver[G0p7yXYiUg]
4NODE             ID              KIND            OBJECTS         BYTES           START           END             STATE
5HAAt8090         G0p7yXYiUg      resilver        11              18.38KiB        13:04:51        13:04:51        Finished
6JVnt8086         G0p7yXYiUg      resilver        10              14.86KiB        13:04:51        13:04:51        Finished
7LDgt8088         G0p7yXYiUg      resilver        14              20.81KiB        13:04:51        13:04:51        Finished
8OBIt8089         G0p7yXYiUg      resilver        4               7.04KiB         13:04:51        13:04:51        Finished
9VUCt8091         G0p7yXYiUg      resilver        9               14.50KiB        13:04:51        13:04:51        Finished
10qVJt8087         G0p7yXYiUg      resilver        9               14.21KiB        13:04:51        13:04:51        Finished
11                                 Total:         57              89.80KiB ✓

Here and elsewhere in the documentation, CLI colors used to highlight certain (notable) items on screen - are not shown.

On a related note: coloring can be disabled via ais config cli set no_color.

Example: continuous monitoring

1$ ais show job --refresh 5
2
3rebalance[g2]
4NODE             ID      KIND            OBJECTS         BYTES           START           END     STATE
5HAAt8090         g2      rebalance       2072            3.05MiB         13:35:00        -       Running
6JVnt8086         g2      rebalance       2               2.08KiB         13:35:00        -       Running
7LDgt8088         g2      rebalance       4               6.46KiB         13:35:00        -       Running
8OBIt8089         g2      rebalance       4               7.39KiB         13:35:00        -       Running
9VUCt8091         g2      rebalance       -               -               13:35:00        -       Running
10qVJt8087         g2      rebalance       -               -               13:35:00        -       Running
11------------------------------------------------------------------------
12rebalance[g2]
13NODE             ID      KIND            OBJECTS         BYTES           START           END     STATE
14HAAt8090         g2      rebalance       2072            3.05MiB         13:35:00        -       Running
15JVnt8086         g2      rebalance       2               2.08KiB         13:35:00        -       Running
16LDgt8088         g2      rebalance       4               6.46KiB         13:35:00        -       Running
17OBIt8089         g2      rebalance       4               7.39KiB         13:35:00        -       Running
18VUCt8091         g2      rebalance       -               -               13:35:00        -       Running
19qVJt8087         g2      rebalance       -               -               13:35:00        -       Running
20------------------------------------------------------------------------
21No running jobs. Use '--all' to show all, '--all' <TAB-TAB> to select, '--help' for details.
22------------------------------------------------------------------------
23ec-get[wHUgmjFaJ]
24NODE             ID              BUCKET                                  OBJECTS         BYTES   ERRORS  QUEUE   AVG TIME        START           END     ABORTED
25OBIt8089         wHUgmjFaJ       ais://TESTAISBUCKET-ec-rebalance        -               -       -       -       -               13:35:24        -       false
26ec-get[P3SqrpURnv]
27NODE             ID              BUCKET                                  OBJECTS         BYTES   ERRORS  QUEUE   AVG TIME        START           END     ABORTED
28VUCt8091         P3SqrpURnv      ais://TESTAISBUCKET-ec-rebalance        -               -       -       -       -               13:35:24        -       false
29ec-get[u3pGCzmh4]
30NODE             ID              BUCKET                                  OBJECTS         BYTES   ERRORS  QUEUE   AVG TIME        START           END     ABORTED
31JVnt8086         u3pGCzmh4       ais://TESTAISBUCKET-ec-rebalance        -               -       -       -       -               13:35:24        -       false
32
33...
34
35list[n2O4CJxUg]
36NODE             ID              KIND    BUCKET                                  OBJECTS         BYTES           START           END     STATE
37HAAt8090         n2O4CJxUg       list    ais://TESTAISBUCKET-ec-rebalance        10              32.51MiB        13:36:55        -       Idle
38JVnt8086         n2O4CJxUg       list    ais://TESTAISBUCKET-ec-rebalance        16              52.96MiB        13:36:55        -       Idle
39LDgt8088         n2O4CJxUg       list    ais://TESTAISBUCKET-ec-rebalance        2               3.59MiB         13:36:55        -       Idle
40OBIt8089         n2O4CJxUg       list    ais://TESTAISBUCKET-ec-rebalance        6               11.72MiB        13:36:55        -       Idle
41VUCt8091         n2O4CJxUg       list    ais://TESTAISBUCKET-ec-rebalance        10              37.53MiB        13:36:55        -       Idle
42i                               Total:                                          44              138.31MiB ✓
43
44list[J-bfCJxVp]
45NODE             ID              KIND    BUCKET                                  OBJECTS         BYTES           START           END     STATE
46qVJt8087         J-bfCJxVp       list    ais://TESTAISBUCKET-ec-rebalance        22              60.91MiB        13:37:18        -       Idle
47
48rebalance[g8]
49NODE             ID      KIND            OBJECTS         BYTES           START           END     STATE
50HAAt8090         g8      rebalance       4               27.46MiB        13:37:18        -       Running
51JVnt8086         g8      rebalance       2               4.42MiB         13:37:18        -       Running
52LDgt8088         g8      rebalance       2               9.96MiB         13:37:18        -       Running
53OBIt8089         g8      rebalance       4               23.49MiB        13:37:18        -       Running
54VUCt8091         g8      rebalance       32              -               13:37:18        -       Running
55qVJt8087         g8      rebalance       6               39.38MiB        13:37:18        -       Running
56^[^C$ ais show job --refresh 5
57
58rebalance[g15]
59NODE             ID      KIND            OBJECTS         BYTES           START           END     STATE
60HAAt8090         g15     rebalance       648             971.49KiB       13:40:54        -       Running
61JVnt8086         g15     rebalance       688             1.01MiB         13:40:54        -       Running
62LDgt8088         g15     rebalance       554             833.21KiB       13:40:54        -       Running
63OBIt8089         g15     rebalance       734             1.07MiB         13:40:54        -       Running
64VUCt8091         g15     rebalance       -               -               13:40:54        -       Running
65qVJt8087         g15     rebalance       694             1.02MiB         13:40:54        -       Running
66------------------------------------------------------------------------
67
68...

See also

ais show cluster

The first command to think of when deploying a new cluster. Useful as well when looking for the shortest quickest summary of what’s running and what’s going on. The subcommands and brief description follows:

1# ais show cluster <TAB-TAB>
2proxy    target   smap     bmd      config   stats
1$ ais show cluster --help
2NAME:
3   ais show cluster - Show cluster: health, version and build, and nodes (including capacity and memory, load averages and alerts)
4
5USAGE:
6   ais show cluster command [NODE_ID] | [target [NODE_ID]] | [proxy [NODE_ID]] | [smap [NODE_ID]] | [bmd [NODE_ID]] | [config [NODE_ID]] | [stats [NODE_ID]] [command options]
7
8COMMANDS:
9   smap    show cluster map (Smap)
10   bmd     show bucket metadata (BMD)
11   config  show cluster and node configuration
12   stats   (alias for "ais show performance") show performance counters, throughput, latency, disks, used/available capacities (press <TAB-TAB> to select specific view)
13
14OPTIONS:
15   --refresh value   interval for continuous monitoring;
16                     valid time units: ns, us (or µs), ms, s (default), m, h
17   --count value     used together with '--refresh' to limit the number of generated reports (default: 0)
18   --json, -j        json input/output
19   --no-headers, -H  display tables without headers
20   --help, -h        show help

Example usage with no parameters and a different endpoint

1$ export AIS_ENDPOINT=http://10.0.1.148:51080
2$ ais show cluster
3
4PROXY            MEM USED(%)     MEM AVAIL       UPTIME  K8s POD         STATUS
5p[EciZrNdH][P]   0.01%           363.51GiB       94d     ais-proxy-2     online
6p[HPpnlgpj]      0.01%           363.66GiB       94d     ais-proxy-0     online
7p[LZOkYuAf]      0.01%           362.62GiB       94d     ais-proxy-8     online
8p[NXDmWuAV]      0.01%           363.68GiB       94d     ais-proxy-6     online
9p[OzRhyuOB]      0.01%           363.73GiB       94d     ais-proxy-4     online
10p[WwpvNugq]      0.01%           363.17GiB       94d     ais-proxy-9     online
11p[cAQpRMET]      0.01%           363.56GiB       94d     ais-proxy-1     online
12p[eYQteCHG]      0.01%           363.81GiB       94d     ais-proxy-7     online
13p[ehkGLcSD]      0.01%           363.57GiB       94d     ais-proxy-3     online
14p[reZqfbjy]      0.01%           363.67GiB       94d     ais-proxy-5     online
15
16TARGET           MEM USED(%)     MEM AVAIL       CAP USED(%)     CAP AVAIL       CPU USED(%)    UPTIME   K8s POD         STATUS
17t[KopwySra]      0.15%           363.54GiB       77.67%          31.825TiB       15.38%         94d      ais-target-1    online
18t[MgHbIvNG]      0.16%           363.65GiB       77.00%          30.941TiB       14.06%         94d      ais-target-4    online
19t[WoLgoQEW]      0.16%           363.72GiB       78.00%          31.763TiB       17.67%         94d      ais-target-7    online
20t[fXFQnenn]      0.11%           363.83GiB       78.00%          31.751TiB       18.12%         94d      ais-target-6    online
21t[fwKlswQP]      0.16%           363.62GiB       78.33%          31.758TiB       19.81%         94d      ais-target-5    online
22t[tFUiHCCO]      0.16%           363.76GiB       79.00%          31.617TiB       19.47%         94d      ais-target-8    online
23t[tfNkAtFk]      0.16%           363.55GiB       77.33%          31.875TiB       19.70%         94d      ais-target-2    online
24t[uxvpIDPc]      0.15%           362.57GiB       77.67%          31.846TiB       16.56%         94d      ais-target-0    online
25t[vAWmZZPv]      0.11%           363.68GiB       78.67%          31.608TiB       23.09%         94d      ais-target-3    online
26t[wSJzGVnU]      0.10%           363.33GiB       76.67%          30.972TiB       17.13%         94d      ais-target-9    online
27
28Cluster:
29   Proxies:             10 (0 unelectable)
30   Targets:             10
31   Endpoint:            http://<cluster-endpoint>:<port>
32   Cluster Map:         version 1512, UUID AGetvIKTz, primary p[EciZrNdH]
33   Backend:             AWS
34   Deployment:          K8s
35   Status:              20 online
36   Rebalance:           n/a
37   Authentication:      disabled
38   Version:             3.12.85636aa
39   Build:               2022-11-16T17:55:50+0000

AIS_ENDPOINT is part of the AIS_** environment.

AIS_ENDPOINT can point to any AIS gateway (proxy) in a given cluster. Does not necessarily have to be the primary gateway.

For CLI, in particular, AIS_ENDPOINT overrides cluster’s endpoint that’s currently configured. To view or change the configured endpoint (or any other CLI configuration item), run ais config cli.

See also

ais show dashboard

The ais show dashboard command provides comprehensive cluster analytics and health monitoring. Unlike the basic ais show cluster command which shows node tables and basic summary, this command focuses on detailed analytics including storage metrics, performance indicators, error tracking, and system health.

Command Overview

1$ ais show dashboard --help
2NAME:
3   ais show dashboard - Show cluster at-a-glance dashboard: node counts, capacity, performance, health, software version, and more
4
5USAGE:
6   ais show dashboard [NODE_ID] [command options]
7
8
9
10OPTIONS:
11   --refresh value   interval for continuous monitoring;
12                     valid time units: ns, us (or µs), ms, s (default), m, h
13   --count value     used together with '--refresh' to limit the number of generated reports (default: 0)
14   --verbose, -v     verbose output
15   --json, -j        json input/output
16   --no-headers, -H  display tables without headers
17   --help, -h        show help

What it shows

The command provides a comprehensive view of your cluster’s health and performance:

Performance and Health Section:

  • State: Overall cluster operational status with summary of affected nodes and issue types
  • Throughput: Current read/write throughput rates (only shown when active)
  • I/O Errors: Total disk I/O errors across all nodes
  • Load Avg: 1-minute load average aggregated across all nodes (avg, min, max)
  • Disk Usage: Average, minimum, and maximum disk usage percentages
  • Network: Network health status (healthy, degraded, etc.)
  • Storage: Total mountpaths and their health status
  • Filesystems: Types and counts of filesystems in use
  • Active Jobs: Currently running job types (download, rebalance, etc.)

Cluster Section:

  • Proxies: Number of proxy nodes and their electability status
  • Targets: Number of target nodes and total disks
  • Endpoint: Cluster endpoint URL for API access
  • Capacity: Used and available storage capacity with percentages
  • Cluster Map: Version, UUID, and primary node information
  • Software: Version and build information
  • Backend: Detected backend type (AWS, GCP, etc.)
  • Deployment: Deployment type (dev, K8s, etc.)
  • Status: Number of online nodes
  • Rebalance: Current rebalance status
  • Authentication: Authentication status (enabled/disabled)
  • Version: Software version information
  • Build: Build timestamp

Example Output

1$ ais show dashboard --refresh 5
2
3Performance and Health:
4   State:               Multiple issues (6 node(s) affected: 2 maintenance, 4 rebalancing)
5      Details:          Use '--verbose' for detailed breakdown
6   Throughput:          Read 2.4MiB/s, Write 13.2MiB/s (5s avg)
7   I/O Errors:          0
8   Load Avg:            avg 0.5, min 0.5, max 0.5 (1m)
9   Disk Usage:          avg 12.0%, min 12.0%, max 12.0%
10   Network:             healthy
11   Storage:             28 mountpaths (all healthy)
12   Filesystems:         ext4(28)
13   Active Jobs:         download, rebalance
14
15Cluster:
16   Proxies:             6 (all electable)
17   Targets:             6 (one disk)
18   Endpoint:            http://<cluster-endpoint>:<port>
19   Capacity:            used 53.03GiB (12%), available 364.47GiB
20   Cluster Map:         version 30, UUID Ba5eThUG9, primary p[rbap8080]
21   Software:            3.29.e230a780d (build: 2025-07-14T11:39:53-0700)
22   Backend:             AWS
23   Deployment:          dev
24   Status:              12 online
25   Rebalance:           -
26   Authentication:      disabled
27   Version:             3.29.e230a780d
28   Build:               2025-07-14T11:39:53-0700

Detailed Issue Breakdown

When cluster issues are detected, use the --verbose flag for a detailed breakdown:

1$ ais show dashboard --verbose
2
3Performance and Health:
4   State:               Multiple issues (6 node(s) affected: 2 maintenance, 4 rebalancing)
5   ...
6
7CLUSTER HEALTH DETAILS:
8Maintenance (2/6):   t[FFIt8090], t[zHut8091]
9Rebalancing (4/6):   t[ZHHt8087], t[atEt8086], t[UTat8088], t[xgAt8089]

Continuous Monitoring

Use the --refresh flag for continuous monitoring:

1# Monitor cluster every 5 seconds
2$ ais show dashboard --refresh 5
3
4# Monitor for 10 iterations (50 seconds total)
5$ ais show dashboard --refresh 5 --count 10
6
7# Monitor with JSON output
8$ ais show dashboard --refresh 10 --json

ais show auth

The following subcommands are currently supported:

1   cluster  show registered clusters
2   role     show existing user roles
3   user     show users or user details
4   config   show AuthN server configuration

Refer to ais auth documentation for details and examples.

ais show bucket

Show bucket properties.

Refer to ais bucket documentation for details and examples.

ais show object

Show object details.

Refer to ais object documentation for details and examples.

ais show storage

Show storage usage and utilization in the cluster. Show disks and mountpaths - for a single selected node or for all storage nodes.

When run with no subcommands, ais show storage defaults to ais show storage disk.

In addition, the command support the following subcommands:

1# ais show storage <TAB-TAB>
2disk        mountpath   capacity    summary

And with brief subcommand descriptions:

1$ ais show storage --help
2NAME:
3   ais show storage - show storage usage and utilization, disks and mountpaths
4
5USAGE:
6   ais show storage command [TARGET_ID] [command options]
7
8COMMANDS:
9   disk       show disk utilization and read/write statistics
10   mountpath  show target mountpaths
11   capacity   show target mountpaths, disks, and used/available capacity
12   summary    show bucket sizes and %% of used capacity on a per-bucket basis
13
14OPTIONS:
15   --refresh value  interval for continuous monitoring;
16                    valid time units: ns, us (or µs), ms, s (default), m, h
17   --count value    used together with '--refresh' to limit the number of generated reports (default: 0)
18   --json, -j       json input/output
19   --help, -h       show help

Refer to ais storage documentation for details and examples.

ais show config

Show daemon configuration.

Refer to ais cluster documentation for details and examples.

ais show remote-cluster

Show information about attached AIS clusters.

Refer to ais cluster documentation for details and examples.

ais show rebalance

Display details about the most recent rebalance xaction.

FlagTypeDescriptionDefault
--refreshdurationWatch global rebalance at a given refresh interval. The usual unit suffixes are supported and include m (for minutes), s (seconds), ms (milliseconds). Press Ctrl-C to stop monitoring.
--allboolIf set, show all rebalance xactionsfalse

Example

1$ ais show rebalance
2REB ID   NODE        OBJECTS RECV   SIZE RECV   OBJECTS SENT   SIZE SENT   START TIME       END TIME   ABORTED
3g1       CASGt8088   0              0B          0              0B          03-25 17:33:54   -          false
4g1       DMwvt8089   0              0B          0              0B          03-25 17:33:54   -          false
5g1       ejpCt8086   0              0B          0              0B          03-25 17:33:54   -          false
6g1       kiuvt8091   0              0B          0              0B          03-25 17:33:54   -          false
7g1       oGvbt8090   0              0B          0              0B          03-25 17:33:54   -          false
8g1       xZntt8087   0              0B          0              0B          03-25 17:33:54   -          false
9
10$ ais show rebalance
11REB ID   NODE        OBJECTS RECV   SIZE RECV   OBJECTS SENT   SIZE SENT   START TIME       END TIME         ABORTED
12g1       CASGt8088   0              0B          0              0B          03-25 17:33:54   03-25 17:34:09   false
13g1       DMwvt8089   0              0B          0              0B          03-25 17:33:54   03-25 17:34:08   false
14g1       ejpCt8086   0              0B          0              0B          03-25 17:33:54   03-25 17:34:08   false
15g1       kiuvt8091   0              0B          0              0B          03-25 17:33:54   03-25 17:34:08   false
16g1       oGvbt8090   0              0B          0              0B          03-25 17:33:54   03-25 17:34:08   false
17g1       xZntt8087   0              0B          0              0B          03-25 17:33:54   03-25 17:34:09   false
18
19Rebalance completed.

ais show log

There are 3 enumerated log severities and, respectively, 3 types of logs generated by each node:

  • error
  • warning
  • info

Example 1. Show “info” log:

1# Use <TAB-TAB> auto-completion to select a node (run `ais show cluster` to show details)
2$ ais show log
3p[f6ytNhIhb]   p[OqlWpgwrY] ...
4t[jkrt8Nkqi]   t[Juwzq371P] ...
5
6# Type `p[O`<TAB-TAB> to complete the node ID and then use `less` (for instance) to search, scroll or page down (or up), etc.
7$ ais show log p[OqlWpgwrY] | less
8
9Log file created at: 2021/04/11 10:58:38
10Running on machine: u18044
11Binary: Built with gc go1.15.3 for linux/amd64
12Log line format: L hh:mm:ss.uuuuuu file:line] msg
13I 10:58:38.122973 config.go:1611 log.dir: "/ais/log"; l4.proto: tcp; port: 51080; verbosity: 3
14...
15:

Example 2: show errors and/or warnings

By default, ais show log shows “info” log (that also contains all warnings and errors).

To show only errors, run:

1ais show log OqlWpgwrY --severity=error
2
3# or, same
4ais show log OqlWpgwrY --severity=e | more

For warnings and errors, run:

1ais show log OqlWpgwrY --severity=warning
2
3# or, same
4ais show log OqlWpgwrY --severity=w | less