In scenarios where a deeper level of access is needed, administrators have the abiltiy to access the local administrative console of managed systems. Fleet Command systems run a customized operating system based on Linux and the remote console should be familiar to users of Linux. The Remote Console feature gives Fleet Command administrators the ability to remotely start and access interactive shell sessions for each system allowing them to gather detailed information not available through higher level tooling. To support admin teams, multiple administrators can access the system through the remote console during troubleshooting sessions. Remote Console will create a unique shell session within the system for each user.
Use caution when using this feature. After accessing a Fleet Command system as a privileged user (root) the integrity of the system can no longer be assured and will be flagged in the console. Always begin troubleshooting by reviewing the logs and limit access to during “break glass” scenarios.
Remote Console was enabled as part of Remote Management in Step 2.
Let’s login to the system’s remote console and gather some information directly.
To get started, navigate to Locations in the navigation menu on the left
Click on on your location to navigate to the details page.
Access the Remote Console from the system ellipsis and click Start Remote Console.
After successful authentication, a session is created in the background. A pop-up will display the remote console with the system name and return to the system with a blue banner message with information about the Remote Console. Click Open.
A browser tab will open with a terminal session for the rcuser. The rcuser is capable of running unprivileged commands, like nvidia-smi, to gather information.
Run
nvidia-smito view details about the system’s GPUrcuser@system-1:~$ nvidia-smi
Fleet Commmand systems have a reserved set of users for different roles. Review the below table that describes the available users for future use.
Console Users USER
Description
rcuser
Default basic user capable of running unprivileged commands
admin
Basic user capable of elevating privileges using the sudo utility.
root
Built-in superuser with unrestricted access.
Become the admin user with the su utility so we can run some privileged commands.
rcuser@system-1:~$ su admin Password: *Enter the Administrator password used during installation*
Run a privileged command with the sudo utility. Here, we are simply gathering some privileged information but can perform more sensetive tasks if needed. Use caution when running commands with sudo.
Get the status of all the kubernetes pods on the system
admin@system-1:~$ sudo kubectl get pods -A [sudo] password for admin:
Optional: The root user is a superuser in linux based systems that has unrestricted access to the system. It is sometimes necessarry to operate as the root user to access the more protected areas of the system for troubleshooting. Open an interactive root session to gain privileged access to tools like kubectl to debug the system.
After switching to root user, you must re-image the location to return it to its initial state.
A warning banner will be displayed on the system in the Fleet Command console recommending reinstallation. This is normal and a recommended practice to ensure the integrity of edge deployed systems. While the system will remain operational and allow you to complete the rest of the tasks, it is not possible to remove the banner message with without redeploying the system.
admin@system-1:~$ sudo su
[sudo] password for admin:
root@system-1:~$ kubectl get pods -A
Now that we know how we can manage the systems, lets optimize our system so we can make better use of it’s resources.