How does minikube ssh work for hyperkit VMs on macOS?

How does minikube ssh work for hyperkit VMs on macOS?

Before we see how the behind-the-scenes thing works, let's look at how to SSH into the minikube VM. You can do it like this -

First, start the minikube VM with hyperkit driver

# Run the minikube VM like this
mk start --driver hyperkit

If you don't have hyperkit, then install it using brew (Homebrew)

brew install hyperkit

Next, let's SSH into the minikube VM

minikube ssh

And the output would look like this -

$ minikube ssh
                         _             _            
            _         _ ( )           ( )           
  ___ ___  (_)  ___  (_)| |/')  _   _ | |_      __  
/' _ ` _ `\| |/' _ `\| || , <  ( ) ( )| '_`\  /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )(  ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____)

$

Now let's look at how this thing works behind the scenes :)

Now, let's think about this - what information does one require to SSH into a machine? Remote or local, regardless

One would need to know the IP address of the server running in the machine (at which it's listening at); one would need to know the port at which the SSH server is running; we also need to know the SSH credentials - for example SSH username and password, or something like SSH username and SSH private key, if the credential is a key. By the way, this is all assuming that one can SSH into the machine - that is, there's an SSH server running at a given port and the port is accessible from the machine we have, with no firewall issues etc

Let's try to get all of this information now

Let's look at what minikube profile we are using first

minikube profile list

You can see output like this -

$ mk profile list
|----------|-----------|---------|--------------|------|---------|---------|-------|--------|
| Profile  | VM Driver | Runtime |      IP      | Port | Version | Status  | Nodes | Active |
|----------|-----------|---------|--------------|------|---------|---------|-------|--------|
| minikube | hyperkit  | docker  | 192.168.65.2 | 8443 | v1.27.4 | Running |     1 | *      |
|----------|-----------|---------|--------------|------|---------|---------|-------|--------|

We have a profile named minikube profile in my case. It's a default profile name I think, when no name is used during minikube start command

Look for the profile's machine details under ~/.minikube/machines/<profile-name>/config.json . For example, for minikube profile, in my case, it looks like this - ~/.minikube/machines/minikube/config.json -

{
    "ConfigVersion": 3,
    "Driver": {
        "IPAddress": "192.168.65.2",
        "MachineName": "minikube",
        "SSHUser": "docker",
        "SSHPort": 22,
        "SSHKeyPath": "/Users/karuppiah/.minikube/machines/minikube/id_rsa",
        "StorePath": "/Users/karuppiah/.minikube",
        "SwarmMaster": false,
        "SwarmHost": "",
        "SwarmDiscovery": "",
        "Boot2DockerURL": "file:///Users/karuppiah/.minikube/cache/iso/amd64/minikube-v1.31.0-amd64.iso",
        "DiskSize": 20000,
        "CPU": 2,
        "Memory": 4000,
        "Cmdline": "loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=minikube",
        "NFSShares": [],
        "NFSSharesRoot": "/nfsshares",
        "UUID": "7aa1c19a-6834-11ee-a8ca-acde48001122",
        "VpnKitSock": "",
        "VSockPorts": [],
        "ExtraDisks": 0
    },
    "DriverName": "hyperkit",
    "HostOptions": {
        "Driver": "",
        "Memory": 0,
        "Disk": 0,
        "EngineOptions": {
            "ArbitraryFlags": null,
            "Dns": null,
            "GraphDir": "",
            "Env": null,
            "Ipv6": false,
            "InsecureRegistry": [
                "10.96.0.0/12"
            ],
            "Labels": null,
            "LogLevel": "",
            "StorageDriver": "",
            "SelinuxEnabled": false,
            "TlsVerify": false,
            "RegistryMirror": [],
            "InstallURL": "https://get.docker.com"
        },
        "SwarmOptions": {
            "IsSwarm": false,
            "Address": "",
            "Discovery": "",
            "Agent": false,
            "Master": false,
            "Host": "tcp://0.0.0.0:3376",
            "Image": "swarm:latest",
            "Strategy": "spread",
            "Heartbeat": 0,
            "Overcommit": 0,
            "ArbitraryFlags": null,
            "ArbitraryJoinFlags": null,
            "Env": null,
            "IsExperimental": false
        },
        "AuthOptions": {
            "CertDir": "/Users/karuppiah/.minikube",
            "CaCertPath": "/Users/karuppiah/.minikube/certs/ca.pem",
            "CaPrivateKeyPath": "/Users/karuppiah/.minikube/certs/ca-key.pem",
            "CaCertRemotePath": "",
            "ServerCertPath": "/Users/karuppiah/.minikube/machines/server.pem",
            "ServerKeyPath": "/Users/karuppiah/.minikube/machines/server-key.pem",
            "ClientKeyPath": "/Users/karuppiah/.minikube/certs/key.pem",
            "ServerCertRemotePath": "",
            "ServerKeyRemotePath": "",
            "ClientCertPath": "/Users/karuppiah/.minikube/certs/cert.pem",
            "ServerCertSANs": null,
            "StorePath": "/Users/karuppiah/.minikube"
        }
    },
    "Name": "minikube"
}

It contains all the details, like SSH key path, SSH server port, SSH username, SSH server IP address

...
    "Driver": {
        "IPAddress": "192.168.65.2",
        "MachineName": "minikube",
        "SSHUser": "docker",
        "SSHPort": 22,
        "SSHKeyPath": "/Users/karuppiah/.minikube/machines/minikube/id_rsa",
...

I found the SSH key just by experimenting and looking for folders and files named ssh and similar in my machine. I found the private key credential for SSHing into the machine at ~/.minikube/machines/minikube/id_rsa , which is mentioned in the profile config JSON

I found the username by using whoami after getting into the machine with minikube ssh and it's the same one mentioned in the profile config JSON. I checked IP using minikube profile list table. You can also find the IP using ifconfig after getting into the machine with minikube ssh, like this -

$ minikube ssh
                         _             _            
            _         _ ( )           ( )           
  ___ ___  (_)  ___  (_)| |/')  _   _ | |_      __  
/' _ ` _ `\| |/' _ `\| || , <  ( ) ( )| '_`\  /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )(  ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____)

$ whoami
docker
$
$ ifconfig 
bridge    Link encap:Ethernet  HWaddr 9A:C4:47:0F:A9:8C  
          inet addr:10.244.0.1  Bcast:10.244.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1078 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1183 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:88539 (86.4 KiB)  TX bytes:118699 (115.9 KiB)

docker0   Link encap:Ethernet  HWaddr 02:42:DE:3F:40:E0  
          inet addr:172.17.0.1  Bcast:172.17.255.255  Mask:255.255.0.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth0      Link encap:Ethernet  HWaddr 66:8F:EF:A0:31:45  
          inet addr:192.168.65.2  Bcast:192.168.65.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:295237 errors:0 dropped:0 overruns:0 frame:0
          TX packets:15928 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:432434274 (412.4 MiB)  TX bytes:1653741 (1.5 MiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:175877 errors:0 dropped:0 overruns:0 frame:0
          TX packets:175877 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:30983261 (29.5 MiB)  TX bytes:30983261 (29.5 MiB)

veth1325a6d7 Link encap:Ethernet  HWaddr 96:06:58:31:BD:D7  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1079 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1183 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:103673 (101.2 KiB)  TX bytes:118687 (115.9 KiB)

$

These details are also in the profile config JSON. The SSH server port is 22, which is the default port.

So, with all that information, let's see how minikube ssh works behind the scenes. We have the private key, we have the IP address along with the port number and we have the username too. Finally, it looks like this -

ssh -i ~/.minikube/machines/minikube/id_rsa docker@192.168.65.2
$ ssh -i ~/.minikube/machines/minikube/id_rsa docker@192.168.65.2
                         _             _            
            _         _ ( )           ( )           
  ___ ___  (_)  ___  (_)| |/')  _   _ | |_      __  
/' _ ` _ `\| |/' _ `\| || , <  ( ) ( )| '_`\  /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )(  ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____)

$

So, that's how minikube ssh works in MacOS with Hyperkit VMs