How to connect via SSH to EC2 instance that got unreachable?

Having an AWS EC2 instance running Discourse for several years.

Since some weeks/months it occurs that after some Discourse upgrades (through the web GUI of Discourse), the system stalls. It is then not reachable via SSH anymore and not even via EC2 SSH web connect.

I ususally did a restart of the whole machine via the AWS ECS web admin panel and then was able to connect again, both via SSH and HTTPS.

(I did not further investigate the cause of the issue, unfortunately).

Today, the system hung again after a Discourse upgrade.

But this time, no reboot or whatsoever did help. The system stays unreachable via SSH, even after waiting for several hours.

The EC2 monitoring also shows no high server load to me:

My question

How can I gain access via SSH to the EC2 Ubuntu instance again?

Update 1

After yet another reboot and some wait time, the system is back again, out of nowhere again.

To me, it seems that the disk has enough free space (no more disk space was one of my assumptions why the system was unreachable).

Filesystem Size Used Avail Use% Mounted on
udev 2.0G 0 2.0G 0% /dev
tmpfs 394M 876K 393M 1% /run
/dev/xvda1 97G 31G 67G 31% /
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
/dev/loop1 18M 18M 0 100% /snap/amazon-ssm-agent/1566
/dev/loop0 98M 98M 0 100% /snap/core/10185
/dev/loop2 29M 29M 0 100% /snap/amazon-ssm-agent/2012
/dev/loop3 98M 98M 0 100% /snap/core/10126
overlay 97G 31G 67G 31% /var/lib/docker/overlay2/5a799ab040002ad2ddec94ae85bcbe987543651a0d9478ddc12ab12715da7340/merged
tmpfs 394M 0 394M 0% /run/user/1000

I'm now trying to figure out whether there are any logs that indicate the reason for the outage.

Update 2

As of request by heynnema, here is the full result of sudo cat /etc/netplan/*.yaml:

# This file is generated from information provided by
# the datasource. Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network: version: 2 ethernets: eth0: dhcp4: true match: macaddress: 06:a0:68:ea:54:4e set-name: eth0

1 Answer

For Amazon AWS EC2 you need to use a .yaml like this. SSH will work then.

# This file is generated from information provided by the datasource. Changes
# to it will not persist across an instance reboot. To disable cloud-init's
# network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network: version: 2 renderer: networkd ethernets: eth0: match: macaddress: 06:a0:68:ea:54:4e set-name: eth0 bridges: br0: dhcp4: true macaddress: 06:a0:68:ea:54:4e interfaces: - eth0

Note: You may need to create /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg

network: {config: disabled}

sudo netplan --debug generate
sudo netplan apply
reboot # mandatory

Check this link for further information.