desperate measures when you can't access your ubuntu server
okay let's say your server's networking just dies, you can access its serial and visual console...
...but you don't know root/user password(s). what would you do?
well, this is something a friend encountered with their two oracle cloud servers.
as a prerequisite, you need to reset the root password. unless you run a hardened system image, this should be doable with changing the boot command line to initialize bash, rather than whatever initsystem the image uses. this has been documented many times.
resetting the root password
to do this; first you need to generate the remote console keys for the virtual instance from the oracle cloud console.
then, connect to the vnc console using the long command oracle gives you. (for windows, you need to install the putty suite tools because plink is required; oh and a vnc client.)
the command should open port 5900 for your vnc client to connect to the console proper.
send a ctrl-alt-del signal or issue a reset from the cloud console.
hit esc right after texts above the tianocore logo shows up. hope that the grub menu appears. if it does, you can follow the many tutorials i linked above. if it doesn't, well, you're fucked.
the hail mary approach
...not that fucked, though.
if you hit esc early, you might've entered the uefi menu instead. for which instead of resetting you enter the boot menu to enter the efi shell. hit any key other than esc to drop into the shell itself.
run type fs0:\EFI\ubuntu\grub.cfg and note the value at the end of the line starting with search.
the efi shell should give a crude editor to mess with the grub config, so run edit fs0:\EFI\ubuntu\grub.cfg and add a space withinin the value of the configfile line. yes you're intentionally breaking grub here. by breaking the configline instruction; you're making grub partially set up to use the correct /boot (or /) partition.
reboot again, you should drop to a regular grub shell this time.
you can use source (the_partition,you_noted)/boot/grub/grub.cfg to load the system environment but not execute it.
run cat (the_partition,you_noted)/boot/grub/grub.cfg and note the boot and initrd lines, if you can see them.
you can then write those two lines by hand, but changing the ro to rw init=/bin/bash in the linux line like in the tutorials and removing the rest after that ro.
then run normal. you should be in bash in a moment. do the passwd trick (like in the tutorials) and then issue a ctrl-alt-del or a reset from the cloud console.
since you broke the config in the efi partition (esp) you will be dropped back into the grub shell, this time run configfile (the_partition,you_noted)/boot/grub/grub.cfg so that the system boots.
then just login with your new password, make sure that you run update-grub to fix the issues you created previously :p
double check if its fixed by checking /boot/efi/EFI/ubuntu/grub.cfg.
exporting files when you don't have working virtual cloud network
i don't know how can this happen @ oracle cloud but i've had first-hand experience, so yeah.
we wanted to export at least a home directory so we did something very cursed.
tarball the folder, reconnect to the console using the text-serial endpoint instead of vnc while piping the output to a file, and run base64.
it did take like 1-2 hours for 2 tarballs but it worked /shrug
Copyright 2018-2024, linuxgemini (İlteriş Yağıztegin Eroğlu). Any and all opinions listed here are my own and not representative of my employers; future, past and present.