Proxmox VE and the case of an unresponsive GUI
For the second time this year, our old FreeNAS storage platform went offline and left our Proxmox VE 6.4 servers in a continuous state of distress. This issue renders the Proxmox VE GUI unresponsive and thus unable to display information about the virtual servers it’s running.
In addition to an unresponsive GUI, there were also several backup jobs spinning into eternity. We use the NFS (Network File System) protocol to access files and directories on the FreeNAS storage platform. That works as expected, as long as the FreeNAS server is online. When the storage platform goes offline, NFS requests are retried indefinitely, and here we are.
Disabling the NFS Backend
Proxmox lets you disable storage backends from the GUI’s “Storage” menu, but in this case, the request to disable the NFS backend would only result in a time-out error.
Using the pvesm (Proxmox VE Storage Manager) command to disable the storage worked as expected. The id of the storage is nasse01, as can be seen on the screenshot above.
pvesm set nasse01 --disable 1
Killing the hung backup jobs
The next step is to get rid of all the backup jobs currently frozen in time. Discover the tasks by issuing ps aux | grep vzdump
and kill them off one by one by issuing kill -9 PID
.
Restarting Proxmox services
The final step is to restart the Proxmox VE services required to kick everything back in motion.
systemctl restart pvedaemon
systemctl restart pvestatd
systemctl restart pveproxy
I should probably have another look at the NFS configuration to avoid this problem in the future. Provided I can get the FreeNas server online again, that is.