I just finished the rescue process after unsuccessful upgrade from Ubuntu 18.04 to 18.10. The process hanged, leaving me with half upgraded unbootable system. I believe the culprit to be the LXD, specifically the failed transition from deb to snap. Luckily for me, Linux already has taught me to do backups and thanks to LVM I only wasted 15 minutes reinstalling grub from LiveCD.
Steps to reproduce:
backup
apt update
apt dist-upgrade
reboot
do-release-upgrade
Here is the last visible terminal output before the process hanged:
dpkg: error processing archive /tmp/apt-dpkg-install-JR1TQB/107-lxd_1%3a0.4_all.deb (--unpack):
new lxd package pre-installation script subprocess returned error exit status 1
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/apt/cache.py", line 265, in __getitem__
rawpkg = self._cache[key]
KeyError: '107-lxd'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 126, in _apt_pkg
return self._cache()[package]
File "/usr/lib/python3/dist-packages/apt/cache.py", line 267, in __getitem__
raise KeyError('The cache has no package named %r' % key)
KeyError: "The cache has no package named '107-lxd'"
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/share/apport/package_hook", line 48, in <module>
pr['SourcePackage'] = apport.packaging.get_source(options.package)
File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 159, in get_source
if self._apt_pkg(package).installed:
File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 128, in _apt_pkg
raise ValueError('package %s does not exist' % package)
ValueError: package 107-lxd does not exist
I will gladly report it in a proper place if @stgraber or @simos would be kind enough to point me where.
This constitutes a bug report, likely on the lxd transition package in Ubuntu 18.10.
The referenced deb package /tmp/apt-dpkg-install-JR1TQB/107-lxd_1%3a0.4_all.deb has a hex character for :, so it’s really /tmp/apt-dpkg-install-JR1TQB/107-lxd_1:0.4_all.deb. Looks weird, but still the error is not about bad filename but just key errors in Python.
As a workaround, I would suggest to migrate from the deb package to the snap package, then upgrade from 18.04.1 to 18.10.
Yep. After restore, I simply purged lxd package (wasn’t using it on that particular machine anyway - maybe that was also a problem, that there was no configuration preseed by lxd init?) and then followed same steps. The rest of the upgrade went almost* flawless.
@simos, do you want me to report it as a bug? If yes, then where? Launchpad, github?
I was kicked out from X session and had to restart but that was probably due to me having replaced gnome with unity desktop.
mmmm… sorry, after failed upgrade I did a rollback via LVM. The only log I have is the part you see above that I just copy-pasted from terminal to a file on another disk. Recent update log won’t be useful since it went without lxd package at all. If you find it useful I can (later this or next week) spinup a VM with vanilla bionic and check how it goes again.
Pretty sure vanilla bionic will work fine with and without containers, as that’s what we’ve been using for our upgrade tests and what’s part of the automated CI.
So you most likely hit some odd edge case, either in the upgrade tool itself, or hit a snapd issue in the middle of the upgrade (though that’d have made it hang and print a retry message normally) or hit something weirder (I/O issue, apt ordering problem, …). Without either a reliable way to reproduce this or an upgrade log, it’s going to be pretty hard to figure out what happened.
I’d need the terminal output of the upgrader, it should show the lxd package upgrade followed by the automated migration code that will pull in the LXD snap.
If the upgrade fails, you should still be able to then attempt a manual upgrade with apt dist-upgrade and see only the failing packages be retried.
Normally I just manually select the output with a mouse and copy-paste it to a file. I’m afraid output od dist-upgrade might exceed the terminal limit of lines displayed. Is there a way to log the output to external file? Would simple do-release-upgrade > /home/upgrade.log redirection work?