Why is there "no new release found" during do-release-upgrade when using an intermediary TLS termination proxy?

The question best matching my situation is described here:

do-release-upgrade failing on Ubuntu 16.04 LTS - says: No new release found

... and questions linked therein; I'd have replied to one of those as an "alternative answer", but my SE rep is insufficient.

In rare cases, the issue stems from a TLS termination proxy being used, a fact that may be obscured from a particular VPS sysadmin, or out of reach to configure.

The issue manifests as follows:

% do-release-upgrade -c Checking for a new Ubuntu release Failed to connect to Check your Internet connection or proxy settings There is no development version of an LTS available. To upgrade to the latest non-LTS develoment release set Prompt=normal in /etc/update-manager/release-upgrades. 

However, retrieving the file manually works, e.g. on Ubuntu 18.04.5 LTS:

% curl -s | grep -A10 focal Dist: focal Name: Focal Fossa Version: 20.04.2 LTS Date: Thu, 23 April 2020 20:04:00 UTC Supported: 1 Description: This is the 20.04.2 LTS release Release-File: ReleaseNotes: ReleaseNotesHtml: UpgradeTool: UpgradeToolSignature: 

do-release-upgrade is written in Python using urllib, its urlopen is tripping up for some reason, and I can't determine why.

1 Answer

I've started with enabling debug mode:

% export DEBUG_UPDATE_MANAGER=yesplz % do-release-upgrade -c 

... and finding the following in the output:

... result of meta-release download: '<urlopen error Tunnel connection failed: 403 SSL CONNECT proxying not configured>' ... 

The first work-around I've tried is changing the URL in file /etc/update-manager/meta-release to have http:// instead of https://. This "works", but not something to take lightly.

I've also tried installing squid (a caching proxy) on the same machine and setting environment variables http[s]_proxy (lowercase and uppercase), guessing that an extra layer of software may cargo-cult releases into becoming reachable; but that didn't work (i.e. same error).

The last workaround - which works! - is downloading the file manually:

% curl -o /home/`whoami`/.cache/update-manager-core/meta-release-lts % sudo curl -o /var/lib/update-manager/meta-release-lts 

... and then forcing this file to be used, by editing /usr/lib/python3/dist-packages/UpdateManager/Core/MetaRelease.py and inserting at line 339, before an urlopen() attempt is made:

print('UGLY: urlopen my XSS! >:(') raise(HTTPError(code=304, url='', msg='', hdrs='', fp=open('/tmp/myXSS', 'w'))) 

So, to answer my own question, this is likely because of:

• the particular TLS terminating proxy configuration (that I don't have control over); or
• Python's urllib's urlopen not picking up the proxy settings, or the settings being incorrect (in the "squid workaround" case).