Micron Document
weird urllib.parse issues

the first problem

T383838urllib.parse.urljoin is broken. Or at least does weird things with gemini URLs.

I wanted to join two urls as a means of resolving relative links by using the current address as the base. Urllib.parse urljoin can do this very well for anything that's not gemini. It does a great job of removing stuff that's not part of a folder name in a base location and then tacking on the relative name.

It even handles a bunch of edge cases. It's really handy. But less so for gemini.

Note:

urljoin("http://example.com", "foo.gmi") -> http://example.com/foo.gmi
urljoin("gemini://example.com", "foo.gmi") -> foo.gmi

Let's say I want to use a dumb hack to work around it:

T282828
Te6edf3parsed_base Tff7b72= Te6edf3urlparseTb4b4b4(Te6edf3base_urlTb4b4b4)
Te6edf3saved_scheme Tff7b72= Te6edf3parsed_baseTff7b72.Td2a8ffscheme

Te6edf3fake_base Tff7b72= Te6edf3parsed_baseTff7b72.Td2a8ff_replaceTb4b4b4(Te6edf3schemeTff7b72=Ta5d6ff"Ta5d6ffhttpTa5d6ff"Tb4b4b4)
Te6edf3fake_base Tff7b72= Te6edf3fake_baseTff7b72.Td2a8ffgeturlTb4b4b4(Tb4b4b4)

Te6edf3parsed_relative Tff7b72= Te6edf3urlparseTb4b4b4(Te6edf3relative_urlTb4b4b4)
Te6edf3joined Tff7b72= Te6edf3urljoinTb4b4b4(Te6edf3fake_baseTb4b4b4, Te6edf3relative_urlTb4b4b4)
Te6edf3parsed_result Tff7b72= Te6edf3urlparseTb4b4b4(Te6edf3joinedTb4b4b4)

T8b949e# revert dumb hack here
Tff7b72if Tff7b72not Te6edf3parsed_relativeTff7b72.Td2a8ffschemeTb4b4b4:
Te6edf3parsed_result Tff7b72= Te6edf3parsed_resultTff7b72.Td2a8ff_replaceTb4b4b4(Te6edf3schemeTff7b72=Te6edf3saved_schemeTb4b4b4)
Te6edf3url Tff7b72= Te6edf3parsed_resultTff7b72.Td2a8ffgeturlTb4b4b4(Tb4b4b4)


So given the inputs from earlier:

gemini://example.com and foo.gmi

The value of url is now:

gemini:example.com/foo.gmi

So close!

So, another hack:

if re.match("gemini:[A-Za-z0-9]", url):
url = "gemini://" + url[7:]
return url

And now the result is finally:

gemini://example.com/foo.gmi

Putting it all together:

def makelink(baseurl, relative_url):
"""
Joins a relative URL to a base URL, ensuring a path exists and
preserving the original scheme.
"""
parsedbase = urlparse(baseurl)
savedscheme = parsedbase.scheme

# Step 1:
# urljoin fouls up gemini:// scheme but works on http:// scheme,
# so pretend every link is http and swap them back after.
fakebase = parsedbase._replace(scheme="http")
fakebase = fakebase.geturl()

parsedrelative = urlparse(relativeurl)
joined = urljoin(fakebase, relativeurl)
parsed_result = urlparse(joined)

# Revert step 1 here:
if not parsed_relative.scheme:
parsedresult = parsedresult.replace(scheme=savedscheme)
url = parsed_result.geturl()

##
# Step 2 - it also fouls up the return in creating a url, it kind of looks
# like a mailto: with no // in it.
if re.match("gemini:[A-Za-z0-9]", url):
url = "gemini://" + url[7:]
return url

the second problem

When calling urlparse, return values are inconsistent for netloc and path when the scheme is absent vs when it is present.

urlparse('gemini://www.example.com') -> netloc = www.example.com
urlparse('www.example.com') -> netloc = '', path = 'www.example.com'

It gets a little worse:

urlparse('www.example.com/foo/bar.gmi') -> netloc='', path='www.example.com/foo/bar.gmi'

I dont have a tested workaround for this, but maybe something like this:

T282828
Tff7b72def Td2a8ffhacky_parseTb4b4b4(Te6edf3urlTb4b4b4)Tb4b4b4:
Te6edf3parsed Tff7b72= Te6edf3urlparseTb4b4b4(Te6edf3urlTb4b4b4)
Tff7b72if Tff7b72not Te6edf3parsedTff7b72.Td2a8ffschemeTb4b4b4:
Te6edf3terrible Tff7b72= Ta5d6ff"Ta5d6ffhttp://Ta5d6ff" Tff7b72+ Te6edf3url
Te6edf3parsed Tff7b72= Te6edf3urlparseTb4b4b4(Te6edf3terribleTb4b4b4)
Te6edf3parsed Tff7b72= Te6edf3parsedTff7b72.Td2a8ff_replaceTb4b4b4(Te6edf3schemeTff7b72=Ta5d6ff'Ta5d6ff'Tb4b4b4)
Tff7b72return Te6edf3parsed


Which at least puts things where I would expect them:

hacky_parse("www.example.com/foo/bar.gmi")

Results in:

netloc='www.example.com', path='/foo/bar.gmi'

Tags: python

Tags
Navigation



created: 2026-02-16

(re)generated: 2026-07-17