pypi-mirror team mailing list archive

[Daniel Kraft <dk-launchpad@xxxxxx>]

>> 1. We register OURDOMAIN.org
>> 2. We setup multiple CNAMEs, one to each mirror like this:
>>    d9t.OURDOMAIN.org. CNAME pypi.d9t.de.
>>    zopyx.OURDOMAIN.org. CNAME pypi.zopyx.com.
> 
> I think mirror should be labeled in a more general way like
> mirror-a.OURDOMAIN.org, mirror-b.OURDOMAIN.org.

ok, I'm +-0.

> Playing load-balancing with the DNS is only have of the story since DNS
> lookups are usually cached on the OS level. The DNS might hand out a
> mirror that might not be available. 

Firefox and IE seem to clear DNS cache and lookup again (or at least
take another A Record from the list) upon timeout.
I haven't had a look at the setuptools code, but I think it should be
possible to hack something like that into it. Maybe even with a very
short timeout value (arount 10 seconds maybe?).
With that, a failure would only be noticed as a small delay.

For future enhancements, we might have a look at [1] to deliver
different zones to different IP locations (by assuming that the
requesting nameserver is in the same region as the requesting client).

> I think the recommended
> configuration for end-users right now would be to specifiy either
> OURMAIN.org within the find-links section (with the risk getting a
> crappy IP) or specify all mirrors explictly.

Did you mean index instead of find-links? Like
index =
    http://OURDOMAIN.org

I think this should be the only recommended solution - or maybe let them
define multiple index= records to select their mirrors themselves. But
we'd lose fault-tolerance again with only relying on one mirror.

Even the "multiple index = " recommendation would be dangerous. If more
mirrors join, we couldn't load balance due to stale configuration.

> For phase two of the project some kind of more smart mirror selection
> should be build-in somewhere (setuptools/zc.buildout).


[1] http://www.caraytech.com/geodns/

Daniel