pypi-mirror team mailing list archive

[Andreas Jung <lists@xxxxxxxxx>]

On 10.10.2008 10:44 Uhr, Daniel Kraft wrote:
> Hi,
>
> here's a proposal for how we could make our cluster HA without effort
> and even be able to use one specific mirror:
>
> 1. We register OURDOMAIN.org
> 2. We setup multiple CNAMEs, one to each mirror like this:
>    d9t.OURDOMAIN.org. CNAME pypi.d9t.de.
>    zopyx.OURDOMAIN.org. CNAME pypi.zopyx.com.

I think mirror should be labeled in a more general way like
mirror-a.OURDOMAIN.org, mirror-b.OURDOMAIN.org.

>    ...
> 3. Additionally, we setup OURDOMAIN.org with multiple A-Records, each
> pointing to one mirror.

+1
> That way we can use one mirror via
>    <name>.OURDOMAIN.org
> plus we can give OURDOMAIN.org as a HA mirror to anyone.
> If setuptools behaves like a browser, it will just use one or even the
> first of the A Records. As DNS answers with randomly sorted A records,
> the cluster should load-balance quite well.

Playing load-balancing with the DNS is only have of the story since DNS 
lookups are usually cached on the OS level. The DNS might hand out a 
mirror that might not be available. I think the recommended 
configuration for end-users right now would be to specifiy either 
OURMAIN.org within the find-links section (with the risk getting a 
crappy IP) or specify all mirrors explictly.

For phase two of the project some kind of more smart mirror selection 
should be build-in somewhere (setuptools/zc.buildout).


>

> If we want to keep requests in a country/region, we can pimp the DNS a
> little by sending different lists of A-Records depending of the
> requesting IP address. Although I think we won't need this right now ;)

see above.

Andreas

begin:vcard
fn:Andreas Jung
n:Jung;Andreas
org:ZOPYX Ltd. & Co. KG
adr;quoted-printable:;;Charlottenstr. 37/1;T=C3=BCbingen;;72070;Germany
email;internet:info@xxxxxxxxx
title:CEO
tel;work:+49-7071-793376
tel;fax:+49-7071-7936840
tel;home:+49-7071-793257
x-mozilla-html:FALSE
url:www.zopyx.com
version:2.1
end:vcard