pypi-mirror team mailing list archive

[Andreas Jung <lists@xxxxxxxxx>]

On 03.11.2008 23:42 Uhr, Daniel Kraft wrote:
> Hi,
>
> I was just thinking about the problem that many packages only provide a
> download URL instead of the files on pypi. We can sync some of them
> (easily if they provide direct download links and ugly/heavily if they
> provide a download-list-page-URL). This code is ready but *very* slow,
> because there are no hints like md5 checksums on foreign sites. In
> addition those sites are sometimes *extremely* slow or even down.

We should not focus on this problem right now (we should mirror packages 
for phase 1 only that are directly hosted on PyPI).

My ideas:

 - maintain a dedicated whitelist of externally-hosted
   packages that are of
   interest for the Zope world (e.g. the Openid stuff often has
   failures). The list could be hosted on a central server and fetched
   from the pypimirror script before starting the update dance

 - seperate the standard PyPI update run from the one where we deal with
   externally-hosted packages. By doing this we can have different
   settings and update frequencies for packages hosted on PyPI and
   the others

 - talk with the maintainers of "important" packages (like 
python-openid) in order to put their packages on PyPI (at least in addition
   to their own primary servers) in order for having a mirror. It's 
about  convincing the people about the advantages having a PyPI mirror


Andreas

--
ZOPYX Ltd. & Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany
Web: www.zopyx.com - Email: info@xxxxxxxxx - Phone +49 - 7071 - 793376
Registergericht: Amtsgericht Stuttgart, Handelsregister A 381535
Geschäftsführer/Gesellschafter: ZOPYX Limited, Birmingham, UK
------------------------------------------------------------------------
E-Publishing, Python, Zope & Plone development, Consulting

begin:vcard
fn:Andreas Jung
n:Jung;Andreas
org:ZOPYX Ltd. & Co. KG
adr;quoted-printable:;;Charlottenstr. 37/1;T=C3=BCbingen;;72070;Germany
email;internet:info@xxxxxxxxx
title:CEO
tel;work:+49-7071-793376
tel;fax:+49-7071-7936840
tel;home:+49-7071-793257
x-mozilla-html:FALSE
url:www.zopyx.com
version:2.1
end:vcard