I don't but it might be due to some of the basic socket problems  
python has, specifically with regards to timeouts...

On Apr 23, 2008, at 4:27 PM, Paul Winkler wrote:

> Every once in a while, the build dies because something retrieved via
> urllib isn't complete. For example,
> http://pw.openplans.org:6020/builders/livablestreets-minimal/builds/153/steps/shell_7/logs/stdio
>
>  == Get opencore bundle tarball ==
>    Downloading tarball from
>    https://svn.openplans.org/eggs//openplans-bundle-opencore-plone25-20080418.tar.bz2
>    to
>    /home/pw/bbslave-0401/buildslave/livable-minimal/build/openplans- 
> bundle-opencore-plone25-20080418.tar.bz2
>  Error: Download url
>    https://svn.openplans.org/eggs//openplans-bundle-opencore-plone25-20080418.tar.bz2
>    failed! Expected 16435200 bytes; got 1556480 in file /home/pw/ 
> bbslave-0401/buildslave/livable-minimal/build/openplans-bundle- 
> opencore-plone25-20080418.tar.bz2
>
> Does anybody have any idea what causes this and how to fix it?
>
> -- 
>
> Paul Winkler
> http://www.openplans.org/people/slinkp/profile
> yahoo: slinkp23
> AIM:   slinkp1970
>
>
> --
> Archive: http://www.openplans.org/projects/fassembler/lists/fassembler-discussion/archive/2008/04/1208982417781
> To unsubscribe send an email with subject unsubscribe to fassembler-discussion@... 
> .  Please contact fassembler-discussion-manager@...  
> for questions.
>
>
> !DSPAM:4037,480f9be710541804284693!
>

Paul Winkler wrote:
> Every once in a while, the build dies because something retrieved via
> urllib isn't complete. For example,
> http://pw.openplans.org:6020/builders/livablestreets-minimal/builds/153/steps/shell_7/logs/stdio
> 
>   == Get opencore bundle tarball ==
>     Downloading tarball from
>     https://svn.openplans.org/eggs//openplans-bundle-opencore-plone25-20080418.tar.bz2
>     to
>     /home/pw/bbslave-0401/buildslave/livable-minimal/build/openplans-bundle-opencore-plone25-20080418.tar.bz2
>   Error: Download url
>     https://svn.openplans.org/eggs//openplans-bundle-opencore-plone25-20080418.tar.bz2
>     failed! Expected 16435200 bytes; got 1556480 in file /home/pw/bbslave-0401/buildslave/livable-minimal/build/openplans-bundle-opencore-plone25-20080418.tar.bz2
> 
> Does anybody have any idea what causes this and how to fix it?

I've noticed urllib specifically doesn't signal errors properly. 
urllib2 seems better.  Neither is setup to retry in case of a failure.

We could switch to using wget for this file specifically, as it's much 
more reliable about fetching files.  I.e., just run_command('wget', 
'-q', url, '-O', output_filename).

Here's something that does retries in Python: 
http://linux.duke.edu/projects/urlgrabber/help/urlgrabber.html

Maybe using one of these would fix it?  I guess it's not that surprising 
that fetching a particularly large file sometimes fails part way through.

   Ian