On Mon, 27 Dec 2010 15:02:19 -0500, corvid <corvid@lavabit.com> wrote:
If we had our own downloader, 1) Downloads would be rejected less often when servers wouldn't see Wget in the user agent string anymore.
Admittedly, there is wget's --user-agent option, though that doesn't resolve the overall ugliness of dependency on an external program. That reminds me, I should have it use prefs.http_user_agent rather than the hard-coded string, since that can be overridden and it would look funny having two different user agents. My own builds use a Mozilla/4.0 agent, which is ugly but makes sites like Google behave a little better.
2) Maybe the matter of proxies would be a bit less tangled.
Libcurl obeys $http_proxy, but not prefs.http_proxy (yet). I don't think curl accepts the full URL_STR, so it would have to set the host, port, etc. from the individual DilloUrl fields.
I wonder whether libcurl makes it simple to retry and continue partial downloads. (I wish dillo itself had some ability to do that, but I don't want to battle with CCC.)
Judging from [1], it looks pretty simple: open the file in append mode, and set CURLOPT_RESUME_FROM to the number of bytes already downloaded. Shall I go ahead and add it in? [1] http://curl.haxx.se/libcurl/c/curl_easy_setopt.html ~Benjamin