January 2010 - Dillo-dev - lists.mailman3.com

some code for better cookie rejection

by corvid＠lavabit.com

Yesterday I made a rough draft of some code to allow the use of the http://publicsuffix.org/list/ to determine which domains dillo shouldn't allow cookies to be set for: http://www.dillo.org/test/public_suffix.diff I just put it there for safekeeping for now, as I have no plans to clean it up and get it into dillo until after the next release, but it occurred to me that I should at least mention it's there for anyone who uses cookies and lives somewhere where the domain situation is complicated (e.g., Japan). A user would have to add a line to cookiesrc like public_suffix_file /path/to/list/file

Jan. 16, 2010

Cookies performance

by Johannes.Hofmann＠gmx.de

On Fri, Jan 15, 2010 at 06:20:41PM +0900, furaisanjin wrote: > 2010/1/14 Johannes Hofmann <Johannes.Hofmann(a)gmx.de>: > > Ok, could you please try this combination: > > > > - TCP_NODELAY not included (e.g. hg up 239f9fc6c313) > > - cookies disabled > > - proxy is used > > > > My guess is that this combination also shows the problem, but of > > course we need to verify this. > > Yes, I see the same error messages with above combination, so > definitely this is communication problem between dillo and my local > proxy. I guess that dillo creates socket connection too often. Ok good. I'm working on a connection limit right now and hope to have something to test soon. Thanks for the continuous testing, Johannes

Jan. 15, 2010

Cookies performance

by furaisanjin＠gmail.com

2010/1/15 corvid <corvid(a)lavabit.com>: > Okay, committed a fix for --disable-cookies. I checked myself and now I can compile with disabling cookies. Regards, furaisanjin

Jan. 15, 2010

Cookies performance

by furaisanjin＠gmail.com

2010/1/14 Johannes Hofmann <Johannes.Hofmann(a)gmx.de>: > Ok, could you please try this combination: > > - TCP_NODELAY not included (e.g. hg up 239f9fc6c313) > - cookies disabled > - proxy is used > > My guess is that this combination also shows the problem, but of > course we need to verify this. Yes, I see the same error messages with above combination, so definitely this is communication problem between dillo and my local proxy. I guess that dillo creates socket connection too often. Regards, furaisanjin

Jan. 15, 2010

Cookies performance

by corvid＠lavabit.com

furaisanjin wrote: > I did gmake clean and then ./configure --disable-cookies. I compiled > codes and found that I had to modify like this. Okay, committed a fix for --disable-cookies.

Jan. 14, 2010

Cookies performance

by Johannes.Hofmann＠gmx.de

On Thu, Jan 14, 2010 at 11:04:03PM +0900, furaisanjin wrote: > 2010/1/14 Johannes Hofmann <Johannes.Hofmann(a)gmx.de>: > > Is this without TCP_NODELAY? > > The code contains TCP_NODELAY since I don't have any problem if I > remove TCP_NODELAY. > > Now I tried > - cookie is enabled > - TCP_NODELAY is included > - no proxy is used > > Then I don't any see error message. So the problem seems to be > communication between dillo and my local proxy. Ok, could you please try this combination: - TCP_NODELAY not included (e.g. hg up 239f9fc6c313) - cookies disabled - proxy is used My guess is that this combination also shows the problem, but of course we need to verify this. Regards, Johannes

Jan. 14, 2010

Cookies performance

by furaisanjin＠gmail.com

2010/1/14 Johannes Hofmann <Johannes.Hofmann(a)gmx.de>: > Is this without TCP_NODELAY? The code contains TCP_NODELAY since I don't have any problem if I remove TCP_NODELAY. Now I tried - cookie is enabled - TCP_NODELAY is included - no proxy is used Then I don't any see error message. So the problem seems to be communication between dillo and my local proxy. Regards, furaisanjin

Jan. 14, 2010

Cookies performance

by Johannes.Hofmann＠gmx.de

On Thu, Jan 14, 2010 at 09:40:58PM +0900, furaisanjin wrote: > 2010/1/14 Johannes Hofmann <Johannes.Hofmann(a)gmx.de>: > > Could you try with cookies disabled and a dillo version before > > the TCP_NODELAY change? > > I did gmake clean and then ./configure --disable-cookies. I compiled > codes and found that I had to modify like this. > > diff -r 9049a634eb0d src/cookies.c > --- a/src/cookies.c Wed Jan 13 15:27:56 2010 +0900 > +++ b/src/cookies.c Thu Jan 14 21:33:37 2010 +0900 > @@ -22,7 +22,11 @@ > */ > void a_Cookies_init(void) > { > - MSG("Cookies: absolutely disabled at compilation time.\n"); > + printf("Cookies: absolutely disabled at compilation time.\n"); > +} > + > +void a_Cookies_freeall() > +{ > } > > I stopped dpid and installed cookie dpi just in case, then tried dillo > again. I feel I see more error messages by this change. Is this without TCP_NODELAY? If so it would indicate that the real problem is the massive number of connections dillo is establishing if there are man images on a page and some servers / proxies don't like it. We probabely need to look at the experimental patch Jorge had posted here: http://lists.auriga.wearlab.de/pipermail/dillo-dev/2009-August/006778.html Cheers, Johannes

Jan. 14, 2010

Cookies performance

by furaisanjin＠gmail.com

2010/1/14 Johannes Hofmann <Johannes.Hofmann(a)gmx.de>: > Could you try with cookies disabled and a dillo version before > the TCP_NODELAY change? I did gmake clean and then ./configure --disable-cookies. I compiled codes and found that I had to modify like this. diff -r 9049a634eb0d src/cookies.c --- a/src/cookies.c Wed Jan 13 15:27:56 2010 +0900 +++ b/src/cookies.c Thu Jan 14 21:33:37 2010 +0900 @@ -22,7 +22,11 @@ */ void a_Cookies_init(void) { - MSG("Cookies: absolutely disabled at compilation time.\n"); + printf("Cookies: absolutely disabled at compilation time.\n"); +} + +void a_Cookies_freeall() +{ } I stopped dpid and installed cookie dpi just in case, then tried dillo again. I feel I see more error messages by this change. Regards, furaisanjin

Jan. 14, 2010

Cookies performance

by Johannes.Hofmann＠gmx.de

On Thu, Jan 14, 2010 at 08:24:54PM +0900, furaisanjin wrote: > 2010/1/14 Johannes Hofmann <Johannes.Hofmann(a)gmx.de>: > > What page is that, or does it happen on all pages with many images? > > Is it using https? > > There would be more pages but these are just examples. > > http://www.hatena.ne.jp/ > http://f.hatena.ne.jp/hotfoto > > The first one doesn't contain so many images but relatively many and > the second one contains lots of images. I don't think this is the site > specific issue. Both seem to work fine here. What happens if you disable cookies? Could you try with cookies disabled and a dillo version before the TCP_NODELAY change? I remember that we once wanted to limit concurrent connections to a reasonable number and maybe the slow DPI communication is just doing that. So I suspect that without cookies even the version without TCP_NODELAY has issues - but that's just a wild guess. Regards, Johannes

Jan. 14, 2010