- Dillo-dev - lists.mailman3.com

Cookies performance

by Johannes.Hofmann＠gmx.de

On Tue, Jan 12, 2010 at 07:06:12PM +0000, corvid wrote: > Johannes wrote: > > On Tue, Jan 12, 2010 at 05:42:03PM +0000, corvid wrote: > > > Johannes wrote: > > > > I noticed that since the recent DPI update pages which use > > > > cookies load slower (e.g. fltk.org) > > > > Do others also see that? It could also be OS specific. > > > > > > I thought it seemed weirdly slow _sometimes_. > > > I'll have to throw some gprof at it and hope for something > > > simple and embarrassing. > > > > > > (I wonder whether it would be easy to do valgrind with dpid and the > > > plugins.) > > > > I did some truss on the cookies dpi and found that it is reading > > ~/.dillo/dpid_comm_keys quite often. Maybe that is related. > > It looks like it checks it through a_Dpip_check_auth() when it > gets or sets a msg. As for whether it needs to, I don't know. > > I tried a little gprof, but a modest amount of browsing and > accepting cookies wasn't enough for the dpi to use up enough > time for gprof to generate stats. Yah, I don't think it's CPU bound. There must be something else. I will have a look at it.

Jan. 12, 2010

display: inline

by corvid＠lavabit.com

Johannes wrote: > On Tue, Dec 08, 2009 at 06:56:55PM +0100, Johannes Hofmann wrote: > > Hi, > > > > I started to experiment with the real solution, which means that the > > display member of Style determines whether a (Text)block is created > > or a ListItem, or whether just normal text is added. > > The patch is still small, but as the change is pretty > > fundamental, so I decided to put it in a separate repo for now. > > You can find it at: > > > > http://www.flpsed.org/hgweb/dillo_display/ > > > > I've updated this prototype a bit. It works quite well, the only > drawback is the added complexity in html.cc. > In addition to Html_tag_open_*() and Html_tag_clone_*(), there now is > an optional Html_tag_content_*() function for tags that create > widgets or other content. This function is not called in the > display:none case. > The reason for the splitting into Html_tag_open_*() and > Html_tag_content_*() is that styleEngine->setNonCssHints() is > needed before we can get the current style. > Only after that is done, we know the value of the display property > and can decide whether to call Html_tag_content_*() or not. > > I'd like to get this into main line soon if you agree. > Please let me know what you think. Maybe if Tags[] gets complicated enough, it will become possible to talk Jorge into using NULL for the other default fields as well. At the mention of setNonCssHints, I was hoping that whatever the prototype did would magically make it so that I could implement tooltips on arbitrary elements easily, but no such luck.

Jan. 12, 2010

Cookies performance

by corvid＠lavabit.com

Johannes wrote: > On Tue, Jan 12, 2010 at 05:42:03PM +0000, corvid wrote: > > Johannes wrote: > > > I noticed that since the recent DPI update pages which use > > > cookies load slower (e.g. fltk.org) > > > Do others also see that? It could also be OS specific. > > > > I thought it seemed weirdly slow _sometimes_. > > I'll have to throw some gprof at it and hope for something > > simple and embarrassing. > > > > (I wonder whether it would be easy to do valgrind with dpid and the > > plugins.) > > I did some truss on the cookies dpi and found that it is reading > ~/.dillo/dpid_comm_keys quite often. Maybe that is related. It looks like it checks it through a_Dpip_check_auth() when it gets or sets a msg. As for whether it needs to, I don't know. I tried a little gprof, but a modest amount of browsing and accepting cookies wasn't enough for the dpi to use up enough time for gprof to generate stats.

Jan. 12, 2010

Cookies performance

by Johannes.Hofmann＠gmx.de

On Tue, Jan 12, 2010 at 05:42:03PM +0000, corvid wrote: > Johannes wrote: > > I noticed that since the recent DPI update pages which use > > cookies load slower (e.g. fltk.org) > > Do others also see that? It could also be OS specific. > > I thought it seemed weirdly slow _sometimes_. > I'll have to throw some gprof at it and hope for something > simple and embarrassing. > > (I wonder whether it would be easy to do valgrind with dpid and the > plugins.) I did some truss on the cookies dpi and found that it is reading ~/.dillo/dpid_comm_keys quite often. Maybe that is related.

Jan. 12, 2010

display: inline

by Johannes.Hofmann＠gmx.de

On Tue, Dec 08, 2009 at 06:56:55PM +0100, Johannes Hofmann wrote: > Hi, > > I started to experiment with the real solution, which means that the > display member of Style determines whether a (Text)block is created > or a ListItem, or whether just normal text is added. > The patch is still small, but as the change is pretty > fundamental, so I decided to put it in a separate repo for now. > You can find it at: > > http://www.flpsed.org/hgweb/dillo_display/ > I've updated this prototype a bit. It works quite well, the only drawback is the added complexity in html.cc. In addition to Html_tag_open_*() and Html_tag_clone_*(), there now is an optional Html_tag_content_*() function for tags that create widgets or other content. This function is not called in the display:none case. The reason for the splitting into Html_tag_open_*() and Html_tag_content_*() is that styleEngine->setNonCssHints() is needed before we can get the current style. Only after that is done, we know the value of the display property and can decide whether to call Html_tag_content_*() or not. I'd like to get this into main line soon if you agree. Please let me know what you think. Cheers, Johannes

Jan. 12, 2010

Cookies performance

by corvid＠lavabit.com

Johannes wrote: > I noticed that since the recent DPI update pages which use > cookies load slower (e.g. fltk.org) > Do others also see that? It could also be OS specific. I thought it seemed weirdly slow _sometimes_. I'll have to throw some gprof at it and hope for something simple and embarrassing. (I wonder whether it would be easy to do valgrind with dpid and the plugins.)

Jan. 12, 2010

web bug save?

by corvid＠lavabit.com

I wrote: > I wrote: > > I wrote: > > > Johannes wrote: > > > > On Thu, Dec 31, 2009 at 12:27:36AM -0300, Jorge Arellano Cid wrote: > > > > > Sometimes I wonder whether > > > > > connecting through to a TOR network could help us a lot with it. > > > > > > > > You can already use dillo with privoxy and tor. We could perhaps > > > > make it more convenient to configure such a setup. > > > > We also would need to make sure that really all requests go through > > > > the proxy, e.g. I'm not sure, whether wget called from dowload.dpi > > > > uses the proxy from dillorc. > > > > > > I have warnings in the FAQ and dillorc that the wget env variables would > > > need to be set appropriately, but it would be nice to have dillo send that > > > to the dpi in requests so that it can set the environment for sure before > > > execing wget. > > > > I just committed a patch to set the environment variable from the preference > > if it wasn't already set. > > > > issues: > > - maybe we should have an ftp_proxy pref, too. > > - ISTR that dillo doesn't understand user/pass in URLs. > > This means that dillo doesn't do the right thing if > > - the env var has the user/pass > > - dillo is setting the env for wget. > > It turns out that if I want to add the ability for ftp/downloads > to do the right thing with the no_proxy pref as well, > I would have to send a no_proxy arg to the dpi. > > It would be cleaner to send the proxy URL to the dpi > (and set/clear it there in the environment for wget) every time > that it's needed than to have the proxy URL in the environment > and send a no_proxy arg when it's needed. > > So I'm going to back out that change. This turns out to be a somewhat complicated situation. I naively imagined that whatever I found in wget's 1500-line manpage was what there was, but since wget is a gnu program, it's a subset selected who-knows-how. wget understands the http_proxy, https_proxy, and ftp_proxy environment vars. dillo understands the http_proxy environment var. If it is not set, dillo will use the http_proxy pref. dillo uses its http_proxy value for both http and the https plugin. Whatever we might do to get *_proxy set a certain way in the environment in the downloads/ftp dpis, wgetrc can override them. - should the https plugin use the https_proxy environment var if it's set? I'm leaning toward no. - should dillo try to set the https_proxy env var from the http_proxy pref for the benefit of the downloads dpi? Well, the downloads dpi doesn't like https anyway right now because we haven't given it any sort of #ifdef ENABLE_SSL add argument "--no-check-certificate" #endif, but we could and probably should add the argument. As for the question, ... not sure. (Maybe we should just let wget do all of our https, at least instead of the current plugin.) - should dillo have https_proxy prefs and ftp_proxy prefs? On one hand it might minimize confusion, but then I could take the same amount of space in dillorc to write a big comment explaining all of this. (The unpleasant part of putting things in dillorc or explaining things in dillorc is that I know that no one who upgrades is going to look through it meticulously to see what tiny things have changed.) wget also understands the no_proxy environment variable and wgetrc option. Dillo understands the no_proxy preference. Coincidentally(?), they have the same format. - Should dillo set the env var for wget from the pref if not set? I think so, for the benefit of http downloads. - Should dillo use the env var for itself if set? I kind of don't want to add the complexity. I don't know. (And to get user/pass into wget for proxy, it seems that the proper thing is to stuff them into the env var *_proxy urls.) Anyway, I'm finding this all somewhat tricky to think about.

Jan. 12, 2010

Cookies performance

by Johannes.Hofmann＠gmx.de

Hi, I noticed that since the recent DPI update pages which use cookies load slower (e.g. fltk.org) Do others also see that? It could also be OS specific. Cheers, Johannes

Jan. 12, 2010

web bug save?

by Johannes.Hofmann＠gmx.de

On Tue, Jan 12, 2010 at 06:50:01AM +0000, corvid wrote: > I wrote: > > I wrote: > > > Johannes wrote: > > > > On Thu, Dec 31, 2009 at 12:27:36AM -0300, Jorge Arellano Cid wrote: > > > > > Sometimes I wonder whether > > > > > connecting through to a TOR network could help us a lot with it. > > > > > > > > You can already use dillo with privoxy and tor. We could perhaps > > > > make it more convenient to configure such a setup. > > > > We also would need to make sure that really all requests go through > > > > the proxy, e.g. I'm not sure, whether wget called from dowload.dpi > > > > uses the proxy from dillorc. > > > > > > I have warnings in the FAQ and dillorc that the wget env variables would > > > need to be set appropriately, but it would be nice to have dillo send that > > > to the dpi in requests so that it can set the environment for sure before > > > execing wget. > > > > I just committed a patch to set the environment variable from the preference > > if it wasn't already set. > > > > issues: > > - maybe we should have an ftp_proxy pref, too. > > - ISTR that dillo doesn't understand user/pass in URLs. > > This means that dillo doesn't do the right thing if > > - the env var has the user/pass > > - dillo is setting the env for wget. > > It turns out that if I want to add the ability for ftp/downloads > to do the right thing with the no_proxy pref as well, > I would have to send a no_proxy arg to the dpi. > > It would be cleaner to send the proxy URL to the dpi > (and set/clear it there in the environment for wget) every time > that it's needed than to have the proxy URL in the environment > and send a no_proxy arg when it's needed. So the idea is to always use command line arguments instead of environment variables? Sounds good to me. Cheers, Johannes

Jan. 12, 2010

web bug save?

by corvid＠lavabit.com

I wrote: > I wrote: > > Johannes wrote: > > > On Thu, Dec 31, 2009 at 12:27:36AM -0300, Jorge Arellano Cid wrote: > > > > Sometimes I wonder whether > > > > connecting through to a TOR network could help us a lot with it. > > > > > > You can already use dillo with privoxy and tor. We could perhaps > > > make it more convenient to configure such a setup. > > > We also would need to make sure that really all requests go through > > > the proxy, e.g. I'm not sure, whether wget called from dowload.dpi > > > uses the proxy from dillorc. > > > > I have warnings in the FAQ and dillorc that the wget env variables would > > need to be set appropriately, but it would be nice to have dillo send that > > to the dpi in requests so that it can set the environment for sure before > > execing wget. > > I just committed a patch to set the environment variable from the preference > if it wasn't already set. > > issues: > - maybe we should have an ftp_proxy pref, too. > - ISTR that dillo doesn't understand user/pass in URLs. > This means that dillo doesn't do the right thing if > - the env var has the user/pass > - dillo is setting the env for wget. It turns out that if I want to add the ability for ftp/downloads to do the right thing with the no_proxy pref as well, I would have to send a no_proxy arg to the dpi. It would be cleaner to send the proxy URL to the dpi (and set/clear it there in the environment for wget) every time that it's needed than to have the proxy URL in the environment and send a no_proxy arg when it's needed. So I'm going to back out that change.

Jan. 12, 2010