March 2008 - Dillo-dev - lists.mailman3.com

multiple-scan images consume a lot of CPU

by Johannes.Hofmann＠gmx.de

On Sat, Mar 15, 2008 at 11:46:59PM +0000, place wrote: > Johannes wrote: > > On Sat, Mar 15, 2008 at 10:16:09PM +0000, place wrote: > > > Johannes wrote: > > > > So as the "TODO" comment in jpeg.c already suggests, we need some > > > > way to switch off multiple-scan when the image is readily available > > > > either from cache, from disk, or from a fast network connection. > > > > > > > > I think some sort of rate limiting would be best that restricts > > > > rescans to - say - once a second. > > > That does not quite work for smaller images. The threshhold would > > need to depend somehow on the total size of the image. > > Do you encounter them much? I was hoping it was too rare to care about. Forget about this comment for now. Your patch below works great for me. I was seeing the problem for example on http://www.tourentipp.de When I went back to that page, it was pretty slow. Thanks, Johannes > > Looking at the bottom of Cache_process_queue(), maybe clients could be > taught to understand CA_Close in place of the last CA_Send for an entry. > Or some interface to get our hands on the cache entry's CA_GotData flag. > I don't know; Jorge will know better. > > > > And I get "Improper call to JPEG library in state 205" but that > > should be easily avoidable. > > Sorry, I was in a bit of a hurry at the time, and I don't have any > multiple scan jpegs now to try it on (could you point me to some?) > > It should have been something more like > > diff -pur dillo2/src/jpeg.c dillo2-cur/src/jpeg.c > --- dillo2/src/jpeg.c 2008-01-16 02:57:32.000000000 +0000 > +++ dillo2-cur/src/jpeg.c 2008-03-15 23:24:21.000000000 +0000 > @@ -293,7 +293,7 @@ static void Jpeg_write(DilloJpeg *jpeg, > * when an image arrives slowly, but should not be used to redisplay > * cached images. > */ > - if (jpeg_has_multiple_scans(&jpeg->cinfo)) > + if ((BufSize < 2000) && jpeg_has_multiple_scans(&jpeg->cinfo)) > jpeg->cinfo.buffered_image = TRUE; > > a_Dicache_set_parms(jpeg->url, jpeg->version, jpeg->Image, > @@ -309,7 +309,7 @@ static void Jpeg_write(DilloJpeg *jpeg, > /* decompression step 5 (see libjpeg.doc) */ > if (jpeg_start_decompress(&(jpeg->cinfo))) { > jpeg->y = 0; > - jpeg->state = jpeg_has_multiple_scans(&jpeg->cinfo) ? > + jpeg->state = jpeg->cinfo.buffered_image ? > DILLO_JPEG_READ_BEGIN_SCAN : DILLO_JPEG_READ_IN_SCAN; > } > } > @@ -355,7 +355,7 @@ static void Jpeg_write(DilloJpeg *jpeg, > > if (jpeg->y == jpeg->cinfo.image_height) { > /* end of scan */ > - if (!jpeg_has_multiple_scans(&jpeg->cinfo)) { > + if (!jpeg->cinfo.buffered_image) { > jpeg->state = DILLO_JPEG_DONE; > break; > } else { > > > _______________________________________________ > Dillo-dev mailing list > Dillo-dev(a)dillo.org > http://lists.auriga.wearlab.de/cgi-bin/mailman/listinfo/dillo-dev

March 16, 2008

multiple-scan images consume a lot of CPU

by place＠gobigwest.com

Johannes wrote: > On Sat, Mar 15, 2008 at 10:16:09PM +0000, place wrote: > > Johannes wrote: > > > So as the "TODO" comment in jpeg.c already suggests, we need some > > > way to switch off multiple-scan when the image is readily available > > > either from cache, from disk, or from a fast network connection. > > > > > > I think some sort of rate limiting would be best that restricts > > > rescans to - say - once a second. > That does not quite work for smaller images. The threshhold would > need to depend somehow on the total size of the image. Do you encounter them much? I was hoping it was too rare to care about. Looking at the bottom of Cache_process_queue(), maybe clients could be taught to understand CA_Close in place of the last CA_Send for an entry. Or some interface to get our hands on the cache entry's CA_GotData flag. I don't know; Jorge will know better. > And I get "Improper call to JPEG library in state 205" but that > should be easily avoidable. Sorry, I was in a bit of a hurry at the time, and I don't have any multiple scan jpegs now to try it on (could you point me to some?) It should have been something more like diff -pur dillo2/src/jpeg.c dillo2-cur/src/jpeg.c --- dillo2/src/jpeg.c 2008-01-16 02:57:32.000000000 +0000 +++ dillo2-cur/src/jpeg.c 2008-03-15 23:24:21.000000000 +0000 @@ -293,7 +293,7 @@ static void Jpeg_write(DilloJpeg *jpeg, * when an image arrives slowly, but should not be used to redisplay * cached images. */ - if (jpeg_has_multiple_scans(&jpeg->cinfo)) + if ((BufSize < 2000) && jpeg_has_multiple_scans(&jpeg->cinfo)) jpeg->cinfo.buffered_image = TRUE; a_Dicache_set_parms(jpeg->url, jpeg->version, jpeg->Image, @@ -309,7 +309,7 @@ static void Jpeg_write(DilloJpeg *jpeg, /* decompression step 5 (see libjpeg.doc) */ if (jpeg_start_decompress(&(jpeg->cinfo))) { jpeg->y = 0; - jpeg->state = jpeg_has_multiple_scans(&jpeg->cinfo) ? + jpeg->state = jpeg->cinfo.buffered_image ? DILLO_JPEG_READ_BEGIN_SCAN : DILLO_JPEG_READ_IN_SCAN; } } @@ -355,7 +355,7 @@ static void Jpeg_write(DilloJpeg *jpeg, if (jpeg->y == jpeg->cinfo.image_height) { /* end of scan */ - if (!jpeg_has_multiple_scans(&jpeg->cinfo)) { + if (!jpeg->cinfo.buffered_image) { jpeg->state = DILLO_JPEG_DONE; break; } else {

March 16, 2008

multiple-scan images consume a lot of CPU

by Johannes.Hofmann＠gmx.de

On Sat, Mar 15, 2008 at 10:16:09PM +0000, place wrote: > Johannes wrote: > > I really like the multiple-scan image feature added to dillo > > recently, but it uses quite some CPU and makes it drag slow when > > going back and forth on pages with a lot of images. > > > > So as the "TODO" comment in jpeg.c already suggests, we need some > > way to switch off multiple-scan when the image is readily available > > either from cache, from disk, or from a fast network connection. > > > > I think some sort of rate limiting would be best that restricts > > rescans to - say - once a second. > > > > What do you think? > > Just as a quick experiment, would something like the following do much good? > > diff -pur dillo2/src/jpeg.c dillo2-cur/src/jpeg.c > --- dillo2/src/jpeg.c 2008-01-16 02:57:32.000000000 +0000 > +++ dillo2-cur/src/jpeg.c 2008-03-15 22:10:59.000000000 +0000 > @@ -293,7 +293,7 @@ static void Jpeg_write(DilloJpeg *jpeg, > * when an image arrives slowly, but should not be used to redisplay > * cached images. > */ > - if (jpeg_has_multiple_scans(&jpeg->cinfo)) > + if ((BufSize < 2000) && jpeg_has_multiple_scans(&jpeg->cinfo)) > jpeg->cinfo.buffered_image = TRUE; > > a_Dicache_set_parms(jpeg->url, jpeg->version, jpeg->Image, > That does not quite work for smaller images. The threshhold would need to depend somehow on the total size of the image. And I get "Improper call to JPEG library in state 205" but that should be easily avoidable. Cheers, Johannes

March 16, 2008

multiple-scan images consume a lot of CPU

by place＠gobigwest.com

Johannes wrote: > I really like the multiple-scan image feature added to dillo > recently, but it uses quite some CPU and makes it drag slow when > going back and forth on pages with a lot of images. > > So as the "TODO" comment in jpeg.c already suggests, we need some > way to switch off multiple-scan when the image is readily available > either from cache, from disk, or from a fast network connection. > > I think some sort of rate limiting would be best that restricts > rescans to - say - once a second. > > What do you think? Just as a quick experiment, would something like the following do much good? diff -pur dillo2/src/jpeg.c dillo2-cur/src/jpeg.c --- dillo2/src/jpeg.c 2008-01-16 02:57:32.000000000 +0000 +++ dillo2-cur/src/jpeg.c 2008-03-15 22:10:59.000000000 +0000 @@ -293,7 +293,7 @@ static void Jpeg_write(DilloJpeg *jpeg, * when an image arrives slowly, but should not be used to redisplay * cached images. */ - if (jpeg_has_multiple_scans(&jpeg->cinfo)) + if ((BufSize < 2000) && jpeg_has_multiple_scans(&jpeg->cinfo)) jpeg->cinfo.buffered_image = TRUE; a_Dicache_set_parms(jpeg->url, jpeg->version, jpeg->Image,

March 15, 2008

multiple-scan images consume a lot of CPU

by Johannes.Hofmann＠gmx.de

Hi, I really like the multiple-scan image feature added to dillo recently, but it uses quite some CPU and makes it drag slow when going back and forth on pages with a lot of images. So as the "TODO" comment in jpeg.c already suggests, we need some way to switch off multiple-scan when the image is readily available either from cache, from disk, or from a fast network connection. I think some sort of rate limiting would be best that restricts rescans to - say - once a second. What do you think? Cheers, Johannes

March 15, 2008

patch: multipart/form-data CRLF

by place＠gobigwest.com

Happened to notice in the RFC that multipart types must use CRLF between body parts. PS It was a bit annoying to figure out that http.c needed to be changed, which suggests that it would have been better for me to store the boundary string in DilloUrl somewhere after all.

March 15, 2008

https plugin status

by onepoint＠starurchin.org

On Sat, Mar 15, 2008 at 03:31:52PM -0400, Jorge Arellano Cid wrote: > > [Jeremey wrote] > > Or maybe Capi_url_uses_dpi() should be checking for "https" and > > *not* adding the "proto." prefix? > > "proto.https" is the right patch. OK, what about my patch to redirect MSG debugging messages from stdout to stderr? Is that correct? Regards, Jeremy Henty

March 15, 2008

[patch] don't restart scanning from start in Html_parse_entities ()

by Johannes.Hofmann＠gmx.de

Hi, below is a small optinization to Html_parse_entities (). Cheers, Johannes diff -r bd486ec2a522 src/html.cc --- a/src/html.cc Sat Mar 15 19:41:01 2008 +0100 +++ b/src/html.cc Sat Mar 15 20:23:03 2008 +0100 @@ -1397,13 +1397,14 @@ static char * { const char *esc_set = "&\xE2\xC2"; char *new_str, buf[4]; - int i, j, k, n, isocode, entsize; + int i, j, k, n, s, isocode, entsize; new_str = dStrndup(token, toksize); - if (new_str[strcspn(new_str, esc_set)] == 0) + s = strcspn(new_str, esc_set); + if (new_str[s] == 0) return new_str; - for (i = j = 0; i < toksize; i++) { + for (i = j = s; i < toksize; i++) { if (token[i] == '&' && (isocode = Html_parse_entity(html, token+i, toksize-i, &entsize)) >= 0) {

March 15, 2008

[patch] further reduce redraws from resizing tables

by Johannes.Hofmann＠gmx.de

Hi Jorge, On Fri, Mar 14, 2008 at 06:53:08PM -0400, Jorge Arellano Cid wrote: > Hi Johannes, > > On Tue, Mar 11, 2008 at 07:27:01PM +0100, Johannes Hofmann wrote: > > Hi, > > > > below is another patch to reduce the redraws caused by resizing > > tables. A new method Layout::queueDrawExcept(), that requests the > > drawing of a rectangle except another rectangle is added. > > It is then used in widget.cc to redraw the area that was previously > > occupied by a widget, but is no longer occupied after a resize or move. > > This makes it possible to redraw less when a table resizes. > > All in all it should reduce the flicker a bit when loading table > > based pages. > > On the topic of reducing the flicker, I'm far of an expert and > please correct me if I'm wrong. AFAIU, the current design draws > the background color before the widget content. > > This is: if you have an html page as big table, in dillo, it > will become a table inside a textblock. When the textblock is > asked to redraw, it will _erase_ the table with its background > color and ask its child (the table) to redraw. Causing flicker. Yes, that's true. In addition, before anything is drawn, the rectangle to be drawn is already cleared with the background color in FltkViewBase::draw() (the fillrect () calls). > > If this is true, a big gain could come from a different > approach: just draw the surroundings of the widgets and ask the > child widgets to redraw. > > This way widgets aren't erased beforehand, and can also decide > not to redraw themselves when nothing has changed. > > I have worked a bit on this idea, and the attached patch mocks > the idea hackishly. It draws the "padding" area of the CSS box > (see the picture in namespacedw_1_1core_1_1style.html in dw docs) > > Please give it a review and share your comments, > corrections and thoughts. I've been thinking about something similar as well, but I don't think it will work out generally. The main problem is text drawing and maybe transparent widgets (don't know whether they exist already). Anyway, for drawing text, you need to erase the area first. Just comment out the fillrect () calls in FltkViewBase::draw() to see what I mean :-) Therefore, we will always have flicker, when drawing text. The good news is that all the flicker can be avoided easily with some double buffering technique. And from dillo1 we even know that this is not necessarily slow :-) So for me the flicker itself is not that much of a problem. It may be an indicator for unnecessary redraws, which should be avoided to save CPU power. But I think current cvs is already pretty good in this regard. So I would propose to start to think about efficient double buffering similar to what I had posted in http://lists.auriga.wearlab.de/pipermail/dillo-dev/2007-November/003379.html but platform independent using fltk2 support. Cheers, Johannes

March 15, 2008

[PATCH]: Fixing https step 1: Have a basic dpi dialog window working again

by Christian.Kellermann＠nefkom.net

Hi Jorge, * Jorge Arellano Cid <jcid(a)dillo.org> [080315 18:58]: > Sorry for the late answer, but I've had little spare time, > and several topics to answer. > > Probably you have already read: > > http://lists.auriga.wearlab.de/pipermail/dillo-dev/2007-September/003211.ht… > > an got some info from there. Yes, thanks for the hint, it was insightful. I have spend a few thoughts on the SSL issue and I'd like to hear your opinion on it. The most work that still needs to be done is to handle certificates and the user interaction required to make it usable. The user needs to add trust values to certificates and to add or remove them offline. I think this can fit well into a server dpi. Adding and deleting bookmarks could be done similar to handling bookmarks. Which might be a bit rough but is independent on fltk widgets. I do think that we already have enough dialog types in dillo. Hard coding the certificate handling would not make the situation any better. Also I can imagine this plugin to handle client certificates which I do believe will become a more common requirement for certain applications in the future. Adding secure storage capabilities will be easy. Also people will have the chance to write some "bridging" plugin that extracts the needed data from existing storage sources elsewhere, i.e. certificate servers,... Handling this all internally means that we put functionality into dillo that are needed for, but basically have nothing to do with browsing. As I have grasped the spirit of this project this may not be the way to go. A nice thing about handling https internally is the opportunity to reuse an established connection for more than one request, which might speed up transfers significantly. But on the other hand couldn't this be done by dpid for all filter plugins? Dpid can keep them alive as long as the following requests belong to the same URL, then closing the connection after a timeout? I am sorry if this has been already decided on before. These thoughts were rumaging aroung my head all day now... Kind regards, Christian -- You may use my gpg key for replies: pub 1024D/47F79788 2005/02/02 Christian Kellermann (C-Keen)

March 15, 2008