- Dillo-dev - lists.mailman3.com

form.cc: more refactoring

by onepoint＠starurchin.org

On Fri, Jun 20, 2008 at 02:37:22PM -0400, Jorge Arellano Cid wrote: > On Thu, Jun 19, 2008 at 08:48:48PM +0100, Jeremy Henty wrote: > > On Wed, Jun 18, 2008 at 08:35:57AM -0400, Jorge Arellano Cid wrote: > > > > > Committed, except 00 and 05. > > > > > > 00: The stash buffer may hold a script, in that case > > > parse_stash is not as clear as parse_entities. > > > > Would a better name make the change acceptable? For instance, > > Html_parse_stash_entries() ? > > Html_parse_stash_entities() ? > > Why is it better than the traditional a_Html_parse_entities()? Sorry, I wasn't very clear. I read you as saying that the new function a_Html_parse_stash() was not very well named, and I meant to suggest a_Html_parse_stash_entities() as an improvement. If you were saying that the refactoring itself is confusing then I misread you. Regards, Jeremy Henty

June 23, 2008

Memory usage

by Johannes.Hofmann＠gmx.de

Hi Jorge, On Sun, Jun 22, 2008 at 10:11:30AM -0400, Jorge Arellano Cid wrote: > Hi there, > > I've been reviewing memory usage these days, and found some > weird numbers that I'd like to understand. > > Taking the huge mysql page (13MB), I made the following > memory measures (with ps). Load a bare dillo, load the simple > directory where the huge page is located, then the huge page, > then back, then forward, back, forward and back. > > I did this for dillo2.19Jun, dillo2.21Jun, dillo-fltk (with > zone allocators) and dillo1. All in GNU/Linux with image_off. > > Here are the numbers: > > ----------- > Memory test > ----------- > > %MEM VSZ RSS TTY STAT START TIME COMMAND > 0.1 9672 4144 pts/10 S+ 09:37 0:00 ./dillo-fltk.19Jun bare > 0.2 10376 4756 pts/10 S+ 09:37 0:00 ./dillo-fltk.19Jun dir list > 6.9 176176 144776 pts/10 S+ 09:37 0:02 ./dillo-fltk.19Jun huge page > 5.4 123556 113216 pts/10 S+ 09:37 0:02 ./dillo-fltk.19Jun back > 7.0 176176 146240 pts/10 S+ 09:37 0:05 ./dillo-fltk.19Jun forward > 5.6 127020 117720 pts/10 S+ 09:37 0:05 ./dillo-fltk.19Jun back > 7.0 176172 146240 pts/10 S+ 09:37 0:07 ./dillo-fltk.19Jun forward > 5.6 127016 117716 pts/10 S+ 09:37 0:08 ./dillo-fltk.19Jun back > > > %MEM VSZ RSS TTY STAT START TIME COMMAND > 0.1 9672 4140 pts/10 S+ 09:26 0:00 ./dillo-fltk.21Jun bare > 0.2 10376 4760 pts/10 S+ 09:26 0:00 ./dillo-fltk.21Jun dir list > 5.9 147904 123988 pts/10 S+ 09:31 0:02 ./dillo-fltk.21Jun huge page > 4.9 111668 101944 pts/10 S+ 09:31 0:02 ./dillo-fltk.21Jun back > 6.0 147780 124792 pts/10 S+ 09:31 0:05 ./dillo-fltk.21Jun forward > 5.0 115008 105780 pts/10 S+ 09:31 0:05 ./dillo-fltk.21Jun back > 6.0 147780 124856 pts/10 S+ 09:31 0:07 ./dillo-fltk.21Jun forward > 5.1 115008 105844 pts/10 S+ 09:31 0:07 ./dillo-fltk.21Jun back > > > %MEM VSZ RSS TTY STAT START TIME COMMAND > 0.1 9676 4084 pts/10 S+ 11:16 0:00 ./dillo-fltk bare > 0.2 10464 4768 pts/10 S+ 11:17 0:00 ./dillo-fltk dir list > 5.4 136960 112920 pts/10 S+ 11:17 0:04 ./dillo-fltk huge page > 4.3 100724 90864 pts/10 S+ 11:17 0:04 ./dillo-fltk back > 5.4 136952 113756 pts/10 S+ 11:17 0:09 ./dillo-fltk forward > 4.5 104180 94744 pts/10 S+ 11:17 0:09 ./dillo-fltk back > 5.4 136888 113776 pts/10 S+ 11:17 0:13 ./dillo-fltk forward > 4.5 104116 94764 pts/10 S+ 11:17 0:14 ./dillo-fltk back > > > %MEM VSZ RSS TTY STAT START TIME COMMAND > 0.1 8408 3524 pts/10 S+ 10:59 0:00 ./dillo1 bare > 0.1 24788 3564 pts/10 S+ 11:01 0:00 ./dillo1 dir list > 8.7 226996 182220 pts/10 S+ 11:01 0:07 ./dillo1 huge page > 6.8 165880 141792 pts/10 S+ 11:01 0:09 ./dillo1 back > 8.8 226952 184176 pts/10 S+ 11:01 0:17 ./dillo1 forward > 6.9 166264 144024 pts/10 S+ 11:01 0:18 ./dillo1 back > 8.8 226968 184216 pts/10 S+ 11:01 0:26 ./dillo1 forward > 6.9 166264 144060 pts/10 S+ 11:01 0:27 ./dillo1 back > > > Besides showing clearly a sane memory usage reduction pattern, > there's a strange fact I don't understand. Once back from the > huge page, I'd expect memory usage to drop to what it was before > plus the 13MB of cached data, plus some KB of overhead, BUT the > RSS keeps near 90MB. > > This happens with dillo1 too. > > My thoughts: > > * If this memory is not freed, it'd be great to find the huge > leak. > * If it's freed but not returned to the OS., dillo would > consume as much memory as the biggest page it has loaded. > Clearly not a desirable situation. > * It may be that the O.S. knows it can claim back this memory > when necessary but prefers to keep it assigned to the > same process until it finds better use for it. I think this is related to the malloc implementation in Linux/glibc. Here on DragonFly I get: USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND hofmann 3331 0.0 0.3 6732 4436 p3 IL+ 8:37PM 0:00.14 ./dillo-fltk dir list hofmann 3331 26.9 7.6 140116 118580 p3 SL+ 8:37PM 0:04.13 ./dillo-fltk huge page hofmann 3331 10.3 1.2 25488 19064 p3 SL+ 8:37PM 0:05.38 ./dillo-fltk back hofmann 3331 79.2 7.6 140112 118584 p3 SL+ 8:37PM 0:09.90 ./dillo-fltk forward hofmann 3331 11.1 1.3 26252 19840 p3 SL+ 8:37PM 0:11.29 ./dillo-fltk back hofmann 3331 91.1 7.6 140548 119232 p3 SL+ 8:37PM 0:15.67 ./dillo-fltk forward hofmann 3331 27.8 1.3 26816 20392 p3 SL+ 8:37PM 0:16.98 ./dillo-fltk back Which is not perfect, but reasonable. I think DragonFly has changed the malloc implementation some time ago. If I remember correctly there was a more Linux-like behaviour before. Cheers, Johannes > > > > ------------------------------------------- > Huge page with different memory allocators: > ------------------------------------------- > > %MEM VSZ RSS TTY STAT START TIME COMMAND > 5.4 134912 113740 pts/17 S+ 18:56 0:02 ./dillo-fltk step: 2x > 5.4 134916 113752 pts/17 S+ 18:57 0:02 ./dillo-fltk step: 2x > 5.4 134828 113660 pts/17 S+ 18:58 0:02 ./dillo-fltk step: 2x > > 5.0 114940 105432 pts/17 S+ 18:53 0:02 ./dillo-fltk step: 1 > 5.0 111976 105488 pts/17 S+ 18:54 0:03 ./dillo-fltk step: 1 > 5.0 114872 105416 pts/17 S+ 18:54 0:02 ./dillo-fltk step: 1 > > 5.1 117796 106436 pts/17 S+ 18:59 0:02 ./dillo-fltk step: cst > 5.1 117568 106404 pts/17 S+ 19:00 0:02 ./dillo-fltk step: cst > 5.1 117820 106460 pts/17 S+ 19:00 0:02 ./dillo-fltk step: cst > > > This data comes from a memory usage reduction by tunning the > simpleVector memory allocator. The first is the current one, > which allocates in chunks of 2*numAlloc, the second allocates one > by one and the third is a custom one I made. > > The 1-allocator is the best memory usage reducer but it's > slower (roughly 20%). The cst-allocator reduces a bit less but is > almost as fast as the 2x-allocator (2% slower or so). > > Memory reduction: > > VSZ RSS > ----------------------------- > 1-allocator 15% 7.3% > cst-allocator 13% 6.5% > ----------------------------- > > Well, compared with the 1000% "leak" above... :-) > > -- > Cheers > Jorge.- > > _______________________________________________ > Dillo-dev mailing list > Dillo-dev(a)dillo.org > http://lists.auriga.wearlab.de/cgi-bin/mailman/listinfo/dillo-dev

June 22, 2008

Memory usage

by jcid＠dillo.org

Hi there, I've been reviewing memory usage these days, and found some weird numbers that I'd like to understand. Taking the huge mysql page (13MB), I made the following memory measures (with ps). Load a bare dillo, load the simple directory where the huge page is located, then the huge page, then back, then forward, back, forward and back. I did this for dillo2.19Jun, dillo2.21Jun, dillo-fltk (with zone allocators) and dillo1. All in GNU/Linux with image_off. Here are the numbers: ----------- Memory test ----------- %MEM VSZ RSS TTY STAT START TIME COMMAND 0.1 9672 4144 pts/10 S+ 09:37 0:00 ./dillo-fltk.19Jun bare 0.2 10376 4756 pts/10 S+ 09:37 0:00 ./dillo-fltk.19Jun dir list 6.9 176176 144776 pts/10 S+ 09:37 0:02 ./dillo-fltk.19Jun huge page 5.4 123556 113216 pts/10 S+ 09:37 0:02 ./dillo-fltk.19Jun back 7.0 176176 146240 pts/10 S+ 09:37 0:05 ./dillo-fltk.19Jun forward 5.6 127020 117720 pts/10 S+ 09:37 0:05 ./dillo-fltk.19Jun back 7.0 176172 146240 pts/10 S+ 09:37 0:07 ./dillo-fltk.19Jun forward 5.6 127016 117716 pts/10 S+ 09:37 0:08 ./dillo-fltk.19Jun back %MEM VSZ RSS TTY STAT START TIME COMMAND 0.1 9672 4140 pts/10 S+ 09:26 0:00 ./dillo-fltk.21Jun bare 0.2 10376 4760 pts/10 S+ 09:26 0:00 ./dillo-fltk.21Jun dir list 5.9 147904 123988 pts/10 S+ 09:31 0:02 ./dillo-fltk.21Jun huge page 4.9 111668 101944 pts/10 S+ 09:31 0:02 ./dillo-fltk.21Jun back 6.0 147780 124792 pts/10 S+ 09:31 0:05 ./dillo-fltk.21Jun forward 5.0 115008 105780 pts/10 S+ 09:31 0:05 ./dillo-fltk.21Jun back 6.0 147780 124856 pts/10 S+ 09:31 0:07 ./dillo-fltk.21Jun forward 5.1 115008 105844 pts/10 S+ 09:31 0:07 ./dillo-fltk.21Jun back %MEM VSZ RSS TTY STAT START TIME COMMAND 0.1 9676 4084 pts/10 S+ 11:16 0:00 ./dillo-fltk bare 0.2 10464 4768 pts/10 S+ 11:17 0:00 ./dillo-fltk dir list 5.4 136960 112920 pts/10 S+ 11:17 0:04 ./dillo-fltk huge page 4.3 100724 90864 pts/10 S+ 11:17 0:04 ./dillo-fltk back 5.4 136952 113756 pts/10 S+ 11:17 0:09 ./dillo-fltk forward 4.5 104180 94744 pts/10 S+ 11:17 0:09 ./dillo-fltk back 5.4 136888 113776 pts/10 S+ 11:17 0:13 ./dillo-fltk forward 4.5 104116 94764 pts/10 S+ 11:17 0:14 ./dillo-fltk back %MEM VSZ RSS TTY STAT START TIME COMMAND 0.1 8408 3524 pts/10 S+ 10:59 0:00 ./dillo1 bare 0.1 24788 3564 pts/10 S+ 11:01 0:00 ./dillo1 dir list 8.7 226996 182220 pts/10 S+ 11:01 0:07 ./dillo1 huge page 6.8 165880 141792 pts/10 S+ 11:01 0:09 ./dillo1 back 8.8 226952 184176 pts/10 S+ 11:01 0:17 ./dillo1 forward 6.9 166264 144024 pts/10 S+ 11:01 0:18 ./dillo1 back 8.8 226968 184216 pts/10 S+ 11:01 0:26 ./dillo1 forward 6.9 166264 144060 pts/10 S+ 11:01 0:27 ./dillo1 back Besides showing clearly a sane memory usage reduction pattern, there's a strange fact I don't understand. Once back from the huge page, I'd expect memory usage to drop to what it was before plus the 13MB of cached data, plus some KB of overhead, BUT the RSS keeps near 90MB. This happens with dillo1 too. My thoughts: * If this memory is not freed, it'd be great to find the huge leak. * If it's freed but not returned to the OS., dillo would consume as much memory as the biggest page it has loaded. Clearly not a desirable situation. * It may be that the O.S. knows it can claim back this memory when necessary but prefers to keep it assigned to the same process until it finds better use for it. ------------------------------------------- Huge page with different memory allocators: ------------------------------------------- %MEM VSZ RSS TTY STAT START TIME COMMAND 5.4 134912 113740 pts/17 S+ 18:56 0:02 ./dillo-fltk step: 2x 5.4 134916 113752 pts/17 S+ 18:57 0:02 ./dillo-fltk step: 2x 5.4 134828 113660 pts/17 S+ 18:58 0:02 ./dillo-fltk step: 2x 5.0 114940 105432 pts/17 S+ 18:53 0:02 ./dillo-fltk step: 1 5.0 111976 105488 pts/17 S+ 18:54 0:03 ./dillo-fltk step: 1 5.0 114872 105416 pts/17 S+ 18:54 0:02 ./dillo-fltk step: 1 5.1 117796 106436 pts/17 S+ 18:59 0:02 ./dillo-fltk step: cst 5.1 117568 106404 pts/17 S+ 19:00 0:02 ./dillo-fltk step: cst 5.1 117820 106460 pts/17 S+ 19:00 0:02 ./dillo-fltk step: cst This data comes from a memory usage reduction by tunning the simpleVector memory allocator. The first is the current one, which allocates in chunks of 2*numAlloc, the second allocates one by one and the third is a custom one I made. The 1-allocator is the best memory usage reducer but it's slower (roughly 20%). The cst-allocator reduces a bit less but is almost as fast as the 2x-allocator (2% slower or so). Memory reduction: VSZ RSS ----------------------------- 1-allocator 15% 7.3% cst-allocator 13% 6.5% ----------------------------- Well, compared with the 1000% "leak" above... :-) -- Cheers Jorge.-

June 22, 2008

Textblock::addText()

by jcid＠dillo.org

On Sat, Jun 21, 2008 at 11:12:34PM +0200, Johannes Hofmann wrote: > On Sat, Jun 21, 2008 at 04:58:50PM -0400, Jorge Arellano Cid wrote: > > On Sat, Jun 21, 2008 at 10:36:52PM +0200, Johannes Hofmann wrote: > > > On Sat, Jun 21, 2008 at 02:38:48PM -0400, Jorge Arellano Cid wrote: > > > > On Sat, Jun 21, 2008 at 06:23:18PM +0200, Johannes Hofmann wrote: > > > > > On Sat, Jun 21, 2008 at 11:31:56AM -0400, Jorge Arellano Cid wrote: > > > > > > On Fri, Jun 20, 2008 at 08:45:18PM +0200, Johannes Hofmann wrote: > > > > > > > Hi, > > > > > > > > > > > > > > On Fri, Jun 20, 2008 at 01:03:26PM -0400, Jorge Arellano Cid wrote: > > > > > > > > Hi there, > > > > > > > > > > > > > > > > - Changed Textblock::addText() to internally allocate its text string, > > > > > > > > making the memory handling opaque to the caller. > > > > > > > > > > > > > > > > Does anybody remmeber the topic of addText() deleting a malloc'ed > > > > > > > > block of memory? > > > > > > > > > > > > > > > > I decided to make the change for API cleanliness/correctness and set > > > > > > > > the Textblock do the whole memory handling (i.e. now the Textblock is > > > > > > > > the one that allocates/deallocates the text string). > > > > > > > > > > > > > > > > The HTML parser was adjusted a bit to keep its old performance. > > > > > > > > > > > > > > Very nice! > > > > > > > I thought this is the right moment to drag out my old text > > > > > > > allocation optimization. > > > > > > > And now it really makes a difference. > > > > > > > I see about 10M less memory usage on the mysql page and most > > > > > > > important, leaving that huge page is quite a bit faster - as there > > > > > > > is less free() calls. > > > > > > > The result may be completely different on Linux however. So please > > > > > > > give it a try on Linux. > > > > > > > > > > > > Here I have: > > > > > > > > > > > > Platform: Debian GNU/Linux, dillo-fltk with image_off > > > > > > Test : huge mysql page > > > > > > > > > > > > %MEM VSZ RSS TTY STAT START TIME COMMAND > > > > > > 0.1 9672 4144 pts/10 S+ 08:51 0:00 ./dillo-fltk.21Jun > > > > > > 5.9 147588 123896 pts/10 S+ 08:51 0:02 ./dillo-fltk.21Jun > > > > > > > > > > > > 0.1 9676 4136 pts/17 S+ 08:54 0:00 ./dillo-fltk > > > > > > 5.4 136684 112872 pts/17 S+ 08:54 0:02 ./dillo-fltk > > > > > > > > > > > > This is roughly 10% of memory usage reduction. > > > > > > > > > > > > Committed. > > > > > > > > > > In this case I have a little cleanup. > > > > > This patch also avoids one strlen() per added word. > > > > > > > > How much is the speed gain? > > > > > > I can't see much of a difference. I'm trying with: > > > > > > --- a/src/html.ccSat Jun 21 15:00:35 2008 +0200 > > > +++ b/src/html.ccSat Jun 21 22:27:30 2008 +0200 > > > @@ -672,6 +672,7 @@ void DilloHtml::finishParsing(int Client > > > > > > freeParseData(); > > > parse_finished = true; > > > +exit(0); > > > } > > > > > > /* > > > > > > and then > > > > > > time dillo-fltk manual.html, but the numbers are varying greatly - > > > at least more than the speed gain, if there is any. > > > Do you know a better method to measure the performance? perhaps > > > parsing the document 10 times in a loop? > > > > I get consistent timing with uicmd.cc:647 set to 1. > > But that only works with images enabled, right? Then all timings are > dominated by image decoding. Without images I get timings around 3s > with images it's about 16s. > But anyway, I agree that there is not much gain in avoiding that > strlen call. So let's just forget about that for now. If we can find > a test case where it really helps, we can come back to it. Sorry, I just committed a halt that works both with/without images. (same line). I'm investigating on memory usage, and have further reduced it, but what I'm about to post for discussion is really weird. Stay tuned... -- Cheers Jorge.-

June 22, 2008

Textblock::addText()

by Johannes.Hofmann＠gmx.de

On Sat, Jun 21, 2008 at 04:58:50PM -0400, Jorge Arellano Cid wrote: > On Sat, Jun 21, 2008 at 10:36:52PM +0200, Johannes Hofmann wrote: > > On Sat, Jun 21, 2008 at 02:38:48PM -0400, Jorge Arellano Cid wrote: > > > On Sat, Jun 21, 2008 at 06:23:18PM +0200, Johannes Hofmann wrote: > > > > On Sat, Jun 21, 2008 at 11:31:56AM -0400, Jorge Arellano Cid wrote: > > > > > On Fri, Jun 20, 2008 at 08:45:18PM +0200, Johannes Hofmann wrote: > > > > > > Hi, > > > > > > > > > > > > On Fri, Jun 20, 2008 at 01:03:26PM -0400, Jorge Arellano Cid wrote: > > > > > > > Hi there, > > > > > > > > > > > > > > - Changed Textblock::addText() to internally allocate its text string, > > > > > > > making the memory handling opaque to the caller. > > > > > > > > > > > > > > Does anybody remmeber the topic of addText() deleting a malloc'ed > > > > > > > block of memory? > > > > > > > > > > > > > > I decided to make the change for API cleanliness/correctness and set > > > > > > > the Textblock do the whole memory handling (i.e. now the Textblock is > > > > > > > the one that allocates/deallocates the text string). > > > > > > > > > > > > > > The HTML parser was adjusted a bit to keep its old performance. > > > > > > > > > > > > Very nice! > > > > > > I thought this is the right moment to drag out my old text > > > > > > allocation optimization. > > > > > > And now it really makes a difference. > > > > > > I see about 10M less memory usage on the mysql page and most > > > > > > important, leaving that huge page is quite a bit faster - as there > > > > > > is less free() calls. > > > > > > The result may be completely different on Linux however. So please > > > > > > give it a try on Linux. > > > > > > > > > > Here I have: > > > > > > > > > > Platform: Debian GNU/Linux, dillo-fltk with image_off > > > > > Test : huge mysql page > > > > > > > > > > %MEM VSZ RSS TTY STAT START TIME COMMAND > > > > > 0.1 9672 4144 pts/10 S+ 08:51 0:00 ./dillo-fltk.21Jun > > > > > 5.9 147588 123896 pts/10 S+ 08:51 0:02 ./dillo-fltk.21Jun > > > > > > > > > > 0.1 9676 4136 pts/17 S+ 08:54 0:00 ./dillo-fltk > > > > > 5.4 136684 112872 pts/17 S+ 08:54 0:02 ./dillo-fltk > > > > > > > > > > This is roughly 10% of memory usage reduction. > > > > > > > > > > Committed. > > > > > > > > In this case I have a little cleanup. > > > > This patch also avoids one strlen() per added word. > > > > > > How much is the speed gain? > > > > I can't see much of a difference. I'm trying with: > > > > --- a/src/html.ccSat Jun 21 15:00:35 2008 +0200 > > +++ b/src/html.ccSat Jun 21 22:27:30 2008 +0200 > > @@ -672,6 +672,7 @@ void DilloHtml::finishParsing(int Client > > > > freeParseData(); > > parse_finished = true; > > +exit(0); > > } > > > > /* > > > > and then > > > > time dillo-fltk manual.html, but the numbers are varying greatly - > > at least more than the speed gain, if there is any. > > Do you know a better method to measure the performance? perhaps > > parsing the document 10 times in a loop? > > I get consistent timing with uicmd.cc:647 set to 1. But that only works with images enabled, right? Then all timings are dominated by image decoding. Without images I get timings around 3s with images it's about 16s. But anyway, I agree that there is not much gain in avoiding that strlen call. So let's just forget about that for now. If we can find a test case where it really helps, we can come back to it. Cheers, Johannes

June 21, 2008

Textblock::addText()

by jcid＠dillo.org

On Sat, Jun 21, 2008 at 10:36:52PM +0200, Johannes Hofmann wrote: > On Sat, Jun 21, 2008 at 02:38:48PM -0400, Jorge Arellano Cid wrote: > > On Sat, Jun 21, 2008 at 06:23:18PM +0200, Johannes Hofmann wrote: > > > On Sat, Jun 21, 2008 at 11:31:56AM -0400, Jorge Arellano Cid wrote: > > > > On Fri, Jun 20, 2008 at 08:45:18PM +0200, Johannes Hofmann wrote: > > > > > Hi, > > > > > > > > > > On Fri, Jun 20, 2008 at 01:03:26PM -0400, Jorge Arellano Cid wrote: > > > > > > Hi there, > > > > > > > > > > > > - Changed Textblock::addText() to internally allocate its text string, > > > > > > making the memory handling opaque to the caller. > > > > > > > > > > > > Does anybody remmeber the topic of addText() deleting a malloc'ed > > > > > > block of memory? > > > > > > > > > > > > I decided to make the change for API cleanliness/correctness and set > > > > > > the Textblock do the whole memory handling (i.e. now the Textblock is > > > > > > the one that allocates/deallocates the text string). > > > > > > > > > > > > The HTML parser was adjusted a bit to keep its old performance. > > > > > > > > > > Very nice! > > > > > I thought this is the right moment to drag out my old text > > > > > allocation optimization. > > > > > And now it really makes a difference. > > > > > I see about 10M less memory usage on the mysql page and most > > > > > important, leaving that huge page is quite a bit faster - as there > > > > > is less free() calls. > > > > > The result may be completely different on Linux however. So please > > > > > give it a try on Linux. > > > > > > > > Here I have: > > > > > > > > Platform: Debian GNU/Linux, dillo-fltk with image_off > > > > Test : huge mysql page > > > > > > > > %MEM VSZ RSS TTY STAT START TIME COMMAND > > > > 0.1 9672 4144 pts/10 S+ 08:51 0:00 ./dillo-fltk.21Jun > > > > 5.9 147588 123896 pts/10 S+ 08:51 0:02 ./dillo-fltk.21Jun > > > > > > > > 0.1 9676 4136 pts/17 S+ 08:54 0:00 ./dillo-fltk > > > > 5.4 136684 112872 pts/17 S+ 08:54 0:02 ./dillo-fltk > > > > > > > > This is roughly 10% of memory usage reduction. > > > > > > > > Committed. > > > > > > In this case I have a little cleanup. > > > This patch also avoids one strlen() per added word. > > > > How much is the speed gain? > > I can't see much of a difference. I'm trying with: > > --- a/src/html.ccSat Jun 21 15:00:35 2008 +0200 > +++ b/src/html.ccSat Jun 21 22:27:30 2008 +0200 > @@ -672,6 +672,7 @@ void DilloHtml::finishParsing(int Client > > freeParseData(); > parse_finished = true; > +exit(0); > } > > /* > > and then > > time dillo-fltk manual.html, but the numbers are varying greatly - > at least more than the speed gain, if there is any. > Do you know a better method to measure the performance? perhaps > parsing the document 10 times in a loop? I get consistent timing with uicmd.cc:647 set to 1. -- Cheers Jorge.-

June 21, 2008

Textblock::addText()

by Johannes.Hofmann＠gmx.de

On Sat, Jun 21, 2008 at 02:38:48PM -0400, Jorge Arellano Cid wrote: > On Sat, Jun 21, 2008 at 06:23:18PM +0200, Johannes Hofmann wrote: > > On Sat, Jun 21, 2008 at 11:31:56AM -0400, Jorge Arellano Cid wrote: > > > On Fri, Jun 20, 2008 at 08:45:18PM +0200, Johannes Hofmann wrote: > > > > Hi, > > > > > > > > On Fri, Jun 20, 2008 at 01:03:26PM -0400, Jorge Arellano Cid wrote: > > > > > Hi there, > > > > > > > > > > - Changed Textblock::addText() to internally allocate its text string, > > > > > making the memory handling opaque to the caller. > > > > > > > > > > Does anybody remmeber the topic of addText() deleting a malloc'ed > > > > > block of memory? > > > > > > > > > > I decided to make the change for API cleanliness/correctness and set > > > > > the Textblock do the whole memory handling (i.e. now the Textblock is > > > > > the one that allocates/deallocates the text string). > > > > > > > > > > The HTML parser was adjusted a bit to keep its old performance. > > > > > > > > Very nice! > > > > I thought this is the right moment to drag out my old text > > > > allocation optimization. > > > > And now it really makes a difference. > > > > I see about 10M less memory usage on the mysql page and most > > > > important, leaving that huge page is quite a bit faster - as there > > > > is less free() calls. > > > > The result may be completely different on Linux however. So please > > > > give it a try on Linux. > > > > > > Here I have: > > > > > > Platform: Debian GNU/Linux, dillo-fltk with image_off > > > Test : huge mysql page > > > > > > %MEM VSZ RSS TTY STAT START TIME COMMAND > > > 0.1 9672 4144 pts/10 S+ 08:51 0:00 ./dillo-fltk.21Jun > > > 5.9 147588 123896 pts/10 S+ 08:51 0:02 ./dillo-fltk.21Jun > > > > > > 0.1 9676 4136 pts/17 S+ 08:54 0:00 ./dillo-fltk > > > 5.4 136684 112872 pts/17 S+ 08:54 0:02 ./dillo-fltk > > > > > > This is roughly 10% of memory usage reduction. > > > > > > Committed. > > > > In this case I have a little cleanup. > > This patch also avoids one strlen() per added word. > > How much is the speed gain? I can't see much of a difference. I'm trying with: --- a/src/html.ccSat Jun 21 15:00:35 2008 +0200 +++ b/src/html.ccSat Jun 21 22:27:30 2008 +0200 @@ -672,6 +672,7 @@ void DilloHtml::finishParsing(int Client freeParseData(); parse_finished = true; +exit(0); } /* and then time dillo-fltk manual.html, but the numbers are varying greatly - at least more than the speed gain, if there is any. Do you know a better method to measure the performance? perhaps parsing the document 10 times in a loop? > > I hesitate because this adds the overhead of one extra > parameter per word. Well, let's look at the numbers. > > You can test by overloading addText and changing the html.cc:1195 call. Cheers, Johannes

June 21, 2008

Textblock::addText()

by jcid＠dillo.org

On Sat, Jun 21, 2008 at 06:23:18PM +0200, Johannes Hofmann wrote: > On Sat, Jun 21, 2008 at 11:31:56AM -0400, Jorge Arellano Cid wrote: > > On Fri, Jun 20, 2008 at 08:45:18PM +0200, Johannes Hofmann wrote: > > > Hi, > > > > > > On Fri, Jun 20, 2008 at 01:03:26PM -0400, Jorge Arellano Cid wrote: > > > > Hi there, > > > > > > > > - Changed Textblock::addText() to internally allocate its text string, > > > > making the memory handling opaque to the caller. > > > > > > > > Does anybody remmeber the topic of addText() deleting a malloc'ed > > > > block of memory? > > > > > > > > I decided to make the change for API cleanliness/correctness and set > > > > the Textblock do the whole memory handling (i.e. now the Textblock is > > > > the one that allocates/deallocates the text string). > > > > > > > > The HTML parser was adjusted a bit to keep its old performance. > > > > > > Very nice! > > > I thought this is the right moment to drag out my old text > > > allocation optimization. > > > And now it really makes a difference. > > > I see about 10M less memory usage on the mysql page and most > > > important, leaving that huge page is quite a bit faster - as there > > > is less free() calls. > > > The result may be completely different on Linux however. So please > > > give it a try on Linux. > > > > Here I have: > > > > Platform: Debian GNU/Linux, dillo-fltk with image_off > > Test : huge mysql page > > > > %MEM VSZ RSS TTY STAT START TIME COMMAND > > 0.1 9672 4144 pts/10 S+ 08:51 0:00 ./dillo-fltk.21Jun > > 5.9 147588 123896 pts/10 S+ 08:51 0:02 ./dillo-fltk.21Jun > > > > 0.1 9676 4136 pts/17 S+ 08:54 0:00 ./dillo-fltk > > 5.4 136684 112872 pts/17 S+ 08:54 0:02 ./dillo-fltk > > > > This is roughly 10% of memory usage reduction. > > > > Committed. > > In this case I have a little cleanup. > This patch also avoids one strlen() per added word. How much is the speed gain? I hesitate because this adds the overhead of one extra parameter per word. Well, let's look at the numbers. You can test by overloading addText and changing the html.cc:1195 call. -- Cheers Jorge.-

June 21, 2008

Textblock::addText()

by Johannes.Hofmann＠gmx.de

On Sat, Jun 21, 2008 at 11:31:56AM -0400, Jorge Arellano Cid wrote: > On Fri, Jun 20, 2008 at 08:45:18PM +0200, Johannes Hofmann wrote: > > Hi, > > > > On Fri, Jun 20, 2008 at 01:03:26PM -0400, Jorge Arellano Cid wrote: > > > Hi there, > > > > > > - Changed Textblock::addText() to internally allocate its text string, > > > making the memory handling opaque to the caller. > > > > > > Does anybody remmeber the topic of addText() deleting a malloc'ed > > > block of memory? > > > > > > I decided to make the change for API cleanliness/correctness and set > > > the Textblock do the whole memory handling (i.e. now the Textblock is > > > the one that allocates/deallocates the text string). > > > > > > The HTML parser was adjusted a bit to keep its old performance. > > > > Very nice! > > I thought this is the right moment to drag out my old text > > allocation optimization. > > And now it really makes a difference. > > I see about 10M less memory usage on the mysql page and most > > important, leaving that huge page is quite a bit faster - as there > > is less free() calls. > > The result may be completely different on Linux however. So please > > give it a try on Linux. > > Here I have: > > Platform: Debian GNU/Linux, dillo-fltk with image_off > Test : huge mysql page > > %MEM VSZ RSS TTY STAT START TIME COMMAND > 0.1 9672 4144 pts/10 S+ 08:51 0:00 ./dillo-fltk.21Jun > 5.9 147588 123896 pts/10 S+ 08:51 0:02 ./dillo-fltk.21Jun > > 0.1 9676 4136 pts/17 S+ 08:54 0:00 ./dillo-fltk > 5.4 136684 112872 pts/17 S+ 08:54 0:02 ./dillo-fltk > > This is roughly 10% of memory usage reduction. > > Committed. In this case I have a little cleanup. This patch also avoids one strlen() per added word. Cheers, Johannes

June 21, 2008

Textblock::addText()

by jcid＠dillo.org

On Sat, Jun 21, 2008 at 11:31:56AM -0400, Jorge Arellano Cid wrote: > Here I have: > > Platform: Debian GNU/Linux, dillo-fltk with image_off > Test : huge mysql page > > %MEM VSZ RSS TTY STAT START TIME COMMAND > 0.1 9672 4144 pts/10 S+ 08:51 0:00 ./dillo-fltk.21Jun > 5.9 147588 123896 pts/10 S+ 08:51 0:02 ./dillo-fltk.21Jun > > 0.1 9676 4136 pts/17 S+ 08:54 0:00 ./dillo-fltk > 5.4 136684 112872 pts/17 S+ 08:54 0:02 ./dillo-fltk > > This is roughly 10% of memory usage reduction. > > Committed. BTW, here there is no noticeable speedup on leaving the page, and it is quite fast. -- Cheers Jorge.-

June 21, 2008