Hi there, - Changed Textblock::addText() to internally allocate its text string, making the memory handling opaque to the caller. Does anybody remmeber the topic of addText() deleting a malloc'ed block of memory? I decided to make the change for API cleanliness/correctness and set the Textblock do the whole memory handling (i.e. now the Textblock is the one that allocates/deallocates the text string). The HTML parser was adjusted a bit to keep its old performance. -- Cheers Jorge.-
Hi, On Fri, Jun 20, 2008 at 01:03:26PM -0400, Jorge Arellano Cid wrote:
Hi there,
- Changed Textblock::addText() to internally allocate its text string, making the memory handling opaque to the caller.
Does anybody remmeber the topic of addText() deleting a malloc'ed block of memory?
I decided to make the change for API cleanliness/correctness and set the Textblock do the whole memory handling (i.e. now the Textblock is the one that allocates/deallocates the text string).
The HTML parser was adjusted a bit to keep its old performance.
Very nice! I thought this is the right moment to drag out my old text allocation optimization. And now it really makes a difference. I see about 10M less memory usage on the mysql page and most important, leaving that huge page is quite a bit faster - as there is less free() calls. The result may be completely different on Linux however. So please give it a try on Linux. Cheers, Johannes
On Fri, Jun 20, 2008 at 08:45:18PM +0200, Johannes Hofmann wrote:
Hi,
On Fri, Jun 20, 2008 at 01:03:26PM -0400, Jorge Arellano Cid wrote:
Hi there,
- Changed Textblock::addText() to internally allocate its text string, making the memory handling opaque to the caller.
Does anybody remmeber the topic of addText() deleting a malloc'ed block of memory?
I decided to make the change for API cleanliness/correctness and set the Textblock do the whole memory handling (i.e. now the Textblock is the one that allocates/deallocates the text string).
The HTML parser was adjusted a bit to keep its old performance.
Very nice! I thought this is the right moment to drag out my old text allocation optimization. And now it really makes a difference. I see about 10M less memory usage on the mysql page and most important, leaving that huge page is quite a bit faster - as there is less free() calls. The result may be completely different on Linux however. So please give it a try on Linux.
Here I have: Platform: Debian GNU/Linux, dillo-fltk with image_off Test : huge mysql page %MEM VSZ RSS TTY STAT START TIME COMMAND 0.1 9672 4144 pts/10 S+ 08:51 0:00 ./dillo-fltk.21Jun 5.9 147588 123896 pts/10 S+ 08:51 0:02 ./dillo-fltk.21Jun 0.1 9676 4136 pts/17 S+ 08:54 0:00 ./dillo-fltk 5.4 136684 112872 pts/17 S+ 08:54 0:02 ./dillo-fltk This is roughly 10% of memory usage reduction. Committed. -- Cheers Jorge.-
On Sat, Jun 21, 2008 at 11:31:56AM -0400, Jorge Arellano Cid wrote:
Here I have:
Platform: Debian GNU/Linux, dillo-fltk with image_off Test : huge mysql page
%MEM VSZ RSS TTY STAT START TIME COMMAND 0.1 9672 4144 pts/10 S+ 08:51 0:00 ./dillo-fltk.21Jun 5.9 147588 123896 pts/10 S+ 08:51 0:02 ./dillo-fltk.21Jun
0.1 9676 4136 pts/17 S+ 08:54 0:00 ./dillo-fltk 5.4 136684 112872 pts/17 S+ 08:54 0:02 ./dillo-fltk
This is roughly 10% of memory usage reduction.
Committed.
BTW, here there is no noticeable speedup on leaving the page, and it is quite fast. -- Cheers Jorge.-
On Sat, Jun 21, 2008 at 11:31:56AM -0400, Jorge Arellano Cid wrote:
On Fri, Jun 20, 2008 at 08:45:18PM +0200, Johannes Hofmann wrote:
Hi,
On Fri, Jun 20, 2008 at 01:03:26PM -0400, Jorge Arellano Cid wrote:
Hi there,
- Changed Textblock::addText() to internally allocate its text string, making the memory handling opaque to the caller.
Does anybody remmeber the topic of addText() deleting a malloc'ed block of memory?
I decided to make the change for API cleanliness/correctness and set the Textblock do the whole memory handling (i.e. now the Textblock is the one that allocates/deallocates the text string).
The HTML parser was adjusted a bit to keep its old performance.
Very nice! I thought this is the right moment to drag out my old text allocation optimization. And now it really makes a difference. I see about 10M less memory usage on the mysql page and most important, leaving that huge page is quite a bit faster - as there is less free() calls. The result may be completely different on Linux however. So please give it a try on Linux.
Here I have:
Platform: Debian GNU/Linux, dillo-fltk with image_off Test : huge mysql page
%MEM VSZ RSS TTY STAT START TIME COMMAND 0.1 9672 4144 pts/10 S+ 08:51 0:00 ./dillo-fltk.21Jun 5.9 147588 123896 pts/10 S+ 08:51 0:02 ./dillo-fltk.21Jun
0.1 9676 4136 pts/17 S+ 08:54 0:00 ./dillo-fltk 5.4 136684 112872 pts/17 S+ 08:54 0:02 ./dillo-fltk
This is roughly 10% of memory usage reduction.
Committed.
In this case I have a little cleanup. This patch also avoids one strlen() per added word. Cheers, Johannes
On Sat, Jun 21, 2008 at 06:23:18PM +0200, Johannes Hofmann wrote:
On Sat, Jun 21, 2008 at 11:31:56AM -0400, Jorge Arellano Cid wrote:
On Fri, Jun 20, 2008 at 08:45:18PM +0200, Johannes Hofmann wrote:
Hi,
On Fri, Jun 20, 2008 at 01:03:26PM -0400, Jorge Arellano Cid wrote:
Hi there,
- Changed Textblock::addText() to internally allocate its text string, making the memory handling opaque to the caller.
Does anybody remmeber the topic of addText() deleting a malloc'ed block of memory?
I decided to make the change for API cleanliness/correctness and set the Textblock do the whole memory handling (i.e. now the Textblock is the one that allocates/deallocates the text string).
The HTML parser was adjusted a bit to keep its old performance.
Very nice! I thought this is the right moment to drag out my old text allocation optimization. And now it really makes a difference. I see about 10M less memory usage on the mysql page and most important, leaving that huge page is quite a bit faster - as there is less free() calls. The result may be completely different on Linux however. So please give it a try on Linux.
Here I have:
Platform: Debian GNU/Linux, dillo-fltk with image_off Test : huge mysql page
%MEM VSZ RSS TTY STAT START TIME COMMAND 0.1 9672 4144 pts/10 S+ 08:51 0:00 ./dillo-fltk.21Jun 5.9 147588 123896 pts/10 S+ 08:51 0:02 ./dillo-fltk.21Jun
0.1 9676 4136 pts/17 S+ 08:54 0:00 ./dillo-fltk 5.4 136684 112872 pts/17 S+ 08:54 0:02 ./dillo-fltk
This is roughly 10% of memory usage reduction.
Committed.
In this case I have a little cleanup. This patch also avoids one strlen() per added word.
How much is the speed gain? I hesitate because this adds the overhead of one extra parameter per word. Well, let's look at the numbers. You can test by overloading addText and changing the html.cc:1195 call. -- Cheers Jorge.-
On Sat, Jun 21, 2008 at 02:38:48PM -0400, Jorge Arellano Cid wrote:
On Sat, Jun 21, 2008 at 06:23:18PM +0200, Johannes Hofmann wrote:
On Sat, Jun 21, 2008 at 11:31:56AM -0400, Jorge Arellano Cid wrote:
On Fri, Jun 20, 2008 at 08:45:18PM +0200, Johannes Hofmann wrote:
Hi,
On Fri, Jun 20, 2008 at 01:03:26PM -0400, Jorge Arellano Cid wrote:
Hi there,
- Changed Textblock::addText() to internally allocate its text string, making the memory handling opaque to the caller.
Does anybody remmeber the topic of addText() deleting a malloc'ed block of memory?
I decided to make the change for API cleanliness/correctness and set the Textblock do the whole memory handling (i.e. now the Textblock is the one that allocates/deallocates the text string).
The HTML parser was adjusted a bit to keep its old performance.
Very nice! I thought this is the right moment to drag out my old text allocation optimization. And now it really makes a difference. I see about 10M less memory usage on the mysql page and most important, leaving that huge page is quite a bit faster - as there is less free() calls. The result may be completely different on Linux however. So please give it a try on Linux.
Here I have:
Platform: Debian GNU/Linux, dillo-fltk with image_off Test : huge mysql page
%MEM VSZ RSS TTY STAT START TIME COMMAND 0.1 9672 4144 pts/10 S+ 08:51 0:00 ./dillo-fltk.21Jun 5.9 147588 123896 pts/10 S+ 08:51 0:02 ./dillo-fltk.21Jun
0.1 9676 4136 pts/17 S+ 08:54 0:00 ./dillo-fltk 5.4 136684 112872 pts/17 S+ 08:54 0:02 ./dillo-fltk
This is roughly 10% of memory usage reduction.
Committed.
In this case I have a little cleanup. This patch also avoids one strlen() per added word.
How much is the speed gain?
I can't see much of a difference. I'm trying with: --- a/src/html.ccSat Jun 21 15:00:35 2008 +0200 +++ b/src/html.ccSat Jun 21 22:27:30 2008 +0200 @@ -672,6 +672,7 @@ void DilloHtml::finishParsing(int Client freeParseData(); parse_finished = true; +exit(0); } /* and then time dillo-fltk manual.html, but the numbers are varying greatly - at least more than the speed gain, if there is any. Do you know a better method to measure the performance? perhaps parsing the document 10 times in a loop?
I hesitate because this adds the overhead of one extra parameter per word. Well, let's look at the numbers.
You can test by overloading addText and changing the html.cc:1195 call.
Cheers, Johannes
On Sat, Jun 21, 2008 at 10:36:52PM +0200, Johannes Hofmann wrote:
On Sat, Jun 21, 2008 at 02:38:48PM -0400, Jorge Arellano Cid wrote:
On Sat, Jun 21, 2008 at 06:23:18PM +0200, Johannes Hofmann wrote:
On Sat, Jun 21, 2008 at 11:31:56AM -0400, Jorge Arellano Cid wrote:
On Fri, Jun 20, 2008 at 08:45:18PM +0200, Johannes Hofmann wrote:
Hi,
On Fri, Jun 20, 2008 at 01:03:26PM -0400, Jorge Arellano Cid wrote:
Hi there,
- Changed Textblock::addText() to internally allocate its text string, making the memory handling opaque to the caller.
Does anybody remmeber the topic of addText() deleting a malloc'ed block of memory?
I decided to make the change for API cleanliness/correctness and set the Textblock do the whole memory handling (i.e. now the Textblock is the one that allocates/deallocates the text string).
The HTML parser was adjusted a bit to keep its old performance.
Very nice! I thought this is the right moment to drag out my old text allocation optimization. And now it really makes a difference. I see about 10M less memory usage on the mysql page and most important, leaving that huge page is quite a bit faster - as there is less free() calls. The result may be completely different on Linux however. So please give it a try on Linux.
Here I have:
Platform: Debian GNU/Linux, dillo-fltk with image_off Test : huge mysql page
%MEM VSZ RSS TTY STAT START TIME COMMAND 0.1 9672 4144 pts/10 S+ 08:51 0:00 ./dillo-fltk.21Jun 5.9 147588 123896 pts/10 S+ 08:51 0:02 ./dillo-fltk.21Jun
0.1 9676 4136 pts/17 S+ 08:54 0:00 ./dillo-fltk 5.4 136684 112872 pts/17 S+ 08:54 0:02 ./dillo-fltk
This is roughly 10% of memory usage reduction.
Committed.
In this case I have a little cleanup. This patch also avoids one strlen() per added word.
How much is the speed gain?
I can't see much of a difference. I'm trying with:
--- a/src/html.ccSat Jun 21 15:00:35 2008 +0200 +++ b/src/html.ccSat Jun 21 22:27:30 2008 +0200 @@ -672,6 +672,7 @@ void DilloHtml::finishParsing(int Client
freeParseData(); parse_finished = true; +exit(0); }
/*
and then
time dillo-fltk manual.html, but the numbers are varying greatly - at least more than the speed gain, if there is any. Do you know a better method to measure the performance? perhaps parsing the document 10 times in a loop?
I get consistent timing with uicmd.cc:647 set to 1. -- Cheers Jorge.-
On Sat, Jun 21, 2008 at 04:58:50PM -0400, Jorge Arellano Cid wrote:
On Sat, Jun 21, 2008 at 10:36:52PM +0200, Johannes Hofmann wrote:
On Sat, Jun 21, 2008 at 02:38:48PM -0400, Jorge Arellano Cid wrote:
On Sat, Jun 21, 2008 at 06:23:18PM +0200, Johannes Hofmann wrote:
On Sat, Jun 21, 2008 at 11:31:56AM -0400, Jorge Arellano Cid wrote:
On Fri, Jun 20, 2008 at 08:45:18PM +0200, Johannes Hofmann wrote:
Hi,
On Fri, Jun 20, 2008 at 01:03:26PM -0400, Jorge Arellano Cid wrote: > Hi there, > > - Changed Textblock::addText() to internally allocate its text string, > making the memory handling opaque to the caller. > > Does anybody remmeber the topic of addText() deleting a malloc'ed > block of memory? > > I decided to make the change for API cleanliness/correctness and set > the Textblock do the whole memory handling (i.e. now the Textblock is > the one that allocates/deallocates the text string). > > The HTML parser was adjusted a bit to keep its old performance.
Very nice! I thought this is the right moment to drag out my old text allocation optimization. And now it really makes a difference. I see about 10M less memory usage on the mysql page and most important, leaving that huge page is quite a bit faster - as there is less free() calls. The result may be completely different on Linux however. So please give it a try on Linux.
Here I have:
Platform: Debian GNU/Linux, dillo-fltk with image_off Test : huge mysql page
%MEM VSZ RSS TTY STAT START TIME COMMAND 0.1 9672 4144 pts/10 S+ 08:51 0:00 ./dillo-fltk.21Jun 5.9 147588 123896 pts/10 S+ 08:51 0:02 ./dillo-fltk.21Jun
0.1 9676 4136 pts/17 S+ 08:54 0:00 ./dillo-fltk 5.4 136684 112872 pts/17 S+ 08:54 0:02 ./dillo-fltk
This is roughly 10% of memory usage reduction.
Committed.
In this case I have a little cleanup. This patch also avoids one strlen() per added word.
How much is the speed gain?
I can't see much of a difference. I'm trying with:
--- a/src/html.ccSat Jun 21 15:00:35 2008 +0200 +++ b/src/html.ccSat Jun 21 22:27:30 2008 +0200 @@ -672,6 +672,7 @@ void DilloHtml::finishParsing(int Client
freeParseData(); parse_finished = true; +exit(0); }
/*
and then
time dillo-fltk manual.html, but the numbers are varying greatly - at least more than the speed gain, if there is any. Do you know a better method to measure the performance? perhaps parsing the document 10 times in a loop?
I get consistent timing with uicmd.cc:647 set to 1.
But that only works with images enabled, right? Then all timings are dominated by image decoding. Without images I get timings around 3s with images it's about 16s. But anyway, I agree that there is not much gain in avoiding that strlen call. So let's just forget about that for now. If we can find a test case where it really helps, we can come back to it. Cheers, Johannes
On Sat, Jun 21, 2008 at 11:12:34PM +0200, Johannes Hofmann wrote:
On Sat, Jun 21, 2008 at 04:58:50PM -0400, Jorge Arellano Cid wrote:
On Sat, Jun 21, 2008 at 10:36:52PM +0200, Johannes Hofmann wrote:
On Sat, Jun 21, 2008 at 02:38:48PM -0400, Jorge Arellano Cid wrote:
On Sat, Jun 21, 2008 at 06:23:18PM +0200, Johannes Hofmann wrote:
On Sat, Jun 21, 2008 at 11:31:56AM -0400, Jorge Arellano Cid wrote:
On Fri, Jun 20, 2008 at 08:45:18PM +0200, Johannes Hofmann wrote: > Hi, > > On Fri, Jun 20, 2008 at 01:03:26PM -0400, Jorge Arellano Cid wrote: > > Hi there, > > > > - Changed Textblock::addText() to internally allocate its text string, > > making the memory handling opaque to the caller. > > > > Does anybody remmeber the topic of addText() deleting a malloc'ed > > block of memory? > > > > I decided to make the change for API cleanliness/correctness and set > > the Textblock do the whole memory handling (i.e. now the Textblock is > > the one that allocates/deallocates the text string). > > > > The HTML parser was adjusted a bit to keep its old performance. > > Very nice! > I thought this is the right moment to drag out my old text > allocation optimization. > And now it really makes a difference. > I see about 10M less memory usage on the mysql page and most > important, leaving that huge page is quite a bit faster - as there > is less free() calls. > The result may be completely different on Linux however. So please > give it a try on Linux.
Here I have:
Platform: Debian GNU/Linux, dillo-fltk with image_off Test : huge mysql page
%MEM VSZ RSS TTY STAT START TIME COMMAND 0.1 9672 4144 pts/10 S+ 08:51 0:00 ./dillo-fltk.21Jun 5.9 147588 123896 pts/10 S+ 08:51 0:02 ./dillo-fltk.21Jun
0.1 9676 4136 pts/17 S+ 08:54 0:00 ./dillo-fltk 5.4 136684 112872 pts/17 S+ 08:54 0:02 ./dillo-fltk
This is roughly 10% of memory usage reduction.
Committed.
In this case I have a little cleanup. This patch also avoids one strlen() per added word.
How much is the speed gain?
I can't see much of a difference. I'm trying with:
--- a/src/html.ccSat Jun 21 15:00:35 2008 +0200 +++ b/src/html.ccSat Jun 21 22:27:30 2008 +0200 @@ -672,6 +672,7 @@ void DilloHtml::finishParsing(int Client
freeParseData(); parse_finished = true; +exit(0); }
/*
and then
time dillo-fltk manual.html, but the numbers are varying greatly - at least more than the speed gain, if there is any. Do you know a better method to measure the performance? perhaps parsing the document 10 times in a loop?
I get consistent timing with uicmd.cc:647 set to 1.
But that only works with images enabled, right? Then all timings are dominated by image decoding. Without images I get timings around 3s with images it's about 16s. But anyway, I agree that there is not much gain in avoiding that strlen call. So let's just forget about that for now. If we can find a test case where it really helps, we can come back to it.
Sorry, I just committed a halt that works both with/without images. (same line). I'm investigating on memory usage, and have further reduced it, but what I'm about to post for discussion is really weird. Stay tuned... -- Cheers Jorge.-
participants (2)
-
jcid＠dillo.org -
Johannes.Hofmann＠gmx.de