- Dillo-dev - lists.mailman3.com

valgrind complaints when running datauri

by corvid＠lavabit.com

Johannes wrote: > Unfortunately I can't reproduce the problem here, but I see an > issue in a_Url_decode_hex_str() (datauri.c): > In line 154 two possibly unrelated pointers are subtracted. Could > you try the following: It doesn't crash anymore! :)

Nov. 25, 2012

valgrind complaints when running datauri

by Johannes.Hofmann＠gmx.de

On Sun, Nov 25, 2012 at 07:46:42PM +0000, corvid wrote: > > Jeremy's valgrind logs have had a couple of these recently, e.g., > http://starurchin.org/dillo/valgrind/5b601af9ce7facc17690b88c4323c194fded01… > > I found that > - after dpidc stop > - valgrind --leak-check=full --trace-children=yes dillo > - going to http://jimbojw.com/wiki/index.php?title=Data_URI_Explained > - and pasting in the second example (the percent-encoded one, not the > base64 one) > gets me: > > ==20817== Invalid read of size 4 > ==20817== at 0x804AED5: dStr_insert_l (dlib.c:287) > ==20817== by 0x804AFB5: dStr_append_l (dlib.c:308) > ==20817== by 0x8049317: main (datauri.c:332) > ==20817== Address 0x41cbbb0 is 0 bytes after a block of size 664 alloc'd > ==20817== at 0x402AE29: realloc (vg_replace_malloc.c:662) > ==20817== by 0x804A736: dRealloc (dlib.c:53) > ==20817== by 0x804918B: datauri_get_data (datauri.c:273) > ==20817== by 0x80492E1: main (datauri.c:324) > ==20817== > ==20817== > ==20817== Process terminating with default action of signal 11 (SIGSEGV): dumping core > > > Valgrind has options for debugging, but I haven't had any luck with > getting them to work so far. > > I'm imagining that this might have to do with only being able to send > part of the data at a time when everything's sluggish due to valgrind, > but I don't know datauri/dpip well enough to go straight for the most > likely spots... > > PS I don't trust the "by 0x8049317: main (datauri.c:332)" line because > I have -O0, and sometimes it's had some 0x40... address instead that it > describes as being below main. Unfortunately I can't reproduce the problem here, but I see an issue in a_Url_decode_hex_str() (datauri.c): In line 154 two possibly unrelated pointers are subtracted. Could you try the following: diff -r 74833d000780 dpi/datauri.c --- a/dpi/datauri.couldThu Nov 15 23:09:50 2012 +0100 +++ b/dpi/datauri.couldSun Nov 25 21:52:22 2012 +0100 @@ -150,8 +150,8 @@ } *dest = 0; + *p_sz = (size_t)(dest - new_str); new_str = dRealloc(new_str, sizeof(char) * (dest - new_str + 1)); - *p_sz = (size_t)(dest - new_str); return new_str; } Cheers, Johannes

Nov. 25, 2012

valgrind complaints when running datauri

by corvid＠lavabit.com

Jeremy's valgrind logs have had a couple of these recently, e.g., http://starurchin.org/dillo/valgrind/5b601af9ce7facc17690b88c4323c194fded01… I found that - after dpidc stop - valgrind --leak-check=full --trace-children=yes dillo - going to http://jimbojw.com/wiki/index.php?title=Data_URI_Explained - and pasting in the second example (the percent-encoded one, not the base64 one) gets me: ==20817== Invalid read of size 4 ==20817== at 0x804AED5: dStr_insert_l (dlib.c:287) ==20817== by 0x804AFB5: dStr_append_l (dlib.c:308) ==20817== by 0x8049317: main (datauri.c:332) ==20817== Address 0x41cbbb0 is 0 bytes after a block of size 664 alloc'd ==20817== at 0x402AE29: realloc (vg_replace_malloc.c:662) ==20817== by 0x804A736: dRealloc (dlib.c:53) ==20817== by 0x804918B: datauri_get_data (datauri.c:273) ==20817== by 0x80492E1: main (datauri.c:324) ==20817== ==20817== ==20817== Process terminating with default action of signal 11 (SIGSEGV): dumping core Valgrind has options for debugging, but I haven't had any luck with getting them to work so far. I'm imagining that this might have to do with only being able to send part of the data at a time when everything's sluggish due to valgrind, but I don't know datauri/dpip well enough to go straight for the most likely spots... PS I don't trust the "by 0x8049317: main (datauri.c:332)" line because I have -O0, and sometimes it's had some 0x40... address instead that it describes as being below main. PPS I find myself wishing that we could have the dpis back in dillo.

Nov. 25, 2012

Extensions for hyphenation

by Johannes.Hofmann＠gmx.de

Hi Sebastian, On Sun, Nov 18, 2012 at 02:29:32PM +0100, Sebastian Geerken wrote: > Hi! > > At http://flpsed.org/hgweb/dillo_hyphen, you'll find some extensions > for hyphenation I've not yet merged into the main repository. Still > needs some documentation, but here is an overview: > > There are now configuration variables for dillorc (see source): > penalties for hyphens, as well as the left and right side of an > em-dash. The suffix "_2" means that this value is used for lines > following a line which ends already with a hyphen. When this value is > larger, two adjacent lines ending with a hyphen are avoided. > > For values, see the definition of the "badness". Typical values: > > 0 = Penalty used for normal spaces. > 1 = A justified line with spaces having 150% or 67% of the ideal space > width has this as badness. > 8 = A justified line with spaces twice as wide as ideally has this as > badness. > > "inf" may be used (preventing a break in any case); also "-inf" > (forcing a break), although the latter makes no sense and may lead to > strange results. > > There is a text page, test/hyphens-etc.html, to play around. This looks great! To me it doesn't look as if it would involve any risky stuff. What do you think about merging dillo_hyphen into main line? Or should we wait until after the release? The table column width issue you are discussing with Jorge also exists in main line, right? So we can improve on that there. Johannes

Nov. 25, 2012

Extensions for hyphenation

by jcid＠dillo.org

On Fri, Nov 23, 2012 at 01:05:39PM +0100, Sebastian Geerken wrote: > Hi Jorge, > > On Wed, Nov 21, Jorge Arellano Cid wrote: > > On Sun, Nov 18, 2012 at 02:29:32PM +0100, Sebastian Geerken wrote: > > [...] > > I wonder how breaking a single word in a line can be penalized > > with these controls. > > See below ... > > > For instance [1], with both main dillo and dillo_hyphen the > > word "hyphenation" is broken twice: > > > > hy- > > phen- > > ation > > > > With the new controls, it could become: > > > > hyphen- > > ation > > > > but, in this particular case it should have been: > > > > hyphenation > > That there is a difference between dillo and dillo_hyphen is probably > just accidental. Ack. > Calculating width extremes, and so table rendering, > is independent of the actual penalty values (except for the value > "inf"i; see below). I beg to differ. > [...] > Penaltie values cannot actually influence table rendering: > > 1. Column widths depend on column min/max widths and available width > (window width at the top). Agreed. > 2. Column min/max widths depend on cell min/max widths (maximum of the > respective values). Agreed. > 3. Cell widths (calculated in dw::Textblock::getExtremesImpl) do not > take the penalty values into account, only three cases are > distinguished: inf (no break allowed), -inf (break forced), and > other values (break possible.) Agreed. > So changing penalties (say, to 5 instead of 1), won't make a > difference. Here I differ. Please let me know what I'm missing. If a penalty can increase the badness of breaking certain word, to the point of it not being broken (ar at least be broken less times), e.g. becomes or hy- hyphenation hyphen- phen- ation ation and that word is the only word in a cell, and that cell is the widest in its column (as in the URL example [1]), then cell max is affected (as you explain in point 2), an so table rendering. > > [...] > > or even simpler, in characters: > > > > penalty_one_word_line=18 > > /* Don't try to break words shorter than 18 chars, when > > alone in a single line */ > > > > > > The advantage I see to a penalty that handles this case is that > > it can help a lot with web rendering and also with more precise > > book rendering with a simple dillorc option. > > Could be, but it sounds a bit hackish, and with an uncertain result. > Could be worth a test, however. It sounds a bit strange, but also the problem is: if we have to format text into a narrow column, breaking a long word to make it fit in, is clearly the way to go. Now, if breaking that word also makes the column narrower for the whole text (as narrow as the broken word's widest split part), then we have to choose which of both cases is better. The above dillorc setting is just a very simple (in the sense of: no need to calculate) heuristic to try to answer the question. > > These are just ideas, not meant to be *the* solution. They have > > relatively simple implementations that could be field tested. > > I've thought on another approach, which unfortunately turned out to be > more complicated than I first thought: calculating width extremes > without considering hyphenation. In this case, "hyphenation" is still > hyphenated, but the minimal width of this cell is based on the whole > word "hyphenation". This would have the following results: > > 1. table rendering is done the same way as by browsers not supporting > hyphenation, and, > 2. OTOH, hyphenation still works as desired. > > As I said, I stumbled about a difficult detail; I'd like to give it > another though. Yes, it would solve the problem, but leaving out the advantage when the answer to the above stated question, favors an hyphenated solution. I'd like to have the best choice rendered! ;-) -- Cheers Jorge.-

Nov. 24, 2012

Extensions for hyphenation

by sgeerken＠dillo.org

Hi Jorge, On Wed, Nov 21, Jorge Arellano Cid wrote: > On Sun, Nov 18, 2012 at 02:29:32PM +0100, Sebastian Geerken wrote: > > At http://flpsed.org/hgweb/dillo_hyphen, you'll find some extensions > > for hyphenation I've not yet merged into the main repository. Still > > needs some documentation, but here is an overview: > > > > There are now configuration variables for dillorc (see source): > > penalties for hyphens, as well as the left and right side of an > > em-dash. The suffix "_2" means that this value is used for lines > > following a line which ends already with a hyphen. When this value is > > larger, two adjacent lines ending with a hyphen are avoided. > > > > For values, see the definition of the "badness". Typical values: > > > > 0 = Penalty used for normal spaces. > > 1 = A justified line with spaces having 150% or 67% of the ideal space > > width has this as badness. > > 8 = A justified line with spaces twice as wide as ideally has this as > > badness. > > > > "inf" may be used (preventing a break in any case); also "-inf" > > (forcing a break), although the latter makes no sense and may lead to > > strange results. > > > > There is a text page, test/hyphens-etc.html, to play around. > > I wonder how breaking a single word in a line can be penalized > with these controls. See below ... > For instance [1], with both main dillo and dillo_hyphen the > word "hyphenation" is broken twice: > > hy- > phen- > ation > > With the new controls, it could become: > > hyphen- > ation > > but, in this particular case it should have been: > > hyphenation That there is a difference between dillo and dillo_hyphen is probably just accidental. Calculating width extremes, and so table rendering, is independent of the actual penalty values (except for the value "inf"i; see below). > In the same page, there're several cases of the same problem > (one line above, a 1 row x 8 col table) where words are broken > into a maximum of 4 times! > > In the web case, it's common to use the longest word in a line > as minimal width. OTOH there's also the problem of too long > "word" strings. I remember your other post some months ago. > In [1] clearly the browser tries to optimize for a minimal page > width. Which is not the case, but that could perfectly have been > as an external constraint to the algoritm (by means of screen > size, TABLE element directives, floats, etc). So it is non > trivial. > > I've worked enough on table rendering to know that making a > decision based on the current textblock's min/max width would > introduce too much complexity. e.g. in [1], just imagine the > problem of deciding which words to break and where for a dynamic > optimum of the table width. :-P > > A much simpler approach would be to introduce a penalty for > breaking single words in a line, above a certain threshold that > could be relative to the browser window's width. > > For instance: > > penalty_one_word_line=5 > /* Penalty = (word_length > 1/4 window width) ? 0 : 5 */ Penaltie values cannot actually influence table rendering: 1. Column widths depend on column min/max widths and available width (window width at the top). 2. Column min/max widths depend on cell min/max widths (maximum of the respective values). 3. Cell widths (calculated in dw::Textblock::getExtremesImpl) do not take the penalty values into account, only three cases are distinguished: inf (no break allowed), -inf (break forced), and other values (break possible.) So changing penalties (say, to 5 instead of 1), won't make a difference. > or even simpler, in characters: > > penalty_one_word_line=18 > /* Don't try to break words shorter than 18 chars, when > alone in a single line */ > > > The advantage I see to a penalty that handles this case is that > it can help a lot with web rendering and also with more precise > book rendering with a simple dillorc option. Could be, but it sounds a bit hackish, and with an uncertain result. Could be worth a test, however. > These are just ideas, not meant to be *the* solution. They have > relatively simple implementations that could be field tested. I've thought on another approach, which unfortunately turned out to be more complicated than I first thought: calculating width extremes without considering hyphenation. In this case, "hyphenation" is still hyphenated, but the minimal width of this cell is based on the whole word "hyphenation". This would have the following results: 1. table rendering is done the same way as by browsers not supporting hyphenation, and, 2. OTOH, hyphenation still works as desired. As I said, I stumbled about a difficult detail; I'd like to give it another though. Sebastian

Nov. 23, 2012

Extensions for hyphenation

by jcid＠dillo.org

Hi Sebastian, On Sun, Nov 18, 2012 at 02:29:32PM +0100, Sebastian Geerken wrote: > Hi! > > At http://flpsed.org/hgweb/dillo_hyphen, you'll find some extensions > for hyphenation I've not yet merged into the main repository. Still > needs some documentation, but here is an overview: > > There are now configuration variables for dillorc (see source): > penalties for hyphens, as well as the left and right side of an > em-dash. The suffix "_2" means that this value is used for lines > following a line which ends already with a hyphen. When this value is > larger, two adjacent lines ending with a hyphen are avoided. > > For values, see the definition of the "badness". Typical values: > > 0 = Penalty used for normal spaces. > 1 = A justified line with spaces having 150% or 67% of the ideal space > width has this as badness. > 8 = A justified line with spaces twice as wide as ideally has this as > badness. > > "inf" may be used (preventing a break in any case); also "-inf" > (forcing a break), although the latter makes no sense and may lead to > strange results. > > There is a text page, test/hyphens-etc.html, to play around. I wonder how breaking a single word in a line can be penalized with these controls. For instance [1], with both main dillo and dillo_hyphen the word "hyphenation" is broken twice: hy- phen- ation With the new controls, it could become: hyphen- ation but, in this particular case it should have been: hyphenation In the same page, there're several cases of the same problem (one line above, a 1 row x 8 col table) where words are broken into a maximum of 4 times! In the web case, it's common to use the longest word in a line as minimal width. OTOH there's also the problem of too long "word" strings. In [1] clearly the browser tries to optimize for a minimal page width. Which is not the case, but that could perfectly have been as an external constraint to the algoritm (by means of screen size, TABLE element directives, floats, etc). So it is non trivial. I've worked enough on table rendering to know that making a decision based on the current textblock's min/max width would introduce too much complexity. e.g. in [1], just imagine the problem of deciding which words to break and where for a dynamic optimum of the table width. :-P A much simpler approach would be to introduce a penalty for breaking single words in a line, above a certain threshold that could be relative to the browser window's width. For instance: penalty_one_word_line=5 /* Penalty = (word_length > 1/4 window width) ? 0 : 5 */ or even simpler, in characters: penalty_one_word_line=18 /* Don't try to break words shorter than 18 chars, when alone in a single line */ The advantage I see to a penalty that handles this case is that it can help a lot with web rendering and also with more precise book rendering with a simple dillorc option. These are just ideas, not meant to be *the* solution. They have relatively simple implementations that could be field tested. HTH. [1] http://www.thefreedictionary.com/hyphenation -- Cheers Jorge.-

Nov. 21, 2012

missing redraw?

by Johannes.Hofmann＠gmx.de

On Sun, Nov 18, 2012 at 08:18:13AM +0000, corvid wrote: > > I was looking at the source for http://omicsomics.blogspot.com/ , > and there was a little bit of first-column background color visible in > the second column. Switching to another tab and back shows that the > first column is really a little wider than what is initially seen. > > This problem appears to pre-date the inclusion of hyphenation. I can't reproduce it here for some reason, but I suspect that in dw/table.cc redrawY is not reset properly when the first column get's wider as line numbers have more digits.

Nov. 18, 2012

Extensions for hyphenation

by sgeerken＠dillo.org

Hi! At http://flpsed.org/hgweb/dillo_hyphen, you'll find some extensions for hyphenation I've not yet merged into the main repository. Still needs some documentation, but here is an overview: There are now configuration variables for dillorc (see source): penalties for hyphens, as well as the left and right side of an em-dash. The suffix "_2" means that this value is used for lines following a line which ends already with a hyphen. When this value is larger, two adjacent lines ending with a hyphen are avoided. For values, see the definition of the "badness". Typical values: 0 = Penalty used for normal spaces. 1 = A justified line with spaces having 150% or 67% of the ideal space width has this as badness. 8 = A justified line with spaces twice as wide as ideally has this as badness. "inf" may be used (preventing a break in any case); also "-inf" (forcing a break), although the latter makes no sense and may lead to strange results. There is a text page, test/hyphens-etc.html, to play around. Sebastian

Nov. 18, 2012

missing redraw?

by corvid＠lavabit.com

I was looking at the source for http://omicsomics.blogspot.com/ , and there was a little bit of first-column background color visible in the second column. Switching to another tab and back shows that the first column is really a little wider than what is initially seen. This problem appears to pre-date the inclusion of hyphenation.

Nov. 18, 2012