- Dillo-dev - lists.mailman3.com

Re: Dillo not rendering HTML with comments before <!DOCTYPE>

by Rodrigo Arias

Hi, On Thu, Apr 25, 2024 at 09:23:38AM +1000, Kevin Koster wrote: >Thanks for the explanation, this also makes clearer an issue I had with >XHTML image indexes generated by ImageMagick Montage which were getting >(by an unusual sequence of events) the incorrect HTTP Content-Type type >of "text/xml" (and they don't contain a meta tag). They'd load properly >via file:// but show as text over http://. Now I know to ideally force >the HTTP Content-Type to "application/xhtml+xml" instead of "text/html" >which I used to fix the problem originally. For Dillo, "application/xhtml+xml" and "text/html" are handled by the same HTML parser, which later identifies which version of HTML/XHTML is the document, based on the doctype. The problem is failing to set the content type to any of those two, like when using "text/xml". AFIK, the proper content type for XHTML is "application/xhtml+xml", which should be set on the HTTP Content-Type header. >> Regarding the type guessing bug, I think I can improve it by assuming >> that if we find the "<!doctype html" string in the first 1024 bytes or >> so, it is an HTML-like type, but it incurrs in more overhead. > >But if it aborts that search upon encountering the first thing that >isn't "spaces, newlines, tabs, and comments", most text files will be >detected within the first few bytes. > >I'm not sure how that approach would work with ImageMagick image index >XHTML pages which start like this though: > <?xml version="1.0" encoding="US-ASCII"?> > <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" > "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> > >Example: >http://www.ombertech.com/cnk/dillo/STS-133_Pictures/photo_index.html > >I don't really understand how XHTML is supposed to work, and I don't >have time to learn, so perhaps I'm ignoring a distinction between >differet flavours of XHTML that can begin in different ways? Anyway I >like how ImageMagick image map pages are viewable in Dillo at the >moment. We can improve the content detection to handle both HTML and XML-style comments, but I prefer to defer it after the 3.1.0 release. Websites shouldn't rely on the browser to guess the content type, it should be stated in the HTTP header or the meta tag. So I don't consider this a priority that should block the release for longer. If you want to work on it, feel free to do so :-) >> So I think for now we can rely on the correction of "text/xhtml" to >> "application/xhtml+xml", which seems to work fine. I don't like adding >> quirks, but I will keep this one as it was already there. Here is the >> PR: >> >> https://github.com/dillo-browser/dillo/pull/140 > >I've built Dillo from that branch and pages on www.lemis.com now render >correctly, thanks! If I save the homepage as lemis.xhtml it still shows >as plain text when loaded with file://, though it is rendered if the >comments before <!DOCTYPE> are removed or if the original file is saved >as lemis.html. Not much of an issue, but it could cause confusion for >someone. I pushed another patch that should fix this issue. It is caused primarily by the ".xhtml" extension not being recognized by the file plugin, which then tries to detect the doctype and fails in the same way, falling back to text/plain. Best, Rodrigo.

April 25, 2024

Re: Dillo not rendering HTML with comments before <!DOCTYPE>

by Kevin Koster

On Thu, 25 Apr 2024 09:23:38 +1000 Kevin Koster <dillo(a)ombertech.com> wrote: > On Tue, 23 Apr 2024 23:29:45 +0200 > Rodrigo Arias <rodarima(a)gmail.com> wrote: > > Regarding the type guessing bug, I think I can improve it by > > assuming that if we find the "<!doctype html" string in the first > > 1024 bytes or so, it is an HTML-like type, but it incurrs in more > > overhead. > > But if it aborts that search upon encountering the first thing that > isn't "spaces, newlines, tabs, and comments", most text files will be > detected within the first few bytes. > > I'm not sure how that approach would work with ImageMagick image index > XHTML pages which start like this though: > <?xml version="1.0" encoding="US-ASCII"?> > <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" > "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> > > Example: > http://www.ombertech.com/cnk/dillo/STS-133_Pictures/photo_index.html > > I don't really understand how XHTML is supposed to work, and I don't > have time to learn, so perhaps I'm ignoring a distinction between > differet flavours of XHTML that can begin in different ways? Anyway I > like how ImageMagick image map pages are viewable in Dillo at the > moment. It seems that the text/html type can be valid for XHTML documents, and the relevent RFC 2854 has a section on recognising HTML and XHTML files: 5. Recognizing HTML files Almost all HTML files have the string "<html" or "<HTML" near the beginning of the file. Documents conformant to HTML 2.0, HTML 3.2 and HTML 4.0 will start with a DOCTYPE declaration "<!DOCTYPE HTML" near the beginning, before the "<html". These dialects are case insensitive. Files may start with white space, comments (introduced by "<!--" ), or processing instructions (introduced by "<?") prior to the DOCTYPE declaration. XHTML documents (optionally) start with an XML declaration which begins with "<?xml" and are required to have a DOCTYPE declaration "<!DOCTYPE html". https://www.ietf.org/rfc/rfc2854.txt Possibly old news for others, but it clears up some of my own XHTML-ignorant confusions. For Dillo it doesn't look like it would harm performance much to add detection of comments and "<? >" on top of the existing detection of whitespace before looking for tags that indicate HTML-compatible content in misc.c. For non-(X)HTML data it will usually only mean checking for '<' as well as whitespace before it finds a byte that shouldn't be in a HTML-compatible document before the first tag. If it does find a '<' first then it will be a little more complicated to check and skip following characters. But only XML documents would normally have much of that and yet still end up displayed as plain text like they are already, so it seems like that would be a rare (and anyway minimal) performance issue.

April 25, 2024

Re: Dillo not rendering HTML with comments before <!DOCTYPE>

by Kevin Koster

On Tue, 23 Apr 2024 23:29:45 +0200 Rodrigo Arias <rodarima(a)gmail.com> wrote: > On Sat, Apr 20, 2024 at 02:35:10PM +0200, Rodrigo Arias wrote: > >Yeah, the current detection mechanism in Dillo for content types is > >not very good. It searches for the doctype line at the beginning of > >the document[1] but it doesn't handle comments. > > > >[1]:https://github.com/dillo-browser/dillo/blob/v3.1.0-rc1/src/misc. > >c#L148 > > > >We should rely on the Content-Type provided by the server, or at > >least improve the detection. > > So, this is a tricky case. > > Dillo has several content types for a single document sorted by > priority, the first one set defines the content type of the document: > > 1. The "override type" used to override the type (highest priority) > 2. The "meta type" given by the <meta ... content="..."> tag in HTML > 3. The "http type" given by the HTTP Content-Type header > 4. The "guessed type" based on the document data (lowest priority) Thanks for the explanation, this also makes clearer an issue I had with XHTML image indexes generated by ImageMagick Montage which were getting (by an unusual sequence of events) the incorrect HTTP Content-Type type of "text/xml" (and they don't contain a meta tag). They'd load properly via file:// but show as text over http://. Now I know to ideally force the HTTP Content-Type to "application/xhtml+xml" instead of "text/html" which I used to fix the problem originally. > Regarding the type guessing bug, I think I can improve it by assuming > that if we find the "<!doctype html" string in the first 1024 bytes or > so, it is an HTML-like type, but it incurrs in more overhead. But if it aborts that search upon encountering the first thing that isn't "spaces, newlines, tabs, and comments", most text files will be detected within the first few bytes. I'm not sure how that approach would work with ImageMagick image index XHTML pages which start like this though: <?xml version="1.0" encoding="US-ASCII"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> Example: http://www.ombertech.com/cnk/dillo/STS-133_Pictures/photo_index.html I don't really understand how XHTML is supposed to work, and I don't have time to learn, so perhaps I'm ignoring a distinction between differet flavours of XHTML that can begin in different ways? Anyway I like how ImageMagick image map pages are viewable in Dillo at the moment. > So I think for now we can rely on the correction of "text/xhtml" to > "application/xhtml+xml", which seems to work fine. I don't like adding > quirks, but I will keep this one as it was already there. Here is the > PR: > > https://github.com/dillo-browser/dillo/pull/140 I've built Dillo from that branch and pages on www.lemis.com now render correctly, thanks! If I save the homepage as lemis.xhtml it still shows as plain text when loaded with file://, though it is rendered if the comments before <!DOCTYPE> are removed or if the original file is saved as lemis.html. Not much of an issue, but it could cause confusion for someone.

April 24, 2024

Re: Dillo not rendering HTML with comments before <!DOCTYPE>

by Rodrigo Arias

Hi, On Sat, Apr 20, 2024 at 02:35:10PM +0200, Rodrigo Arias wrote: >Hi, > >On Sat, Apr 20, 2024 at 02:00:05PM +1000, Kevin Koster wrote: >>This problem was present in 3.0.5 as well as in 3.1.0-rc1. >> >>URL: http://www.lemis.com/ >>CSS: enabled or disabled >>Summary: Won't render HTML with comments before <!DOCTYPE>. >>Pages on this website aren't rendered, just displayed as source code. >>Although they are XHTML, this doesn't appear to be due to this bug: >>https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1036382 >>If I save the file as lemis.xhtml and remove the two comment lines >>before the <!DOCTYPE> declaration, then it renders when I load it with >>file:// or http://, otherwise it doesn't. >> >>My reading is that comments there are valid for HTML 4 (which is >>declared in the page's <!DOCTYPE>) since the standard says: >>White space (spaces, newlines, tabs, and comments) may appear before or >>after each section. >>https://www.w3.org/TR/html401/struct/global.html#h-7.1 > >Yeah, the current detection mechanism in Dillo for content types is >not very good. It searches for the doctype line at the beginning of >the document[1] but it doesn't handle comments. > >[1]:https://github.com/dillo-browser/dillo/blob/v3.1.0-rc1/src/misc.c#L148 > >We should rely on the Content-Type provided by the server, or at least >improve the detection. So, this is a tricky case. Dillo has several content types for a single document sorted by priority, the first one set defines the content type of the document: 1. The "override type" used to override the type (highest priority) 2. The "meta type" given by the <meta ... content="..."> tag in HTML 3. The "http type" given by the HTTP Content-Type header 4. The "guessed type" based on the document data (lowest priority) They all start set to NULL. At first, the server sends "text/html; charset=UTF-8" which defines the http type: > % curl -sI http://www.lemis.com/ | grep Content > Content-Type: text/html; charset=UTF-8 The guessed type is also wrongly set to "text/plain" due to the comments in the beginning which cause a mismatch of the "<!doctype". This is the *first bug*. As the document continues loading, the <meta> tag is found: > % curl -s http://www.lemis.com/ | grep Content > <meta http-equiv="Content-Type" content="text/xhtml; charset=utf-8"/> Which sets the meta type to "text/xhtml". So far we have this situation: override_type = NULL meta_type = "text/xhtml; charset=utf-8" http_type = "text/html; charset=UTF-8" guessed_type = "text/plain" While setting the meta type, there is also an special rule as a workaround for Doxygen pages, which checks if the the content type of the meta tag begins with "text/xhtml" (which it does) and if so sets the override type to the guessed type: https://github.com/dillo-browser/dillo/blob/a0151cbc86166731465b963ea3addb0… So the types are left as follows: override_type = "text/plain" meta_type = "text/xhtml; charset=utf-8" http_type = "text/html; charset=UTF-8" guessed_type = "text/plain" This causes the type of the document to be handled as "text/plain". The "text/xhtml" type should be defined as "application/xhtml+xml", as the W3 describes: https://www.w3.org/TR/xhtml-media-types/#media-types Which Dillo handles fine. So, I'm thinking in transforming the "text/xhtml" to "application/xhtml+xml", better than relying on the guessed type. The types end up being: override_type = "application/xhtml+xml; charset=utf-8" meta_type = "text/xhtml; charset=utf-8" http_type = "text/html; charset=UTF-8" guessed_type = "text/plain" That solves the problem. AFAIK, the "text/xhtml" is not standardized. It was mentioned in the XHTML 1.0 draft on February 1999: https://www.w3.org/TR/1999/WD-html-in-xml-19990224/#h-5.1.3 On March they raised concern about it: https://www.w3.org/TR/1999/WD-html-in-xml-19990304/ > There is one issue that is still mildly contentious within the working > group, and that we are especially interested in receiving comments on: > whether we should register a new Internet media type "text/xhtml". > > Very briefly the two opinions are: yes - that is the only way to > recognise the application type without accessing the resource; no - > all XML applications are going to have this problem, and the answer > isn't to register every single application. And on May it got removed: https://www.w3.org/TR/1999/xhtml1-19990505/ And is not part of the XHTML 1.0 standard: https://www.w3.org/TR/xhtml1/ So I don't think is should be ever used. Regarding the type guessing bug, I think I can improve it by assuming that if we find the "<!doctype html" string in the first 1024 bytes or so, it is an HTML-like type, but it incurrs in more overhead. So I think for now we can rely on the correction of "text/xhtml" to "application/xhtml+xml", which seems to work fine. I don't like adding quirks, but I will keep this one as it was already there. Here is the PR: https://github.com/dillo-browser/dillo/pull/140 I'll check with some Doxygen pages and see it they don't break anything. Interestingly, to this day they continue to generate documents with the wrong "text/xhtml" content type (since at least 13 years, based on the git blame): https://github.com/doxygen/doxygen/blame/78422d3905e57acebf0374feefafa6578d… I'll open an issue on their repo too. Best regards, Rodrigo.

April 23, 2024

Re: scroll_step description

by Kevin Koster

On Sat, 20 Apr 2024 14:19:32 +0200 Rodrigo Arias <rodarima(a)gmail.com> wrote: > On Sat, Apr 20, 2024 at 01:54:26PM +1000, Kevin Koster wrote: > >There's a new "scroll_step" option in the example configuration file > >for 3.1.0-rc1. The comment "Mouse wheel scroll step in pixels" > >suggests that this only affects the mouse wheel, but it also changes > >the distance the page moves when the arrow keys are pressed and the > >overlap of previous page content that's left when you press Page > >Up/Down. I think that's a little confusing, especially as the value > >has changed since the default behaviour in 3.0.5 (the old value > >seems to have been about 25). > > Thanks, I agree. Could you provide a patch? OK, see the attached dillorc_scroll_step_description.patch (applied directly to the file, eg. "patch dillo/dillorc < dillorc_scroll_step_description.patch"). The patch also includes a fix for a misspelling of "differentiates" in a later comment there. > Please consider reviewing the manual on that topic[1], in case some > parts need to be clarified too. > [1]:https://dillo-browser.github.io/user_help.html#scrolling I'd only add that a scroll_step is subtracted from the Pg. Up/Down movement, as I've done in the other attached patch (user_help_page_step.patch).

April 21, 2024

Re: Links closed empty due to <a> containing <div>

by Kevin Koster

On Sat, 20 Apr 2024 13:43:12 +0200 Rodrigo Arias <rodarima(a)gmail.com> wrote: > Thank you for testing 3.1.0-rc1 and reporting the problems. I will > try to address them in the following days. Thank you for the quick and informative responses. Great to see someone so active with Dillo development again. > If we allow broken pages to be displayed properly, there is no > incentive for them to be fixed. My recommendation is that you contact > the website so they fix it (along with the 92 errors reported by the > HTML validator)[3]. > > [3]:https://validator.w3.org/check?uri=http%3A%2F%2Fmy-addr.com%2F That's a reasonable approach, and I've done that. I'm sure it's hard enough trying to support modern web standards without attempting to support websites that break them as well. I also notice that Dillo 0.8.5 actually has the same objections to those <div> tags as 3.1.0-rc1, so its tollerance for such things has varied over the years.

April 21, 2024

Re: Dillo not rendering HTML with comments before <!DOCTYPE>

by Rodrigo Arias

Hi, On Sat, Apr 20, 2024 at 02:00:05PM +1000, Kevin Koster wrote: >This problem was present in 3.0.5 as well as in 3.1.0-rc1. > >URL: http://www.lemis.com/ >CSS: enabled or disabled >Summary: Won't render HTML with comments before <!DOCTYPE>. >Pages on this website aren't rendered, just displayed as source code. >Although they are XHTML, this doesn't appear to be due to this bug: >https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1036382 >If I save the file as lemis.xhtml and remove the two comment lines >before the <!DOCTYPE> declaration, then it renders when I load it with >file:// or http://, otherwise it doesn't. > >My reading is that comments there are valid for HTML 4 (which is >declared in the page's <!DOCTYPE>) since the standard says: > White space (spaces, newlines, tabs, and comments) may appear before or > after each section. >https://www.w3.org/TR/html401/struct/global.html#h-7.1 Yeah, the current detection mechanism in Dillo for content types is not very good. It searches for the doctype line at the beginning of the document[1] but it doesn't handle comments. [1]:https://github.com/dillo-browser/dillo/blob/v3.1.0-rc1/src/misc.c#L148 We should rely on the Content-Type provided by the server, or at least improve the detection. Best, Rodrigo.

April 20, 2024

Re: scroll_step description

by Rodrigo Arias

Hi, On Sat, Apr 20, 2024 at 01:54:26PM +1000, Kevin Koster wrote: >There's a new "scroll_step" option in the example configuration file for >3.1.0-rc1. The comment "Mouse wheel scroll step in pixels" suggests that >this only affects the mouse wheel, but it also changes the distance the >page moves when the arrow keys are pressed and the overlap of >previous page content that's left when you press Page Up/Down. I think >that's a little confusing, especially as the value has changed since >the default behaviour in 3.0.5 (the old value seems to have been about >25). Thanks, I agree. Could you provide a patch? Please consider reviewing the manual on that topic[1], in case some parts need to be clarified too. [1]:https://dillo-browser.github.io/user_help.html#scrolling Best, Rodrigo.

April 20, 2024

Re: <td> width attributes ignored

by Rodrigo Arias

Hi, On Sat, Apr 20, 2024 at 01:49:13PM +1000, Kevin Koster wrote: >Another page rendering issue that's introduced after the 3.0.5 release, >this time affecting my own website. > >URL: http://www.ombertech.com/ >CSS: enabled or disabled >Summary: <td> width attributes ignored. >3.0.5: >Heading table at the top of the page displays correctly scaled with >links in the middle formatted in a horizontal row if they all fit in >1/3rd of the browser window's width. >3.1.0-rc1: >The "width" attributes of the <td> elements are ignored and the links in >the middle are thereby squashed into a vertical row. Further testing >with modified versions of the page shows that this table cell width >problem is only present if the width attribute in the <td> elements is a >percentage and the <table> element also has a width attribute set as a >percentage, then the <td> width attributes get ignored. If either (or >both) table or td width is in pixels, the problem isn't present, only >when table and td width are both specified as percentages. > >So far as I can tell setting width in <table> and <td> elements as a >percentage is valid for HTML 4.01 Transitional, and other browsers >(Links, Firefox) render the table correctly. Thanks. This is caused by commit f5a0b6[1], which should be already solved[2]. Reverting it fixes this specific problem, I'll add some regression tests and see if it doesn't break anything else. [1]:https://github.com/dillo-browser/dillo/commit/f5a0b6030c2001e5a0fcacf813… [2]:https://github.com/dillo-browser/dillo/commit/b0f6a3f055039c5c9c3ab65102… Best, Rodrigo.

April 20, 2024

Re: Links closed empty due to <a> containing <div>

by Rodrigo Arias

Hi Kevin, On Sat, Apr 20, 2024 at 01:45:24PM +1000, Kevin Koster wrote: >I use Dillo 3.0.5 (with SNI support added to the HTTPS plug-in) for the >majority of my web browsing, so it's great to see some movement towards >the next version. Testing 3.1.0-rc1 I still notice some HTML rendering >regressions and other bugs which I'll report in this and the following >emails. Thank you for testing 3.1.0-rc1 and reporting the problems. I will try to address them in the following days. >I don't see any of these the GitHub Issues section, but I won't post >them there because I hate the GitHub website - it doesn't work in my >favourite web browser for one thing! Other people are welcome to, though >I'll only participate in discussion here. It is fine. I also don't like the GitHub website and I'm looking to switch to something more convenient with multi-platform CI. >URL: http://my-addr.com/ >CSS: enabled or disabled >Summary: Links closed empty due to <a> containing <div>. >3.0.5: >All links on page are usable and <a> elements don't have errors listed >in the bug window. >3.1.0-rc1: >Links below the horizontal heading bar at the top of the page aren't >usable, they're just shown as plain text. The Bug window shows all of >these <a> elements are closed when a <div> element is found inside them, >leaving no text present inside the link. I see that <div> isn't allowed >inside <a> according to the standard, but other browsers (Links and >Firefox) still render this website's links in a functional way and it's >a pretty inconvenient breakage. That page declares compliance with HTML 4.01 Transitional but then proceeds to not follow it. In HTML 4.01, anchors cannot contain div elements[1] so Dillo closes the anchor[2]. In HTML 5 they are allowed, so those anchors would work properly. [1]:https://www.w3.org/TR/REC-html40/sgml/loosedtd.html#inline [2]:https://github.com/dillo-browser/dillo/blob/v3.1.0-rc1/src/html.cc#L3742 We try to stick to the standards rather than implementing a quirks mode like Firefox. If we allow broken pages to be displayed properly, there is no incentive for them to be fixed. My recommendation is that you contact the website so they fix it (along with the 92 errors reported by the HTML validator)[3]. [3]:https://validator.w3.org/check?uri=http%3A%2F%2Fmy-addr.com%2F Best, Rodrigo.

April 20, 2024