- Dillo-dev - lists.mailman3.com

[Dillo-dev]Buggy HTML, the new parser and webmasters

by Andreas Schweitzer

Hi all, First : my apologees for a rather large mail ! Now, that the bug counter is in dillo, and with Madis' last post, it's a good coincidence that I just tried to convince webmasters of 3 sites I usually visit to fix their HTML. From 2 I haven't heard a thing (and they are not even THAT huge), but from www.osnews.com I got a reply very fast - and it turned out to be an extremely interesting conversation. I asked her for permission to forward it to the list, and here it goes. I sorted it chronolically, with some comments of mine in between. I omitted my first mail - it was just the HTML warnings of the new parser along with a kind request to fix the page. WARNING, BIG REMINDER : Don't flame her - it's not about this individual site, it's about general practice and the general attitude towards standards ... After all, the e-mail conversation started in privat and she didn't have in mind that a lot of people would read it. And I think she deserves credit for taking her time and lay out her arguments and explaining everything in a very friendly manner. On Sat, Jan 24, 2004 at 06:36:27PM -0800, Eugenia Loli-Queru wrote: > Hello Andreas, > thanks for the heads up. > I looked at the error messages, and I know what is wrong. Please note that > all forms and tables ARE closing in the code, it is just that your parser is > not realizing it. > You see, I start the form inside a table's cell, but I close the form > outside the table. This is not 100% valid code, but it is like that for a > reason. This is causing your parser to spit all these errors (all the errors > below the form error are just a side effect of the first error). > So, the reason I close the form outside that table is because I am trying to > go around bugs on Netscape 4 and IE. As you can see, the cell that holds the > form has a black background color. If I close the form inside that cell, IE > and NS4 create a new paragraph inside that cell, even if my code never asked > for it. And that looks ugly. Up to here, you could argue a lot about how to design a web page, to not wanting to dictate how the page looks and leave it to the browser, to us CSS and whatnot. With enough arguing one could even convince this very author to change (not likely though, see below). BUT : the interesting part comes now : > So, the "trick" that many web developers of my time use a lot (see: > hand-written HTML), is to close the </FORM> outside of tables or places > where it screws up the design. It is a well-known trick, and so your new > parser is going to meet it a number of times in the future. I suggest you The above is frightening .... > fix your parser to understand "non-normal" code like that, and realize that > the form does close as expected. Of course, for us that's not fixing but breaking ;-) .... Here is now my reply (rather long, and you will know the arguments if you follow the lamiling list) : > Hi Eugenia, > > Thanks for the reply and explanation. > First of all : I am not a core developer of that browser - just > a steady user and occasional contributor. > > However, if this mail came from one of the developers or if I told > them to follow your advice, you would either be ignored or flamed ;-) ... > I will try to do neither. So please do keep in mind that my response > is meant as a friendly food for thought or possibly some constructive > criticism. > > Discussions about fixing broken HTML do come up on the mailing list > from time to time. And the consent is to just not do it - for a number > of reasons (I probably will forget some). > - what good are standards if nobody conforms to them, and instead an > own (not very strict !) psuedo-standard develops ? > - It's not a browser's job to guess what the author wanted to have displayed. > - It costs man-power and CPU cycles to detect and fix broken HTML. > We prefer to spend the man-power on developing something useful instead > of broekn HTML-detection code. And we prefer to give the CPU cycles > to the user (BTW, dillo is the snappiest grafic browser I know - maybe Links2) > > All right, but we already detect the problem of your page, you may say :-) .. > and indeed, the *old* parser was closing out of order. In fact, I could > configure dillo to still use the old parser and your page will look great. > BUT : then other pages break :-( - they have buggy HTML, too. And there, > closing tags with the new method is better then with the old method. > So, in chosing which parser to use, the default is the standards complying > method of interpreting HTML (also, the old parser was buggy and not quite > right). Oh - and if we care about the web sites we also tell/told the > authors if these other pages that they have buggy code ! > Now, why then are Mozilla or the other browsers displaying all correctly ? > Probably because they did spend man-power and CPU-cycles to guess what the > author wanted. > I had to smile when you wrote, "Fix your parser to understand non-normal HTML", > because I thought, "That's not fixing it, that's breaking it". > > When I design web pages, I usually make them pass (if possible) > validator.w3.org and if they look ugly in ancient - and buggy ! - browsers, > bad luck ! I realize, my pages are nothing professional like your site, > but still, is it worth to support buggy broswers ? How many visitors > are actually using Netscape 4 ? Probably more than dillo, but the comparison > should be non-buggy vs. buggy browsers. About IE - are all versions > of IE bahving like this ? Then my argument would be, why support MS's > interpretation of standards ? (and outside of HTML you probably will > agrre ;-) ... after all, don't we all know lots of alternatives and fight > for them ?) You as a professional web site probably do have to support > buggy broswers, but the question still may be asked. > > Finally, some links to archived previous discussions : > http://news.gmane.org/group/gmane.comp.web.dillo.devel/thread=1324/force_lo… > (if you don't want to read all of it, read at least Eric Gaudet's posts) > And here is the explanation of the new parser : > http://article.gmane.org/gmane.comp.web.dillo.devel/1434 > > By now, you know that the other problematic site was slashdot ;-) ... > (among others !) which displays ok with the current parser - but also is > buggy HTML - and they know it ! > > And before I forget : Yes, I saw a lot of pages that look quite ugly > with the strict interpretation. And I also only tell the webmasters > of pages for which I care :-) > > And as a really last word : the new parser may indeed have room for > improvement, and some more closing may indeed be possible as a compromise, > BUT, as I said, I'm not a core developer. > > As I said in the beginning, this was meant as food for thought, not flaming. > If you should decide to change HTML after all, that would be fantastic. If > not, hopefully I at least did explain myself well. > > Cheers > Andreas And here is her 2nd answer: > Thank you for the email Andreas, > however, I must say that the code in that form won't change. IE has more > than 70,000 viewers daily on osnews, and Dillo has about 20. So, as you can > understand, being compatible with the broken IE is more important to us, > even if that means that we have to not be validated. In fact, not being > validated, it is a given for big sites, because in order to give a > specific/certain look, you have to do tricks like the one I described to you > in order to render the same on all big browsers. Now you could, of course, go and say "screw this site" .... but the following is at least ineresting food for thought. I don't fully agree, but I can understand the point of view. > Also, please put in mind: my husband is in the browser business. He is a web > browser developer for OpenWave.com among his team. They don't have the > specific problem of out-of-order-tag-closing because they know that many > sites are like that, so they have to go around this problem, OR they won't > be able to sell their browser to anyone. > > This is a market. Even for the free Dillo. Dillo's job is not create the > most "correct" parser in the world, but to render most sites in the world. > If the first was true, then Dillo is nothing but an "Amaya", a test, a proof > of concept. But if the second is true, then Dillo is really a usable by > people browser. > > So, depends what its developers want. If they want a proof of concept, no > problem with me, osnews and most other sites won't work. If they want a > browser that works for most people, they need to sit down and go around > buggy HTML (created by either amateur developers or from developers that > they had to go around bugs). For example, IE even goes around this bug <a/> > and understands that the developer wanted to write </a> instead. Big > browsers do such error checking and go around buggy HTML, because it is the > only way they can be widely used. In my case of course, it was me that I had > to go around their bugs. > > So, it all depends what the Dillo people want. It is not what I, or you, > want, but what their real targets are: proof of concept project or a real > USABLE browser. > > If that was me, it would have being the second. And it ain't my fault that > IE has these bugs that I have to go around, it is my job to make sure my > page renders well for most of my readers. :) > > Rgds, > Eugenia > > --- > Editor in Chief > http://www.OSNews.com When I asked her for permission to forward her mails to the list she added another intersting quote : On Sun, Jan 25, 2004 at 08:53:17PM -0800, Eugenia Loli-Queru wrote: > You can go ahead and quote some of the stuff I said. > BTW, I discussed the matter with my husband (who as I said is in the browser > business) and he told me: > > "Creating an HTML browser and a Web browser are two different things. The > first one just needs to do the job right and parse HTML correctly, the other > one just has to be able to render web pages in any possible mean. These are > two different beasts and targets --unfortunately." > > I am afraid this is the reality of any product though, not just browsers... > Reality market... :( > > Eugenia > Bottom line : it may be possible to convince individual sites to fix their HTML, but what about really big sites that use these ... err ... "workarounds" ? Do they really care how a little browser interprets their HTML ? They care if HTML looks good in IE - reality sucks. It's extremely frustrating. And even more so, when I have to look at garbeled pages ;-) .... The situation may improve over time when more and more sites use CSS and don't have to rely on such dirty formating tricks - but then again, how many bugs has IE there, that authors want/will work around ? And to repeat my reminder : Don't flame her - it's not about this individual site, it's about general practice and the general attitude towards standards ... And again I think she deserves credit for taking her time and lay out her arguments and explaining everything in a very friendly manner. Cheers, Andreas -- **************************** NEW ADDRESS ****************************** Hamburger Sternwarte Universitaet Hamburg Gojenbergsweg 112 Tel. ++49 40 42891 4016 D-21029 Hamburg, Germany Fax. ++49 40 42891 4198

Jan. 27, 2004

RE: [Dillo-dev] worried about dillo (cathedral and bazaar)

by Eric GAUDET

-- En reponse de "[Dillo-dev] worried about dillo (cathedral and bazaar)" de linux seaq, le 26-Jan-2004 : ... > I recently re-read the cathedral and bazar paper, and thought to my > self that dillo was heading in another direction. It seems to me that > there's no intention to receive help from others, or get into account > other's contributions. I believed that the main force behind a project > weren't its main developers but its users and contributors. That mix > made projects grow and grow. And dillo rejected that "magical" force. > > I'm just writing these thoughts, because as i said, i'm worried about > dillo. ... I wanted to react to that now the dust has settled. Dillo is back to normal. Even an (arguably) small project like Dillo takes a lot of time and energy to maintain, to develop and to get right. A few months ago a few individual started submitting patches against the advices of the core developers. The features were sometimes interesting, sometimes not. The code style did not integrate well the rest of Dillo's code. (not to mention the form in which the patches were presented). The maintainers of Dillo decided these contributions were not good for Dillo, for many reasons. While you might disagree, similar decisions were made during the development of Dillo, and the result is the great browser we have now. We have the focus and we have a plan. Of course Dillo is not finished, and your contributions are still welcome. Countless contributions have been made since dillo was started, and the mailing list has always been courteous and people helpful. Dillo is that good now not only thanks to the core developers, but also to many contributors helping with the spirit of being part of a group. Not to take control to see the one feature they like be included. I understand how a Linus Torvald saying no to a patch for Linux might induce more respect to young coders than for a smaller project like Dillo. But you should learn to repect Jorge, Sebatian, Livio, and the others contributors (and not only the so-called "core developer"). These people put a lot of their energy and sweat into Dillo over the years, because that's what it takes to make a successful project. Dillo is an Open Source project, and anybody whom's not happy with the development policy could start his own project with it. And he'll be welcome. But nobody did so far, and I take it as a confirmation that one needs more than being vocal in mailing list to maintain a project like Dillo. It takes dedication and talent. "The main job of a maintainer is to say No", said Linus Torvald. That's what happened. Nothing more: people were not happy to see their patch rejected and they tried to push back. Dillo's still active, and not less than it was before: do not confuse bubbling in the mailing list with activity. Out-of-control patch submissions is not equivalent to improvement. The people who made Dillo are still there, and new contributors are joining. Those whom wanted their features included so badly prefered to leave rather than change their attitude and work the issues. I guess we're back to normal: a sane and nice working environment. Please keep contributing. Best, Eric PS: even if you're not a coder you can contribute, by doing bug reports for instance.

Jan. 27, 2004

Re: [Dillo-dev] Re: https dpi

by Madis Janson

On Mon, 3 Nov 2003, Jorge Arellano Cid wrote: > On Sat, 25 Oct 2003, Madis Janson wrote: > > > Also, there seems to be one process for one connection, which makes hard > > to implement SSL session cache. > > Yes, that happens with dpi filters. The idea in this case is to > have an http dpi that's able to handle several connections at the > same time (like the downloads dpi). That is: a dpi server instead > of a dpi filter. With that scheme session cache is easy to > implement. > > (and BTW much easier for the developer because he can arrange > things much more freely than when patching inside dillo). > > > content-type > > (as above). > > The https dpi would need to listen the HTTP stream and so has > access to the content-type sent by the remote https server. > (http://article.gmane.org/gmane.comp.web.dillo.devel/1407) if someone would create non-ssl dpi proxy plugin (that just makes http requests), i will add the ssl part to it (i have working ssl proxy code, which currently works using a patched dillo).

Jan. 27, 2004

Re: [Dillo-dev] small politically incorrect html parser patch

by Madis Janson

On Tue, 27 Jan 2004, Madis Janson wrote: > > when you want to use dillo for browsing instead of html test tool, then > this little patch is useful for many stupid webs, which randomly close > or not close font tags whenever happens and whereever happens... > sorry for spam, but in the mean time i (re)found better one... diff -pru -x CVS -x configure -x 'aclocal*' -x 'Makefile*' -x config.h.in dillo-tab/src/html.c dillo/src/html.c --- dillo-tab/src/html.c 2004-01-25 04:37:21.000000000 +0200 +++ dillo/src/html.c 2004-01-27 04:27:18.000000000 +0200 @@ -1176,7 +1183,8 @@ static void Html_tag_cond_cleanup(DilloH while (stack_idx && strcmp(Html_tags_get_name(html->CurrTagIdx), html->stack[stack_idx].tag) && - (old_mode || + ((old_mode && + !Html_match_tag(html->stack[stack_idx].tag, "table>", 6)) || Html_tags_get_endtag(html->stack[stack_idx].tag_idx) == 'O')){ --stack_idx; e_pop = 1; and to the dillorc file: use_old_parser=YES only note: when you use it, don't whine to list that it renders something incorrectly - look the output with use_old_parser=NO and you'll probably discover why...

Jan. 27, 2004

small politically incorrect html parser patch

by Madis Janson

when you want to use dillo for browsing instead of html test tool, then this little patch is useful for many stupid webs, which randomly close or not close font tags whenever happens and whereever happens... diff -pru -x CVS -x configure -x 'aclocal*' -x 'Makefile*' -x config.h.in dillo-tab/src/html.c dillo/src/html.c --- dillo-tab/src/html.c 2004-01-25 04:37:21.000000000 +0200 +++ dillo/src/html.c 2004-01-27 04:27:18.000000000 +0200 @@ -4280,7 +4372,7 @@ static const TagInfo Tags[] = { {"dt", 'O', Html_tag_open_dt, Html_tag_close_par}, {"em", 'R', Html_tag_open_em, Html_tag_close_default}, /* fieldset */ - {"font", 'R', Html_tag_open_font, Html_tag_close_default}, + {"font", 'O', Html_tag_open_font, Html_tag_close_default}, {"form", 'R', Html_tag_open_form, Html_tag_close_form}, {"frame", 'F', Html_tag_open_frame, Html_tag_close_nop}, {"frameset", 'R', Html_tag_open_frameset, Html_tag_close_default}, It makes a bit less likely, that the font tags screw up every other one (like tables and forms)...

Jan. 27, 2004

Dillo cookies

by pete

I am getting this error with Dillo 0.8.0: [pete@happy:.dillo, 07:25 PM, Mon Jan 26]> dillo dillo_dns_init: Here we go! The cookies file has a file lock: disabling cookies in this dillo! Nav_open_url: Url=>about:splash< How do I go about 'unlocking' this file? It was working initially with 0.7.3 Thanks, -- Pete http://milneweb.com http://nomorevirus.com

Jan. 27, 2004

Re: [Dillo-dev] An odd request, I know...

by Tom Barnes-Lawrence

Incidentally, I'm familiar with getting 2 copies of various emails when people reply to both me and the list... but *3*? That's just confusing. On Mon, Jan 26, 2004 at 03:08:32PM +0100, Thorben Thuermer wrote: > On Sat, 24 Jan 2004 00:04:06 +0000 > Tom Barnes-Lawrence <tomble(a)usermail.com> wrote: > > -I understand you not wanting the referer info being sent, even if > > it does make the logs from my hit-counter a bit less useful, *but*, > > some sites do seem to use the referer tags to work out if you're > > accessing their images, etc deep-linked from some other page. > > I'm afraid I can sort of sympathise with this, as the idea of > > "stealing bandwidth" does have quite a lot of truth in it (it > > does cost them money). > > there is one important detail there that i think is very important: > even if you want to avoid image linking by checking referers, you > should allow EMPTY referers... Are we considering empty referers to be the same as no referers? > that way: > a) linking the images will still fail with the majority of browsers > b) browsers that don't send referers will still work. Probably. I have a strange image of *all* browsers ending up sending no referers in order to be able to see remotely hosted images, but I rather doubt that would happen, so you're probably right here. > c) users can for example post a link to a image on irc or > whereever and the link will work (this is something that makes > users like me really hate your site) You hate my site? Wah! But seriously (*unless* you're meaning "your" as in "anybody's"), I know Angelfire *used* to block images on no referers, but I noticed recently that Dillo *was* showing the few images I had on my pages, and after reading your comment today, I copy-pasted the URL of my site's logo into Dillo. Showed up fine. I don't know if you just assumed that Angelfire was still doing that but you hadn't notice since you'd patched your copy of Dillo, or if you'd just thought that my site wasn't downloading all the graphics (there are only about 3 on the whole site, 5 if you count thumbnails as different to their full-size images). > d) users can use download programs to download files off the site without > having to send fake referers (wget --referer, anybody?) Yeah, I'm familiar with *that*... > (and to control those, there's robots.txt) I know what robots.txt is for, how does it apply here? Are you talking about a download program that gets the *whole* site? > imho not allowing the referer to be empty is just stupidity > on the side of the people setting up the filtering, and it just Perhaps it just didn't occur to them? They want to keep their bandwidth costs down, so they'd be worrying more about that issue than supporting an obscure little niche browser like ours (even if it is lovely). > makes their own problems worse, by forcing people to send > fake referer headers (like my dillo does for QUITE some time now). > > (the patch is trivial, just add pairs of > + "Referer: http://%s%s/\r\n" > + URL_HOST(url), s_port->str, > into src/IO/http.c (and don't get the order wrong)) Hey, that's neat, thanks! I was considering doing something like this myself (but wouldn't know where to start). > > So, I'd love it if the bookmarks code stuck each section in its own > > page, and let me nest one section in another. > isn't this is kind of the point of seperating stuff into dpis?: > if you don't like the default one, you can relatively easily > change it without having to deal with the browser code or even > recompile it. Uh, yes, I know. Apart from the fact I got the impression the DPI interface was unstable (as in changing, not crashing) at the moment, I pointed out that I was quite busy right now, and wasn't sure of some of the details, either. I find working with other peoples' code fairly daunting, personally. Perhaps that's lack of experience. Tom Barnes-Lawrence

Jan. 26, 2004

my.yahoo.com

by pete

Hi all, I am now running the CVS version of Dillo. I am seeming to have a lot of lockups with it. Now I know this is common with CVS but is this something that should be mentioned here? Just wondering what is the proper thing to do. Another, I am still trying to access my yahoo with it but still not possible. Am I missing something that I need to do or is this just typical for Yahoo? -- Pete http://milneweb.com http://nomorevirus.com

Jan. 26, 2004

Thanks!

by Arjen Meek

Just a quick "thank you" to all Dillo-devs here. It has replaced Firebird as my default browser less than 5 minutes after installing it. Some experiences: - Dillo combines perfectly with the Ion window manager. A browser shouldn't have to perform window manager tasks, and tabbed browsing becomes a rather silly concept when a sensible window manager combines with a sensible browser. - 0.7.3 compiles & runs cleanly on my 60 Mhz HP PA-RISC system (712/60 running Debian Woody-hppa), and actually renders pages faster on that system than Firebird does on my 600 Mhz Athlon. - A suggestion: why not have CTRL+L switch focus to the URL bar when it is visible, instead of popping up a dialog box? Keep up the good work! Arjen

Jan. 26, 2004

Dillo IRC

by pete

Good morning, Is there an IRC channel for Dillo at this time? And of course the next question where is it or why not? Thanks, -- Pete http://milneweb.com http://nomorevirus.com

Jan. 26, 2004