- Dillo-dev - lists.mailman3.com

Re: [Dillo-dev] dillo-0.8.6-rc3.tar.bz2

by Jorge Arellano Cid

Hi, Thanks for the excellent information! Don't worry guys, this patch obviously needs major rethinking. Please read these articles (suggested in this thread): http://www.mozilla.org/docs/web-developer/mimetypes.html http://diveintomark.org/archives/2004/08/13/safari-content-sniffing http://ppewww.ph.gla.ac.uk/%7Eflavell/www/content-type.html It's somewhat funny. The third one has thoughtful comments and suggestions; it attacks IE for its against-specs content-type sniffing. Note that the first article shows that mozilla is doing this too (I guess not for the same reasons though :-). The second one shows a shameful case of browser-sniffing bugs. This is a complex problem, but has some solutions. BTW, I agree a lot with what Kelson wrote (attached below). On Thu, Mar 30, 2006 at 02:16:36PM -0800, Kelson Vibber wrote: > Yeah, I'm seeing problems as well. > > Putting the character set into the Content-Type header in the form > 'text/html; charset=utf-8' is actually recommended by the W3C: > http://www.w3.org/International/O-HTTP-charset > > Some of the problems with the current implementation might be resolved > by normalizing the content-type. > > In general, I'm not a big fan of content sniffing unless it's done in > very limited circumstances. Often the server has a reason to use that > content-type. > > The way Mozilla handled this was to only do content sniffing if the > server used text/plain. It's more likely that you'll find a server > that's sending the default type* than one that's deliberately serving > something incorrect.** > http://www.mozilla.org/docs/web-developer/mimetypes.html > > Even then, you have to be careful, as illustrated here: > http://diveintomark.org/archives/2004/08/13/safari-content-sniffing > It's a text document, properly served as text/plain, that mentions XHTML > in the first line. The then-current version of Safari decided to render > it as XHTML. > > * Apache uses a default of text/plain for all files it can't identify. > I think this is a bad idea, but it's probably still used for historical > reasons. IIS uses application/octet-stream, which doesn't tell you > anything, but at least it usually triggers a download instead of trying > to display binary data as text. This is one of those rare cases where I > think IIS got it right and Apache got it wrong. > > ** This is also why so many RPM packages are served as RealPlayer files. > RealPlayer was the first popular use for the .rpm extension, so lots of > servers got configured that way. > > -- > Kelson Vibber > www.hyperborea.org and also want to quote this part from te third article: <q> The HTTP protocol specifications (1.0 and 1.1) effectively forbid a browser that has received a valid Content-type header from the server, from making its own unilateral determination of the content-type - see RFC2616 section 7.2.1 (my emphasis): If and only if the media type is not given by a Content-Type field, the recipient MAY attempt to guess the media type via inspection of its content and/or the name extension(s) of the URI used to identify the resource. The consequences of this when a server is misconfigured aren't always immediately evident; for example, consider an HTML page (sent out correctly as text/html) which calls out a stylesheet and a number of in-lined images: if the server sends these out with a wrong Content-type, then the browser might be displaying the HTML page's main content, but the browser has every right to ignore the offending stylesheet, or to omit the offending image(s) from the display: indeed a strict interpretation of the rules would say that it must behave that way. Faking the wrong Content-type from the server is potentially a way of compromising security, so there's a genuine reason for this rule being the way that it is. </q> Solutions I see so far: 1.- Keep it as is 2.- Do Content-type sniffing and follow the SPEC. i.e. as stated above, to take the right of ignoring the offending contents. 3.- Do Content-type sniffing and take actions (like mozilla). I like best the second one. Basically this is, If I get a binary stream as "text/plain" or "text/html", or an image that's not an image, then issue a warning and ignore it (abort). Note: this is a basic security procedure. This has the advantege of protecting the browser against attacks and following the SPEC. The user is left to decide (for instance to retry with "save link as"). The third option looks more "user friendly" but it goes against the SPEC. Note: One huge problem of option 1, is that if you start a download this way, it ends in _main memory_. If you get an ISO or movie, eventually the browser will trigger swapping or abort (out-of-memory) and lose the file and time spent on the download. A No-no. -- Cheers Jorge.-

April 2, 2006

Re: [Dillo-dev] dillo-0.8.6-rc3.tar.bz2

by Jorge Arellano Cid

Hi, On Sat, Apr 01, 2006 at 02:13:05PM +0100, robert w hall wrote: > In article <FCZNR2Ai+lLEFwiV(a)n-cantrell.demon.co.uk>, robert w hall > <bobh(a)n-cantrell.demon.co.uk> writes > >But, the latest fltk does not compile on the ps2 (...'input_callback...' > >is private) - looks like the age of the ps2 base-distribution may be > >catching us out? > > > > > but fltk-1.1.7 _does_ compile OK - is it possible to fix the download > dpi to run with the earlier version this time? Unfortunately not. -- Cheers Jorge.-

April 2, 2006

Re: [Dillo-dev] dillo-0.8.6-rc3.tar.bz2

by Jorge Arellano Cid

Hi, On Sun, Apr 02, 2006 at 12:19:18PM +0200, Thomas-Martin Seck wrote: > [...] > fltk2 needs gcc 3 to compile "as is" - does linux/ps2 only come with gcc > 2.95? > > This bites me on FreeBSD 4, too, where the system compiler is gcc > 2.95.4. > > (Not counting the fact that snapshots past 4825 do not compile with gcc > 3.4.x either, sigh.) Yes, the latest online versions have had some compilation problems. Revision 4886 works great here though. > > Could you try this patch? It is stolen from fltk's bug database, > STR#1124. It enables me to compile fltk2 r4825 with gcc 2.95.4 on > FreeBSD. > > --- fltk/ComboBox.h.orig Sun Apr 2 12:05:31 2006 > +++ fltk/ComboBox.h Sun Apr 2 12:07:16 2006 > @@ -103,7 +103,7 @@ > ComboBox *combo_; > }; > ComboInput *input_; > -#ifdef _WIN32 > +#if defined ( _WIN32 ) || defined ( __GNUC__ ) && __GNUC__ < 3 > public: > #endif > static void input_callback_(Widget*,void*); > > _______________________________________________ > Dillo-dev mailing list > Dillo-dev(a)dillo.org > http://lists.auriga.wearlab.de/cgi-bin/mailman/listinfo/dillo-dev > -- Cheers Jorge.-

April 2, 2006

Re: [Dillo-dev] dillo-0.8.6-rc3.tar.bz2

by Thomas-Martin Seck

* Jorge Arellano Cid (jcid(a)dillo.org) > Hi there! > > The new dillo-0.8.6-rc3 tarball is ready for download. Get it > from: > > http://www.dillo.org/download/dillo-0.8.6-rc3.tar.bz2 Compiles and works on FreeBSD 4, 5, and 6 with and without downloads.dpi. > Finally I decided to include a big patch for the MIME type > detection and its handling. This is an important change in the > way Dillo treats its incoming files (HTTP streams). > > The point: as this release comes with a new downloads GUI, and > downloads are mainly the focus, I disliked the situation where a > Web server "lied" with regard to the Content/Type of a file. I am not too happy with this. As already pointed out on this list, mp3's will be treated as test because they start with "ID3". There may be other interesting cases: When I first read your post, I immediately thought of Windows' Explorer and the WMF vulnerability - no matter how you named a WMF file, it would always be rendered as if it were named .wmf, completely breaking Windows POLA where the extension of a file is authoritative. You might open an interesting backdoor with content sniffing applications. Is is possible to make this a compile time/runtime option somehow, defaulting to disabled?

April 2, 2006

Re: [Dillo-dev] dillo-0.8.6-rc3.tar.bz2

by Thomas-Martin Seck

* robert w hall (bobh(a)n-cantrell.demon.co.uk) > In article <$N7zyiApBaLEFw0m(a)n-cantrell.demon.co.uk>, robert w hall > <bobh(a)n-cantrell.demon.co.uk> writes > >I _hope_ to test on the ps2 over the weekend, when I can get the creaky > >box back up proerly > >Bob > > > > > >In article <20060330155817.GC29028(a)dillo.org>, Jorge Arellano Cid > ><jcid(a)dillo.org> writes > >>Hi there! > >> > >> The new dillo-0.8.6-rc3 tarball is ready for download. Get it > >>from: > >> > >> http://www.dillo.org/download/dillo-0.8.6-rc3.tar.bz2 > OK, I can compile this with --disable-dlgui (for this relief much > thanks...) > > But, the latest fltk does not compile on the ps2 (...'input_callback...' > is private) - looks like the age of the ps2 base-distribution may be > catching us out? fltk2 needs gcc 3 to compile "as is" - does linux/ps2 only come with gcc 2.95? This bites me on FreeBSD 4, too, where the system compiler is gcc 2.95.4. (Not counting the fact that snapshots past 4825 do not compile with gcc 3.4.x either, sigh.) Could you try this patch? It is stolen from fltk's bug database, STR#1124. It enables me to compile fltk2 r4825 with gcc 2.95.4 on FreeBSD. --- fltk/ComboBox.h.orig Sun Apr 2 12:05:31 2006 +++ fltk/ComboBox.h Sun Apr 2 12:07:16 2006 @@ -103,7 +103,7 @@ ComboBox *combo_; }; ComboInput *input_; -#ifdef _WIN32 +#if defined ( _WIN32 ) || defined ( __GNUC__ ) && __GNUC__ < 3 public: #endif static void input_callback_(Widget*,void*);

April 2, 2006

Re: [Dillo-dev] dillo-0.8.6-rc3.tar.bz2

by robert w hall

In article <FCZNR2Ai+lLEFwiV(a)n-cantrell.demon.co.uk>, robert w hall <bobh(a)n-cantrell.demon.co.uk> writes >But, the latest fltk does not compile on the ps2 (...'input_callback...' >is private) - looks like the age of the ps2 base-distribution may be >catching us out? > > but fltk-1.1.7 _does_ compile OK - is it possible to fix the download dpi to run with the earlier version this time? -- robert w hall

April 1, 2006

OSDL Fellowship Fund

by Michal Nowak

Hi Jorge, I don't know if you've heard about 'OSDL Fellowship Fund', which can be found here (http://www.osdl.org/lab_activities/fellowship_fund/) I hope it can help funding your work on Dillo. Check it out. Regards, Michal -- # Michal Nowak # jabber: nowak(a)jabber.cz # irc: #wmii @ irc.oftc.net ---> newman # icq: 170814295

April 1, 2006

Re: [Dillo-dev] dillo-0.8.6-rc3.tar.bz2

by Jeremy Henty

On Thu, Mar 30, 2006 at 10:30:32PM +0100, Jeremy Henty (ie. me) wrote: > > It breaks http://news.independent.co.uk/ , -rc3 renders it as plain > > text, -rc2 is fine. > > It breaks http://www.slate.com/ too. Aargh! *More* breakage! I've discovered MP3s often start wtih "ID3<lots of whitespace>" so -rc3 guesses they are text/plain . Don't know how to fix this. http://www.nostalgia.com/nf_moreinfo.html?sku=10576 starts with "<!HAS_WEBDNA_TAGS>", again -rc3 guesses text/plain . I can work around this by skipping all leading "<!...>" tags except the "<!DOCTYPE...>". http://www.techworld.com/applications/news/index.cfm?NewsID=5685&inkc=0 starts with a *huge* amount of whitespace and triggers a bug in the content type guesser: it gets a buffer full of whitespace, skips all the way to the end and guesses based on the following garbage in memory. This turns out to be 8 or so binary characters followed by ascii text from a previous buffer. Sometimes the text is part of a previous page, sometimes it contains the message "waiting for the server". -rc3 guesses text/plain . I fixed this bug by adding "if (i==Size) return st;" before doing any guessing - clearly Dillo should not try to guess if it has nothing guess with. I vote against content type guessing unless it can be improved a lot. It just doesn't work well enough. (BTW, I've frequently run Dillo -rcs before and this is the first one that gave me any trouble at all.) Jeremy Henty

April 1, 2006

Re: [Dillo-dev] dillo-0.8.6-rc3.tar.bz2

by robert w hall

In article <$N7zyiApBaLEFw0m(a)n-cantrell.demon.co.uk>, robert w hall <bobh(a)n-cantrell.demon.co.uk> writes >I _hope_ to test on the ps2 over the weekend, when I can get the creaky >box back up proerly >Bob > > >In article <20060330155817.GC29028(a)dillo.org>, Jorge Arellano Cid ><jcid(a)dillo.org> writes >>Hi there! >> >> The new dillo-0.8.6-rc3 tarball is ready for download. Get it >>from: >> >> http://www.dillo.org/download/dillo-0.8.6-rc3.tar.bz2 OK, I can compile this with --disable-dlgui (for this relief much thanks...) But, the latest fltk does not compile on the ps2 (...'input_callback...' is private) - looks like the age of the ps2 base-distribution may be catching us out? -- robert w hall

April 1, 2006

Re: [Dillo-dev] dillo-0.8.6-rc3.tar.bz2

by robert w hall

I _hope_ to test on the ps2 over the weekend, when I can get the creaky box back up proerly Bob In article <20060330155817.GC29028(a)dillo.org>, Jorge Arellano Cid <jcid(a)dillo.org> writes >Hi there! > > The new dillo-0.8.6-rc3 tarball is ready for download. Get it >from: > > http://www.dillo.org/download/dillo-0.8.6-rc3.tar.bz2 > > Finally I decided to include a big patch for the MIME type >detection and its handling. This is an important change in the >way Dillo treats its incoming files (HTTP streams). > > The point: as this release comes with a new downloads GUI, and >downloads are mainly the focus, I disliked the situation where a >Web server "lied" with regard to the Content/Type of a file. > > For instance, www.linuxpackages.net has many packages and a >system of distributed mirrors. Some of these mirrors serve .tgz >files with "text/plain" HTTP-Content/Type labels, with the end >result of Dillo trying to render the .tgz as text instead of >offering a file-save dialog. > > This patch fixes this. > > Now Dillo uses its detected Content/Type instead of the one >sent by the remote server (there's also a bug fix in the >detection code). > > e.g. > > http://www.software-mirror.com/linuxpackages/Slackware-10.2/ > X11/kpacman/kpacman-0.3.3-i486-1ron.tgz > > (all in one line) > > Try it with dillo <= rc2. It will try to render. > > FWIW, the remote server answer begins with: > > HTTP request sent, awaiting response... > HTTP/1.0 200 OK > Date: Wed, 29 Mar 2006 15:53:23 GMT > Server: Apache > Last-Modified: Wed, 12 Oct 2005 11:31:54 GMT > ETag: "262819-28292-f8a5de80" > Accept-Ranges: bytes > Content-Length: 164498 >>>> Content-Type: text/plain > Age: 95 > X-Cache: HIT from sa8.woowy.com > Connection: keep-alive > Length: 164,498 (161K) [text/plain] > 200 OK > > I'm a bit surprised nobody complained about this bug before, >but I guess with the download support being as crude as it was, >it's no wonder only a few people used it. ;-) > > Bottom line: > > PLease test this extensively! > > -- robert w hall

March 31, 2006