On Tue, 24 Feb 2004, Gavin Henry wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
- ---------- Original Message ----------- From: Jorge Arellano Cid <jcid@dillo.org> To: Gavin Henry <ghenry@fedoranews.org> Cc: Dillo mailing list <dillo-dev@auriga.wearlab.de> Sent: Tue, 24 Feb 2004 13:22:38 -0300 (CLST) Subject: Re: Upcoming review.
[list: this was originally a personal answer to Gavin, but as it has general interest information I'm CC to dillo-dev. It's about a browser comparison article. ]
[...] There're some interesting tests to be done. Which one you choose depends on the time you have, and on what you want to test for.
For instance, IMHO, the common "web experience" has four important factors:
So far I have tested the browsers on slashdot, and bbc.co.uk and a few others.
They all seem to wait until some pictures arrive until rendering anything. I haven't time these yet. Dillo, displays text immediately, but has some difficulty with CSS. For instance fedoranews.org looks bad :-)
Dillo doesn't have support for CSS yet. The closest we've got is a prototype. More info at: http://article.gmane.org/gmane.comp.web.dillo.devel/1827
Are you familiar with http://wwww.udcast.com/ whcih runs freebsd and has customised apache etc.
I thought maybe you have done something like this. As my day job www.invsat.com we had a demo from them and they explained the priciple of http precaching.
Instead of requesting 4 images then waiting, it requests all images at once, as this then reduces the latency across a satellite link.
Is this how your browser works? (*I haven't look at any code as of yet ;-) )
Yes, dillo does that and more. There's a paper explaining dillo's internals. You can find it at the bottom of the links section. It's large though.
* Network wait time
* Rendering wait time
* Time to start seeing content - streamed rendering
* Site usability
These will be what I base the benchmarks on.
Good!
These are key factors; for instance the same site may be unusable depending on whether you're on a dialup or broad band. The same is true for rendering time (influenced by CPU power and available memory); you may end going to some site only when you're sitting at a powerful workstation.
I don't have access to dial-up, but I have a celeron 400 and a athlon xp 2000 to test on.
The dialup test is really a must. Got to see to believe. There are slow-connection emulators. Sometime ago someone suggested one in this mailing list. Does anyone remember?
[...]
* Rendering wait time
Very simple, load a semi complex page (like slashdot with comments --a 500KB one), or a large one (as the mysql manual -- I have one which is more than two Megabytes long).
Press back, the forward. (you can have some coffee :)
Slashdot works perfect. What do you define as a compliant site?
A site that follows the W3C standards.
All done in CSS? no java script or flash? xhtml or html? Oops, meant for usability section.
You can find these answers here: http://www.dillo.org/funding/technical.html#fea In brief: CSS, java script, flash: NO xhtml, html: YES (note: xhtml works when served as "text/html" because the standard requires a validation we don't have for xhtml).
[...]
Simple usability experience can be explained to less technical people:
For instance, take dillo, and dig for something using google. You'll find that the time you use to make different queries and to check for alternative links until you find what you look for is better than with almost anything you've used before.
Noticed that.
Great!
Thanks Jorge, I really appreciate your time and expertise.
I think these will be used as the basis for browser benchmarks, stealling the reasoning you gave for the introductions :-)
Great! "The eyes can't see what the mind doesn't comprehend". The average user (and some Ph.Ds in this country ;) don't know about those concepts. Explaining them simply and briefly will certainly contribute to understanding the test data. Note that getting accurate hard numbers for each one of the listed points is close to impossible, it depends a lot on how the browser was built, and instrumentating the source code for each one is out of reach. For instance for testing rendering time (in any browser), you have to separate it from networking time. Having the page already in memory helps, but you don't know how much of the rendering is cached. Dillo can cache decompressed images (use_dicache=YES). This will make back and forward extremely fast with pages that contain images because dillo will have them already buffered as RGB. With firebird, you can quickly switch from one tab to another (the rendering is clearly cached), but doing the same with back and forward yields different results! Now, with browsers that cache to disk, HD access time can count... The tests I suggested take this into account and can provide "meningful" numbers, but not strictly accurate ones.
I will give you a preview of it as it starts to resemble something readable.
Thxs. Cheers Jorge.-