-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - ---------- Original Message ----------- From: Jorge Arellano Cid <jcid@dillo.org> To: Gavin Henry <ghenry@fedoranews.org> Cc: Dillo mailing list <dillo-dev@auriga.wearlab.de> Sent: Tue, 24 Feb 2004 13:22:38 -0300 (CLST) Subject: Re: Upcoming review.
[list: this was originally a personal answer to Gavin, but as it has general interest information I'm CC to dillo-dev. It's about a browser comparison article. ]
Gavin,
[...] Thanks, I saw a few of these. I will check out the wording etc. Mine might not be so in depth, more of a general features thing. But there will be figures ie load times etc. which I am sure yours will win.
:)
Know of any monster sites, that would be a good benchmark for the browsers. I have ADSL, so banwidth shouldn't be an issue.
There're some interesting tests to be done. Which one you choose depends on the time you have, and on what you want to test for.
For instance, IMHO, the common "web experience" has four important factors:
So far I have tested the browsers on slashdot, and bbc.co.uk and a few others. They all seem to wait until some pictures arrive until rendering anything. I haven't time these yet. Dillo, displays text immediately, but has some difficulty with CSS. For instance fedoranews.org looks bad :-) Are you familiar with http://wwww.udcast.com/ whcih runs freebsd and has customised apache etc. I thought maybe you have done something like this. As my day job www.invsat.com we had a demo from them and they explained the priciple of http precaching. Instead of requesting 4 images then waiting, it requests all images at once, as this then reduces the latency across a satellite link. Is this how your browser works? (*I haven't look at any code as of yet ;-) )
* Network wait time
* Rendering wait time
* Time to start seeing content - streamed rendering
* Site usability
These will be what I base the benchmarks on.
These are key factors; for instance the same site may be unusable depending on whether you're on a dialup or broad band. The same is true for rendering time (influenced by CPU power and available memory); you may end going to some site only when you're sitting at a powerful workstation.
I don't have access to dial-up, but I have a celeron 400 and a athlon xp 2000 to test on.
The time to start seeing content is also important. If it takes too long, it may exceed the user's patience (there're studies about this). For instance, if a browser needs the whole table to start rendering, some table-based sites will be exceedingly annoying to visit (unless your band-width can mitigate this).
Dillo is very fast on this.
The site usability is another key factor. For instance if it's designed for a certain browser, it may be impossible to use with another.
Alot seem broken. I will properly read the review site you posted so I can see what result they found.
Now, getting back to the test suggestions you asked, these points can be isolated.
* Network wait time:
Get to a site with multiple images, ideally with some coming from other sites. I'm sure you will find a better example for an english speaking audience, but this link illustrates the idea:
Alot of pics n this one ;)
* Rendering wait time
Very simple, load a semi complex page (like slashdot with comments --a 500KB one), or a large one (as the mysql manual -- I have one which is more than two Megabytes long).
Press back, the forward. (you can have some coffee :)
Slashdot works perfect. What do you define as a compliant site? All done in CSS? no java script or flash? xhtml or html? Oops, meant for usability section.
* Time to start seeing content - streamed rendering
Get to a table-based semi complex page (300Kb-600Kb) and measure the time between asking for the URL and when you can start reading.
* Site usability
Well, this depends on whether the page respects standards, how broken the site implementation is, the browser's support for the technologies involved, user preferences (javascript may be disabled, same for cookies, a JVM may not be present, same for flash, etc).
Got it.
--
Now that's for having some numbers to show inside a table for people with above the average knowledge about browsers :-)
(very interesting, but not massive).
Nope, but very interesting. I will need a quick hand on the stop watch :-)
Simple usability experience can be explained to less technical people:
For instance, take dillo, and dig for something using google. You'll find that the time you use to make different queries and to check for alternative links until you find what you look for is better than with almost anything you've used before.
Noticed that.
The same is true for browsing local documentation (all of those html files under the /usr/doc or /usr/share/doc directories). This happens because the combined rendering speed and low download time (a filesystem) let you move very quickly, back and forward, make text searches inside the pages, go to another link, saving you lots of time.
Agreed.
Going with dillo to any web site is different, if it respects standards (ans so renders OK) its great, but if not, you'll have to resort to a bigger browser.
Agreed. :-(
The other important points to consider is to test with different band widths and CPU power. You'll barely notice how fast dillo is on a 2.2 Ghz machine, and if you only test on broad band you'll never know that it's possible to have a good browsing experience with a simple dialup.
That's it.
Cheers Jorge.-
- ------- End of Original Message ------- Thanks Jorge, I really appreciate your time and expertise. I think these will be used as the basis for browser benchmarks, stealling the reasoning you gave for the introductions :-) I will give you a preview of it as it starts to resemble something readable. Thanks again. P.S> I posted this to the list from my company account too. - -- Kind Regards, Gavin Henry. Director. Open Source. Open Solutions. http://www.suretecsystems.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFAO9X0eWseh9tzvqgRAr/uAJ9piEKRTm8E6Ndqi71Oaecg4kcL9gCfYdma x2S++i7TSEVnL4W8Fmu6J9k= =cwPk -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I was thinking about the timing of entering url to complete rendered page. How about a perl or bash script that simply checks the time a cache item was first entered into the cache dir until the last one. And that should be the full rendering time? I checked out ~/.dillo/ to look for a cache dir. Is it this setting use_dicache=NO to YES?? Sorry, I should RTFM. - -- Kind Regards, Gavin Henry. Director. Open Source. Open Solutions. http://www.suretecsystems.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFAO9zbeWseh9tzvqgRAuM9AJwIEayFmaLw0+7Tp1xuXFWmUhXivACgkdbG avlygLLvJbcVY7VaB3bfUcg= =ocI4 -----END PGP SIGNATURE-----
On Tue, 24 Feb 2004, Gavin Henry wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I was thinking about the timing of entering url to complete rendered page.
How about a perl or bash script that simply checks the time a cache item was first entered into the cache dir until the last one. And that should be the full rendering time?
This varies from browser to browser, and finding a custom way for each one is what I'd do. For instance: to visually check the images once the html page has arrived is one way (just select a page with closely placed images to avoid much scrolling).
I checked out ~/.dillo/ to look for a cache dir. Is it this setting use_dicache=NO to YES??
No (explained in the former email). Dillo doesn't cache to disk, but it will tell you when a page is done (look at the top meters). Note that when a page includes images inside iframe these will not be loaded, but you can check that in advance and wait for the image counter to reach a certain number. As I wrote above, the page completion check depends on the browser. Cheers Jorge.-
On Tue, 24 Feb 2004, Gavin Henry wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
- ---------- Original Message ----------- From: Jorge Arellano Cid <jcid@dillo.org> To: Gavin Henry <ghenry@fedoranews.org> Cc: Dillo mailing list <dillo-dev@auriga.wearlab.de> Sent: Tue, 24 Feb 2004 13:22:38 -0300 (CLST) Subject: Re: Upcoming review.
[list: this was originally a personal answer to Gavin, but as it has general interest information I'm CC to dillo-dev. It's about a browser comparison article. ]
[...] There're some interesting tests to be done. Which one you choose depends on the time you have, and on what you want to test for.
For instance, IMHO, the common "web experience" has four important factors:
So far I have tested the browsers on slashdot, and bbc.co.uk and a few others.
They all seem to wait until some pictures arrive until rendering anything. I haven't time these yet. Dillo, displays text immediately, but has some difficulty with CSS. For instance fedoranews.org looks bad :-)
Dillo doesn't have support for CSS yet. The closest we've got is a prototype. More info at: http://article.gmane.org/gmane.comp.web.dillo.devel/1827
Are you familiar with http://wwww.udcast.com/ whcih runs freebsd and has customised apache etc.
I thought maybe you have done something like this. As my day job www.invsat.com we had a demo from them and they explained the priciple of http precaching.
Instead of requesting 4 images then waiting, it requests all images at once, as this then reduces the latency across a satellite link.
Is this how your browser works? (*I haven't look at any code as of yet ;-) )
Yes, dillo does that and more. There's a paper explaining dillo's internals. You can find it at the bottom of the links section. It's large though.
* Network wait time
* Rendering wait time
* Time to start seeing content - streamed rendering
* Site usability
These will be what I base the benchmarks on.
Good!
These are key factors; for instance the same site may be unusable depending on whether you're on a dialup or broad band. The same is true for rendering time (influenced by CPU power and available memory); you may end going to some site only when you're sitting at a powerful workstation.
I don't have access to dial-up, but I have a celeron 400 and a athlon xp 2000 to test on.
The dialup test is really a must. Got to see to believe. There are slow-connection emulators. Sometime ago someone suggested one in this mailing list. Does anyone remember?
[...]
* Rendering wait time
Very simple, load a semi complex page (like slashdot with comments --a 500KB one), or a large one (as the mysql manual -- I have one which is more than two Megabytes long).
Press back, the forward. (you can have some coffee :)
Slashdot works perfect. What do you define as a compliant site?
A site that follows the W3C standards.
All done in CSS? no java script or flash? xhtml or html? Oops, meant for usability section.
You can find these answers here: http://www.dillo.org/funding/technical.html#fea In brief: CSS, java script, flash: NO xhtml, html: YES (note: xhtml works when served as "text/html" because the standard requires a validation we don't have for xhtml).
[...]
Simple usability experience can be explained to less technical people:
For instance, take dillo, and dig for something using google. You'll find that the time you use to make different queries and to check for alternative links until you find what you look for is better than with almost anything you've used before.
Noticed that.
Great!
Thanks Jorge, I really appreciate your time and expertise.
I think these will be used as the basis for browser benchmarks, stealling the reasoning you gave for the introductions :-)
Great! "The eyes can't see what the mind doesn't comprehend". The average user (and some Ph.Ds in this country ;) don't know about those concepts. Explaining them simply and briefly will certainly contribute to understanding the test data. Note that getting accurate hard numbers for each one of the listed points is close to impossible, it depends a lot on how the browser was built, and instrumentating the source code for each one is out of reach. For instance for testing rendering time (in any browser), you have to separate it from networking time. Having the page already in memory helps, but you don't know how much of the rendering is cached. Dillo can cache decompressed images (use_dicache=YES). This will make back and forward extremely fast with pages that contain images because dillo will have them already buffered as RGB. With firebird, you can quickly switch from one tab to another (the rendering is clearly cached), but doing the same with back and forward yields different results! Now, with browsers that cache to disk, HD access time can count... The tests I suggested take this into account and can provide "meningful" numbers, but not strictly accurate ones.
I will give you a preview of it as it starts to resemble something readable.
Thxs. Cheers Jorge.-
participants (2)
-
Gavin Henry -
Jorge Arellano Cid