- Dillo-dev - lists.mailman3.com

experimental patch: Re: dillo2 wrapping Chinese

by corvid＠lavabit.com

I made yet another version of this patch the other day, and furaisanjin was kind enough to run on it a bit and ensure it's doing what we want. With the previous version, compiling without optimization, gprof showed 3-4% loss of speed on a page that's mostly English text. - I removed the strdup/free from the common case. - For nonideographic, it goes character by character instead of checking every byte. This should help for, e.g., Russian, but means some 'unnecessary' length-checking in the mostly-ASCII case. And gprof suggested the slowdown might be 3% now, although at this point, unless someone is paying you to perform a lot of careful, drudgery-filled runs, it gets hard to pick the signal out of the noise when examining the time spent in relevant functions. I tried adding in some code to bypass the ideographic call for ASCII, but 1) it made the code a little messy 2) it looked like it might be saving a few tenths of a percent 3) the compiler might perform optimizations under certain circumstances making my efforts unnecessary or counterproductive. I could definitely be talked into this one, though. I tried adding an addText() with a length argument and thought it might help by saving me the trouble of writing NULL bytes into the strings and saving the trouble of some strlen()ning in dw. I couldn't see any difference in speed, although it did make things cleaner and I would want to add it in eventually if this all makes it into the repository. There will be a duplicated ideographic() test, for instance, if a word starts with digits and ends with some Japanese characters. 1) uncommon 2) complexity This patch is really two patches because it also handles the case where whitespace numeric character references are inside a word, but I wanted to see the effects of both and they are each an excuse for the overhead of the loop. I'd split it into two pieces if committing it.

Aug. 2, 2009

Planning

by corvid＠lavabit.com

Jorge wrote: > On Thu, Jul 30, 2009 at 02:13:52AM +0000, corvid wrote: > > Jorge wrote: > > > On Tue, Jul 28, 2009 at 10:01:12PM -0700, bf wrote: > > > > --- On Fri, 7/24/09, Jorge Arellano Cid <jcid(a)dillo.org> wrote: > > > > I don't see any mention of a limit on concurrent connections > > > > that was proposed earlier, and as far as I know it hasn't been > > > > implemented. In addition to the reasons given earlier, it is very > > > > frustrating to have to implement your own traffic-shaping just to > > > > avoid being blocked by a rate/connection-limiting firewall, or to > > > > avoid being blocklisted by a site you had hoped to visit. I'd > > > > hate to see dillo gain a reputation as a problematic UserAgent, > > > > and perhaps be blocked by some heavy-handed administrators. > > > > If I had magical make-Jorge-do-my-bidding powers, this would be on the > > top of my list. > > Interesting... > > I'll give it a look, try to come up with raw patch and > send it to you for polish. Would you? I can try. It would be nice if we could have Keep-Alive. (Conceivably pipelining someday, but there might be too many annoying failure modes to recover from.) > > > * Document CCC, dlib and dillo in doxygen format. > > > > The situation seems to me as follows: > > pre-doxygen: easy-to-read code, useless documentation > > post-doxygen: hard-to-read code, useless documentation > > Oh, sounds harsh! > > For me Dw docs have been a bless. Without them I would not be > able to rewrite the table-formating algorithms. The conceptual dw docs (doc/*.doc) have helped me understand some things at times (but not due to their doxygenicity), but the doxygen stuff in the source makes it harder for me to read comments. > Non-dw documents under doc/ are outdated and I have to agree > they are of little use. I'll try to write one about CCC that will > hopefully help you with the limit on concurrent connections. Yeah, I should dig through some of doc/*.txt and update some things in the areas I'm familiar with.

July 31, 2009

Planning

by jcid＠dillo.org

On Fri, Jul 31, 2009 at 07:40:20AM +0200, Johannes Hofmann wrote: > On Thu, Jul 30, 2009 at 07:55:43PM -0400, Jorge Arellano Cid wrote: > > On Thu, Jul 30, 2009 at 02:13:52AM +0000, corvid wrote: > > > Jorge wrote: > > > > On Tue, Jul 28, 2009 at 10:01:12PM -0700, bf wrote: > > > > > --- On Fri, 7/24/09, Jorge Arellano Cid <jcid(a)dillo.org> wrote: > > > > > I don't see any mention of a limit on concurrent connections > > > > > that was proposed earlier, and as far as I know it hasn't been > > > > > implemented. In addition to the reasons given earlier, it is very > > > > > frustrating to have to implement your own traffic-shaping just to > > > > > avoid being blocked by a rate/connection-limiting firewall, or to > > > > > avoid being blocklisted by a site you had hoped to visit. I'd > > > > > hate to see dillo gain a reputation as a problematic UserAgent, > > > > > and perhaps be blocked by some heavy-handed administrators. > > > > > > If I had magical make-Jorge-do-my-bidding powers, this would be on the > > > top of my list. > > > > Interesting... > > > > I'll give it a look, try to come up with raw patch and > > send it to you for polish. Would you? > > > > > > * Document CCC, dlib and dillo in doxygen format. > > > > > > The situation seems to me as follows: > > > pre-doxygen: easy-to-read code, useless documentation > > > post-doxygen: hard-to-read code, useless documentation > > > > Oh, sounds harsh! > > Yes that's a bit harsh, but I think corvid is right to some degree. > doxygen style comments that describe every method and their > parameters are not needed in dillo. It is no library after all. > What is really important is an up-to-date overview document > that explains the concepts. The rest can mostly be figured out from > the code. > > > > > For me Dw docs have been a bless. Without them I would not be > > able to rewrite the table-formating algorithms. > > Yes. the dw/* documentation is good because it describes the > concepts. But if I want to find out what some method does, I now > look at the code, not the docs. FWIW, when I said "doxygen format" I meant something similar to dw. As you remark, the concepts are the important part. The functions are well commented in the code. I don't know whether there's a fixed doxygen standard, and surely didn't mean it. Sorry for the confusion. -- Cheers Jorge.-

July 31, 2009

Planning

by Johannes.Hofmann＠gmx.de

On Thu, Jul 30, 2009 at 07:55:43PM -0400, Jorge Arellano Cid wrote: > On Thu, Jul 30, 2009 at 02:13:52AM +0000, corvid wrote: > > Jorge wrote: > > > On Tue, Jul 28, 2009 at 10:01:12PM -0700, bf wrote: > > > > --- On Fri, 7/24/09, Jorge Arellano Cid <jcid(a)dillo.org> wrote: > > > > I don't see any mention of a limit on concurrent connections > > > > that was proposed earlier, and as far as I know it hasn't been > > > > implemented. In addition to the reasons given earlier, it is very > > > > frustrating to have to implement your own traffic-shaping just to > > > > avoid being blocked by a rate/connection-limiting firewall, or to > > > > avoid being blocklisted by a site you had hoped to visit. I'd > > > > hate to see dillo gain a reputation as a problematic UserAgent, > > > > and perhaps be blocked by some heavy-handed administrators. > > > > If I had magical make-Jorge-do-my-bidding powers, this would be on the > > top of my list. > > Interesting... > > I'll give it a look, try to come up with raw patch and > send it to you for polish. Would you? > > > > * Document CCC, dlib and dillo in doxygen format. > > > > The situation seems to me as follows: > > pre-doxygen: easy-to-read code, useless documentation > > post-doxygen: hard-to-read code, useless documentation > > Oh, sounds harsh! Yes that's a bit harsh, but I think corvid is right to some degree. doxygen style comments that describe every method and their parameters are not needed in dillo. It is no library after all. What is really important is an up-to-date overview document that explains the concepts. The rest can mostly be figured out from the code. > > For me Dw docs have been a bless. Without them I would not be > able to rewrite the table-formating algorithms. Yes. the dw/* documentation is good because it describes the concepts. But if I want to find out what some method does, I now look at the code, not the docs. > > Non-dw documents under doc/ are outdated and I have to agree > they are of little use. I'll try to write one about CCC that will > hopefully help you with the limit on concurrent connections. That would be great. Cheers, Johannes

July 31, 2009

Planning

by jcid＠dillo.org

On Thu, Jul 30, 2009 at 02:13:52AM +0000, corvid wrote: > Jorge wrote: > > On Tue, Jul 28, 2009 at 10:01:12PM -0700, bf wrote: > > > --- On Fri, 7/24/09, Jorge Arellano Cid <jcid(a)dillo.org> wrote: > > > I don't see any mention of a limit on concurrent connections > > > that was proposed earlier, and as far as I know it hasn't been > > > implemented. In addition to the reasons given earlier, it is very > > > frustrating to have to implement your own traffic-shaping just to > > > avoid being blocked by a rate/connection-limiting firewall, or to > > > avoid being blocklisted by a site you had hoped to visit. I'd > > > hate to see dillo gain a reputation as a problematic UserAgent, > > > and perhaps be blocked by some heavy-handed administrators. > > If I had magical make-Jorge-do-my-bidding powers, this would be on the > top of my list. Interesting... I'll give it a look, try to come up with raw patch and send it to you for polish. Would you? > > * Document CCC, dlib and dillo in doxygen format. > > The situation seems to me as follows: > pre-doxygen: easy-to-read code, useless documentation > post-doxygen: hard-to-read code, useless documentation Oh, sounds harsh! For me Dw docs have been a bless. Without them I would not be able to rewrite the table-formating algorithms. Non-dw documents under doc/ are outdated and I have to agree they are of little use. I'll try to write one about CCC that will hopefully help you with the limit on concurrent connections. -- Cheers Jorge.-

July 31, 2009

Planning

by corvid＠lavabit.com

Jorge wrote: > On Tue, Jul 28, 2009 at 10:01:12PM -0700, bf wrote: > > --- On Fri, 7/24/09, Jorge Arellano Cid <jcid(a)dillo.org> wrote: > > > > > From: Jorge Arellano Cid <jcid(a)dillo.org> > > > Subject: [Dillo-dev] Planning > > > To: "Dillo mailing list" <dillo-dev(a)dillo.org> > > > Date: Friday, July 24, 2009, 11:33 AM > > > Hi, > > > > > > ? Here? are? some? pending? issues > > > which I've partially sorted by > > > priority (at least the top half). Please add the missing > > > ones and > > > let's try to finish sorting it, with a view to planify our > > > work. > > > > > * Add blocker (this also has to do with privacy). > > > ???A? full? RE-based? > > > one? may? have? significant? overhead > > > (test > > > required),? but? simple string-matching may do > > > most of the trick. > > > I've? also? thought? for? a? > > > long? time? that? a "don't load from > > > other? sites"? preference? may? help a > > > lot. The whole topic needs > > > some thought but it's not hard to implement. > > > ???e.g. doubleclick. > > > > Or you could just point users to privoxy, junkbuster, WWWOFFLE, and/or polipo. Perhaps so. I would like some way to make it so that dillo would always ask me before following redirections, but maybe I should just argue for a pref for that specific thing rather than going for a whole general adblockerplus deal. > > I don't see any mention of a limit on concurrent connections > > that was proposed earlier, and as far as I know it hasn't been > > implemented. In addition to the reasons given earlier, it is very > > frustrating to have to implement your own traffic-shaping just to > > avoid being blocked by a rate/connection-limiting firewall, or to > > avoid being blocklisted by a site you had hoped to visit. I'd > > hate to see dillo gain a reputation as a problematic UserAgent, > > and perhaps be blocked by some heavy-handed administrators. If I had magical make-Jorge-do-my-bidding powers, this would be on the top of my list. > * Document CCC, dlib and dillo in doxygen format. The situation seems to me as follows: pre-doxygen: easy-to-read code, useless documentation post-doxygen: hard-to-read code, useless documentation > Please Johannes or corvid, sort the titles of the items we have > so far in this thread so we can sort them and do our planning. I guess I'll paw through it and stuff everything into Plans.html.

July 30, 2009

Planning

by jcid＠dillo.org

On Wed, Jul 29, 2009 at 02:14:30PM -0400, Jorge Arellano Cid wrote: > [...] > That was an incomplete list to start sorting priorities. BTW, we haven't even mentioned the javascript prototype, which is certainly a major topic. This mainly because it needs a lot of thought, a decision on whether to do it through dpip, etc. We'd be in a better shape to decide on scripting once the floats are in place, and some of the related topics are solved. -- Cheers Jorge.-

July 29, 2009

Planning

by jcid＠dillo.org

On Tue, Jul 28, 2009 at 10:01:12PM -0700, bf wrote: > > > > --- On Fri, 7/24/09, Jorge Arellano Cid <jcid(a)dillo.org> wrote: > > > From: Jorge Arellano Cid <jcid(a)dillo.org> > > Subject: [Dillo-dev] Planning > > To: "Dillo mailing list" <dillo-dev(a)dillo.org> > > Date: Friday, July 24, 2009, 11:33 AM > > Hi, > > > > ? Here? are? some? pending? issues > > which I've partially sorted by > > priority (at least the top half). Please add the missing > > ones and > > let's try to finish sorting it, with a view to planify our > > work. > > > * Add blocker (this also has to do with privacy). > > ???A? full? RE-based? > > one? may? have? significant? overhead > > (test > > required),? but? simple string-matching may do > > most of the trick. > > I've? also? thought? for? a? > > long? time? that? a "don't load from > > other? sites"? preference? may? help a > > lot. The whole topic needs > > some thought but it's not hard to implement. > > ???e.g. doubleclick. > > Or you could just point users to privoxy, junkbuster, WWWOFFLE, and/or polipo. > > I don't see any mention of a limit on concurrent connections > that was proposed earlier, and as far as I know it hasn't been > implemented. In addition to the reasons given earlier, it is very > frustrating to have to implement your own traffic-shaping just to > avoid being blocked by a rate/connection-limiting firewall, or to > avoid being blocklisted by a site you had hoped to visit. I'd > hate to see dillo gain a reputation as a problematic UserAgent, > and perhaps be blocked by some heavy-handed administrators. That was an incomplete list to start sorting priorities. * Limit number of concurrent connections. is certainly a valid item, also as is * Document CCC, dlib and dillo in doxygen format. Please Johannes or corvid, sort the titles of the items we have so far in this thread so we can sort them and do our planning. -- Cheers Jorge.-

July 29, 2009

Re: dillo crash at dlib/dlib.c:173

by stevegigijoe＠yahoo.com.tw

Hello guys I'm Sorry. Corvid is right. I must mass up something :p Steve ______________________________________________________________________________________________________ ¥I¶O€~®e¶qµL€W¡HYahoo!©_Œ¯¹q€l«Hœc2.0§K¶Oµ¹§A¡A«H¥ó¥Ã»·€£¥²§R¡I http://tw.mg0.mail.yahoo.com/dc/landing

July 29, 2009

Planning

by bf2006a＠yahoo.com

--- On Fri, 7/24/09, Jorge Arellano Cid <jcid(a)dillo.org> wrote: > From: Jorge Arellano Cid <jcid(a)dillo.org> > Subject: [Dillo-dev] Planning > To: "Dillo mailing list" <dillo-dev(a)dillo.org> > Date: Friday, July 24, 2009, 11:33 AM > Hi, > > ? Here? are? some? pending? issues > which I've partially sorted by > priority (at least the top half). Please add the missing > ones and > let's try to finish sorting it, with a view to planify our > work. > * Add blocker (this also has to do with privacy). > ???A? full? RE-based? > one? may? have? significant? overhead > (test > required),? but? simple string-matching may do > most of the trick. > I've? also? thought? for? a? > long? time? that? a "don't load from > other? sites"? preference? may? help a > lot. The whole topic needs > some thought but it's not hard to implement. > ???e.g. doubleclick. Or you could just point users to privoxy, junkbuster, WWWOFFLE, and/or polipo. I don't see any mention of a limit on concurrent connections that was proposed earlier, and as far as I know it hasn't been implemented. In addition to the reasons given earlier, it is very frustrating to have to implement your own traffic-shaping just to avoid being blocked by a rate/connection-limiting firewall, or to avoid being blocklisted by a site you had hoped to visit. I'd hate to see dillo gain a reputation as a problematic UserAgent, and perhaps be blocked by some heavy-handed administrators. b.

July 29, 2009