July 2010 - Dillo-dev - lists.mailman3.com

Re: dillo mysteriously freezes

by corvid＠lavabit.com

Dennis wrote: > You're right. About everything :). The freezes don't happen with 2.0. I > miss the good blazing-fast old days, sometimes. They also don't seem to > happen after I downgraded my kernel from 2.6.35-rc3 to 2.6.34. > (Although now I have other networking problems -- nfs/scp/etc > stalls :S). I wonder whether there's any connection with http://kerneltrap.org/mailarchive/linux-kernel/2010/6/14/4582843/thread

July 6, 2010

Re: dillo mysteriously freezes

by dennisn＠dennisn.dyndns.org

On Mon, 5 Jul 2010 08:47:29 -0400, Jorge Arellano Cid wrote: > On Sun, Jul 04, 2010 at 10:46:50AM -0400, Dennis Nezic wrote: > > On Sun, 27 Jun 2010 08:22:38 -0400, Dennis Nezic wrote: > > > On Wed, 23 Jun 2010 09:19:10 -0400, Dennis Nezic wrote: > > > > On Sun, 20 Jun 2010 10:02:50 -0400, Jorge Arellano Cid wrote: > > > > > Hi, > > > > > > > > > > At this point, there seems to be something wrong with the > > > > > way the OS is handling its sockets and making them > > > > > available to the application (after all, the freezes happen > > > > > within system calls). > > > > > > > > > > Please try dillo and don't use any dpis, this will use > > > > > remote TCP sockets only. Just plain web browsing. If > > > > > this doesn't freeze, then there's a problem with local > > > > > sockets. > > > > > > > > Yea -- the freezes only occur with the dpis. (Either > > > > cookie-websites or local file browsing, or bookmarks, etc) > > > > > > > > > > > > > > If plain web browsing works, you can try letting a > > > > > chance to the OS's defaults by commenting setsockopt in: > > > > > > > > > > IO/dpi.c // line 442 in repo > > > > > dpid/dpid.c // line 525 in repo > > > > > > > > > > recompile > > > > > stop dpid // `dpidc stop` > > > > > reinstall // make install > > > > > retry > > > > > > > > That actually helped! I hesitated to post, because I did get one > > > > freeze a few days ago when trying to download a file (file.c), I > > > > didn't strace it to find which call exactly caused it, but > > > > otherwise that did seem to fix things! (Any ideas why the > > > > TCP_NODELAY flag is causing problems? :P) > > > > > > I should have hesitated. The freezes are back, about as > > > frequently as before. But I guess there's nothing dillo can do > > > about it :\. > > > > > > socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 5 > > > > > > connect(5, {sa_family=AF_INET, sin_port=htons(5020), > > > sin_addr=inet_addr ("127.0.0.1")}, 16 **~minute freeze**) = -1 > > > ETIMEDOUT (Connection timed out) > > > > Blargh. I'm not sure how to debug further. Netstat shows that dpid's > > sockets are in a LISTEN state during the freezes. Dillo's > > connection to dpid, according to netstat, is SYN_SENT. Telnet'ing > > to the dpid ports doesn't work during the freezes -- which sounds > > like a problem with dpid? But these freezes aren't "hard" -- > > sometimes they work -- once in a while it is able to unfreeze > > itself, at least for a bit, and I am able to telnet into dpid. > > Maybe dpid (or linux) is throttling socket connections or > > something? :| > > I believe the problem is in the way your particular kernel > interacts with dillo. No other GNU/Linux (nor BSD) that we know > of has this problem. As I'm no kernel expert, it's hard for me to > suggest a way to further debug the problem. > > You may try compiling a vanilla kernel from kernel.org or > another kernel version from gentoo, or if you really want to find > out where the freezes come from, asking a knowledgeable guy from > the TCP sockets area in Linux. > > BTW, I'd bet it doesn't happen with Unix domain sockets, that's > why I suggested trying dillo-2.0. Have you tried that? You're right. About everything :). The freezes don't happen with 2.0. I miss the good blazing-fast old days, sometimes. They also don't seem to happen after I downgraded my kernel from 2.6.35-rc3 to 2.6.34. (Although now I have other networking problems -- nfs/scp/etc stalls :S). Thanks for the help and tips!

July 6, 2010

Re: dillo mysteriously freezes

by jcid＠dillo.org

On Sun, Jul 04, 2010 at 10:46:50AM -0400, Dennis Nezic wrote: > On Sun, 27 Jun 2010 08:22:38 -0400, Dennis Nezic wrote: > > On Wed, 23 Jun 2010 09:19:10 -0400, Dennis Nezic wrote: > > > On Sun, 20 Jun 2010 10:02:50 -0400, Jorge Arellano Cid wrote: > > > > Hi, > > > > > > > > At this point, there seems to be something wrong with the way > > > > the OS is handling its sockets and making them available to the > > > > application (after all, the freezes happen within system calls). > > > > > > > > Please try dillo and don't use any dpis, this will use remote > > > > TCP sockets only. Just plain web browsing. If this doesn't > > > > freeze, then there's a problem with local sockets. > > > > > > Yea -- the freezes only occur with the dpis. (Either cookie-websites > > > or local file browsing, or bookmarks, etc) > > > > > > > > > > > If plain web browsing works, you can try letting a chance to > > > > the OS's defaults by commenting setsockopt in: > > > > > > > > IO/dpi.c // line 442 in repo > > > > dpid/dpid.c // line 525 in repo > > > > > > > > recompile > > > > stop dpid // `dpidc stop` > > > > reinstall // make install > > > > retry > > > > > > That actually helped! I hesitated to post, because I did get one > > > freeze a few days ago when trying to download a file (file.c), I > > > didn't strace it to find which call exactly caused it, but otherwise > > > that did seem to fix things! (Any ideas why the TCP_NODELAY flag is > > > causing problems? :P) > > > > I should have hesitated. The freezes are back, about as frequently as > > before. But I guess there's nothing dillo can do about it :\. > > > > socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 5 > > > > connect(5, {sa_family=AF_INET, sin_port=htons(5020), > > sin_addr=inet_addr ("127.0.0.1")}, 16 **~minute freeze**) = -1 > > ETIMEDOUT (Connection timed out) > > Blargh. I'm not sure how to debug further. Netstat shows that dpid's > sockets are in a LISTEN state during the freezes. Dillo's connection to > dpid, according to netstat, is SYN_SENT. Telnet'ing to the dpid ports > doesn't work during the freezes -- which sounds like a problem with > dpid? But these freezes aren't "hard" -- sometimes they work -- once in > a while it is able to unfreeze itself, at least for a bit, and I am > able to telnet into dpid. Maybe dpid (or linux) is throttling socket > connections or something? :| I believe the problem is in the way your particular kernel interacts with dillo. No other GNU/Linux (nor BSD) that we know of has this problem. As I'm no kernel expert, it's hard for me to suggest a way to further debug the problem. You may try compiling a vanilla kernel from kernel.org or another kernel version from gentoo, or if you really want to find out where the freezes come from, asking a knowledgeable guy from the TCP sockets area in Linux. BTW, I'd bet it doesn't happen with Unix domain sockets, that's why I suggested trying dillo-2.0. Have you tried that? -- Cheers Jorge.-

July 5, 2010

Re: dillo mysteriously freezes

by dennisn＠dennisn.dyndns.org

On Sun, 27 Jun 2010 08:22:38 -0400, Dennis Nezic wrote: > On Wed, 23 Jun 2010 09:19:10 -0400, Dennis Nezic wrote: > > On Sun, 20 Jun 2010 10:02:50 -0400, Jorge Arellano Cid wrote: > > > Hi, > > > > > > At this point, there seems to be something wrong with the way > > > the OS is handling its sockets and making them available to the > > > application (after all, the freezes happen within system calls). > > > > > > Please try dillo and don't use any dpis, this will use remote > > > TCP sockets only. Just plain web browsing. If this doesn't > > > freeze, then there's a problem with local sockets. > > > > Yea -- the freezes only occur with the dpis. (Either cookie-websites > > or local file browsing, or bookmarks, etc) > > > > > > > > If plain web browsing works, you can try letting a chance to > > > the OS's defaults by commenting setsockopt in: > > > > > > IO/dpi.c // line 442 in repo > > > dpid/dpid.c // line 525 in repo > > > > > > recompile > > > stop dpid // `dpidc stop` > > > reinstall // make install > > > retry > > > > That actually helped! I hesitated to post, because I did get one > > freeze a few days ago when trying to download a file (file.c), I > > didn't strace it to find which call exactly caused it, but otherwise > > that did seem to fix things! (Any ideas why the TCP_NODELAY flag is > > causing problems? :P) > > I should have hesitated. The freezes are back, about as frequently as > before. But I guess there's nothing dillo can do about it :\. > > socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 5 > > connect(5, {sa_family=AF_INET, sin_port=htons(5020), > sin_addr=inet_addr ("127.0.0.1")}, 16 **~minute freeze**) = -1 > ETIMEDOUT (Connection timed out) I have also noticed that, after 11 tries, ... Dpi_blocking_start_dpid: try 11 Dpi_check_dpid_ids: Connection timed out dpid seems to respawn itself, with a new mksecret (updated dpid_comm_keys). Why is it respawning itself without stopping it's earlier instances?

July 4, 2010

Re: dillo mysteriously freezes

by dennisn＠dennisn.dyndns.org

On Sun, 27 Jun 2010 08:22:38 -0400, Dennis Nezic wrote: > On Wed, 23 Jun 2010 09:19:10 -0400, Dennis Nezic wrote: > > On Sun, 20 Jun 2010 10:02:50 -0400, Jorge Arellano Cid wrote: > > > Hi, > > > > > > At this point, there seems to be something wrong with the way > > > the OS is handling its sockets and making them available to the > > > application (after all, the freezes happen within system calls). > > > > > > Please try dillo and don't use any dpis, this will use remote > > > TCP sockets only. Just plain web browsing. If this doesn't > > > freeze, then there's a problem with local sockets. > > > > Yea -- the freezes only occur with the dpis. (Either cookie-websites > > or local file browsing, or bookmarks, etc) > > > > > > > > If plain web browsing works, you can try letting a chance to > > > the OS's defaults by commenting setsockopt in: > > > > > > IO/dpi.c // line 442 in repo > > > dpid/dpid.c // line 525 in repo > > > > > > recompile > > > stop dpid // `dpidc stop` > > > reinstall // make install > > > retry > > > > That actually helped! I hesitated to post, because I did get one > > freeze a few days ago when trying to download a file (file.c), I > > didn't strace it to find which call exactly caused it, but otherwise > > that did seem to fix things! (Any ideas why the TCP_NODELAY flag is > > causing problems? :P) > > I should have hesitated. The freezes are back, about as frequently as > before. But I guess there's nothing dillo can do about it :\. > > socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 5 > > connect(5, {sa_family=AF_INET, sin_port=htons(5020), > sin_addr=inet_addr ("127.0.0.1")}, 16 **~minute freeze**) = -1 > ETIMEDOUT (Connection timed out) Blargh. I'm not sure how to debug further. Netstat shows that dpid's sockets are in a LISTEN state during the freezes. Dillo's connection to dpid, according to netstat, is SYN_SENT. Telnet'ing to the dpid ports doesn't work during the freezes -- which sounds like a problem with dpid? But these freezes aren't "hard" -- sometimes they work -- once in a while it is able to unfreeze itself, at least for a bit, and I am able to telnet into dpid. Maybe dpid (or linux) is throttling socket connections or something? :|

July 4, 2010

youtube-dl

by corvid＠lavabit.com

Jeremy wrote: > corvid wrote: > > Maybe it would be nice to have the dpis able to register interest in > > resources meeting certain criteria or something. Or maybe the trick > > is to simply :) implement the NPAPI. > > ??? What is this "NPAPI" of which you speak, Earth being? :-) I > guess I've not been keeping up. Sorry, I wrote more and then I guess I trimmed it down too much. Netscape Plugin API The mighty wikipedia claims: The following web browsers support NPAPI plugins: o Epiphany o Google Chrome o Konqueror o Mozilla project applications, including o Camino o Firefox o Mozilla Application Suite o Seamonkey o Netscape Navigator and Communicator o Opera

July 4, 2010

youtube-dl

by onepoint＠starurchin.org

corvid wrote: > Maybe it would be nice to have the dpis able to register interest in > resources meeting certain criteria or something. Or maybe the trick > is to simply :) implement the NPAPI. ??? What is this "NPAPI" of which you speak, Earth being? :-) I guess I've not been keeping up. Jeremy Henty

July 3, 2010

youtube-dl

by corvid＠lavabit.com

Jeremy wrote: > The problem is that, last time I looked at any > rate, dpis are invoked by specific URL schemes. What I'm thinking of > would require dillo to invoke dpis when it sees particular bits of > HTML. That might be non-trivial to implement. Yeah, everything's awfully hardcoded in capi.c. I imagine it would require a fair amount of untangling and thinking. Maybe it would be nice to have the dpis able to register interest in resources meeting certain criteria or something. Or maybe the trick is to simply :) implement the NPAPI. BTW, for <object>s, I remember from the html4 spec that one is _supposed_ to use the data attribute, but nearly everyone ignores that and just makes up a... <param movie=[url]> tag instead, IIRC.

July 3, 2010

youtube-dl

by onepoint＠starurchin.org

robert w hall wrote: > Would it be 'easy' to turn youtube-dl (written in python) into a > dillo 'plugin'?? That's a fun idea! I'm a big fan of youtube-dl. But I'm not sure what you're suggesting. My dream would be to click on an embedded youtube video and have the download dpi (or something else like it) pull down the video. The problem is that, last time I looked at any rate, dpis are invoked by specific URL schemes. What I'm thinking of would require dillo to invoke dpis when it sees particular bits of HTML. That might be non-trivial to implement. As a side question, how much work would it be to get dillo to render <embed> and <object> elements in a useful way? Now there's a thought! How about if dillo rendered them as clickable boxes linked to some URL like embed://?html=<escaped_html>? Then they could be handled by a dpi that might eg. extract Youtube video URLs and run youtube-dl! Does this sound feasible? Regards, Jeremy Henty

July 2, 2010

youtube-dl

by bobh＠n-cantrell.demon.co.uk

Would it be 'easy' to turn youtube-dl (written in python) into a dillo 'plugin'?? Bopb -- robert w hall

July 1, 2010