January 2010 - Dillo-dev - lists.mailman3.com

dpi send cmd error msg (Re: [Dillo-dev] Re: even more cookies SECURITY)

by corvid＠lavabit.com

Jorge wrote: > On Sun, Jan 24, 2010 at 11:04:52AM +0000, corvid wrote: > > Jorge wrote: > > > On Sun, Jan 10, 2010 at 02:00:35AM +0000, corvid wrote: > > > > Jorge wrote: > > > > > On Mon, Jan 04, 2010 at 11:38:03PM +0000, corvid wrote: > > > > > > Do we currently have anything that needs to be > > > > > > fixed/implemented before a release would be sensible? > > > > > > > > > > Not that I remember. The only problem I recall was the error > > > > > message you reported with cookies and later seemed to be working > > > > > somehow. > > > > > > > > a_Dpi_send_blocking_cmd() gets upset when > > > > Dpi_blocking_read() doesn't get anything. > > > > > > > > So in the cookie dpi's srv_parse_tok(), > > > > after the Cookies_set(), I tried putting a > > > > a_Dpip_dsh_write_str(sh, 1, "text"), and > > > > that makes the error msgs go away for me. > > > > > > Sorry for the long time to answer. Here's near 35 Celsius among > > > other things... :-P > > > > > > You're right, there's one end expecting an answer that never > > > comes. Something like the attached patch should be enough. > > > > > > Note that this bug may cause some networking delays depending > > > on how the underlaying OS handles closing the socket when there's > > > a blocking reader (it may try to delay the close). > > > > > > The patch avoids this. > > > > I'm a bit hesitant to return a msg if we don't plan to do anything with it, > > but then if we have to send back something, it does make sense to put some > > information in there... > > That's the canonical way of doing it in DPIP. If the message > sender is not interested in the answer, he can ignore it (as in > this case). This allows for a simpler communication algorithm and > a uniform way of dealing with DPIP messages. > > The patch sends an answer that says whether the cookie was set. > It is easy to extend it to say what happened if the need arises. Committed.

Jan. 24, 2010

dpi send cmd error msg (Re: [Dillo-dev] Re: even more cookies SECURITY)

by jcid＠dillo.org

On Sun, Jan 24, 2010 at 11:04:52AM +0000, corvid wrote: > Jorge wrote: > > On Sun, Jan 10, 2010 at 02:00:35AM +0000, corvid wrote: > > > Jorge wrote: > > > > On Mon, Jan 04, 2010 at 11:38:03PM +0000, corvid wrote: > > > > > Do we currently have anything that needs to be > > > > > fixed/implemented before a release would be sensible? > > > > > > > > Not that I remember. The only problem I recall was the error > > > > message you reported with cookies and later seemed to be working > > > > somehow. > > > > > > a_Dpi_send_blocking_cmd() gets upset when > > > Dpi_blocking_read() doesn't get anything. > > > > > > So in the cookie dpi's srv_parse_tok(), > > > after the Cookies_set(), I tried putting a > > > a_Dpip_dsh_write_str(sh, 1, "text"), and > > > that makes the error msgs go away for me. > > > > Sorry for the long time to answer. Here's near 35 Celsius among > > other things... :-P > > > > You're right, there's one end expecting an answer that never > > comes. Something like the attached patch should be enough. > > > > Note that this bug may cause some networking delays depending > > on how the underlaying OS handles closing the socket when there's > > a blocking reader (it may try to delay the close). > > > > The patch avoids this. > > I'm a bit hesitant to return a msg if we don't plan to do anything with it, > but then if we have to send back something, it does make sense to put some > information in there... That's the canonical way of doing it in DPIP. If the message sender is not interested in the answer, he can ignore it (as in this case). This allows for a simpler communication algorithm and a uniform way of dealing with DPIP messages. The patch sends an answer that says whether the cookie was set. It is easy to extend it to say what happened if the need arises. -- Cheers Jorge.-

Jan. 24, 2010

dpi send cmd error msg (Re: [Dillo-dev] Re: even more cookies SECURITY)

by corvid＠lavabit.com

Jorge wrote: > On Sun, Jan 10, 2010 at 02:00:35AM +0000, corvid wrote: > > Jorge wrote: > > > On Mon, Jan 04, 2010 at 11:38:03PM +0000, corvid wrote: > > > > Do we currently have anything that needs to be > > > > fixed/implemented before a release would be sensible? > > > > > > Not that I remember. The only problem I recall was the error > > > message you reported with cookies and later seemed to be working > > > somehow. > > > > a_Dpi_send_blocking_cmd() gets upset when > > Dpi_blocking_read() doesn't get anything. > > > > So in the cookie dpi's srv_parse_tok(), > > after the Cookies_set(), I tried putting a > > a_Dpip_dsh_write_str(sh, 1, "text"), and > > that makes the error msgs go away for me. > > Sorry for the long time to answer. Here's near 35 Celsius among > other things... :-P > > You're right, there's one end expecting an answer that never > comes. Something like the attached patch should be enough. > > Note that this bug may cause some networking delays depending > on how the underlaying OS handles closing the socket when there's > a blocking reader (it may try to delay the close). > > The patch avoids this. I'm a bit hesitant to return a msg if we don't plan to do anything with it, but then if we have to send back something, it does make sense to put some information in there...

Jan. 24, 2010

dpi send cmd error msg (Re: [Dillo-dev] Re: even more cookies SECURITY)

by jcid＠dillo.org

On Sun, Jan 10, 2010 at 02:00:35AM +0000, corvid wrote: > Jorge wrote: > > On Mon, Jan 04, 2010 at 11:38:03PM +0000, corvid wrote: > > > Do we currently have anything that needs to be > > > fixed/implemented before a release would be sensible? > > > > Not that I remember. The only problem I recall was the error > > message you reported with cookies and later seemed to be working > > somehow. > > a_Dpi_send_blocking_cmd() gets upset when > Dpi_blocking_read() doesn't get anything. > > So in the cookie dpi's srv_parse_tok(), > after the Cookies_set(), I tried putting a > a_Dpip_dsh_write_str(sh, 1, "text"), and > that makes the error msgs go away for me. Sorry for the long time to answer. Here's near 35 Celsius among other things... :-P You're right, there's one end expecting an answer that never comes. Something like the attached patch should be enough. Note that this bug may cause some networking delays depending on how the underlaying OS handles closing the socket when there's a blocking reader (it may try to delay the close). The patch avoids this. -- Cheers Jorge.-

Jan. 23, 2010

connection limit (was: display: inline)

by jcid＠dillo.org

On Sat, Jan 23, 2010 at 09:07:10PM +0100, Johannes Hofmann wrote: > > Rogut??s Sparnuotos : > > > > I have skimmed over hundreds of photos on flickr.com with both, vanilla > > dillo-2.1.1 and dillo-hg+connlimit2, with proxy and without. Browsing > > seems to work fine with all variations (I didn't try to analyze the > > packets), but I had a hard time trying to see the difference: I guess my > > internet connection is too fast and my X server too slow. Since I have > > a good internet connection, the images loaded faster without the patch, > > but sometimes the ones at the top of a page appeared sooner when the patch > > was applied. Openning all the connections at once helps speed by making the wait for connections parallel for all images. This is noticeable with a big network latency. > Yes, that's expected. The purpose of the patch is not to improve > loading speed or user experience, but to play nice with servers and > infrastructure. > Modern sites especially big ones like flickr are well prepared > to handle the load caused by the few dillo users out there. > Nevertheless it's important to use network resources in a sensible > and compliant way. I expect most of the mainstream servers to queue multiple requests and thence serve from their queue as much as they want. Other applications may have trouble with a herd of requests at the same time and hence prefer a connection limit. Pages with hundreds of non-tiny images (e.g. some blogs) may *feel* better with throttled requests; here users see the nice effect of images loading "in order". > > Since dillo and the servers seem to work with unlimited connections, > > perhaps defaulting to a limit of 6 wouldn't hurt? > > Yes, the current limit is certainly debatable. > We could even make it configurable. > Any opinions on this? Configurable with a dillorc option would be welcomed. We could comment suggested values for dialup, low band, broadband, ... > > > > Dillo crashed twice in the beginning of my testing, but I am not sure > > anymore which build it was (because I've applied the gnutls-0.patch at > > first - https seemed to work). I couldn't reproduce the crashes > > afterwards. > > Please keep an eye on potential crashes. > Those need to be ironed out of course. I remember when making the first connection limit patch to have apprehensions on the stop/abort process. This is very complex inside Dillo. If you want to crash-test, try going to pages then pressing stop (or quiting that tab), or opening a new one while the other loads, then aborting the first one. Things like that will make dillo stop, clean and keep going. That's the gist. Thanks for your feedback. -- Cheers Jorge.-

Jan. 23, 2010

connection limit (was: display: inline)

by Johannes.Hofmann＠gmx.de

> Rogut?s Sparnuotos : > > I have skimmed over hundreds of photos on flickr.com with both, vanilla > dillo-2.1.1 and dillo-hg+connlimit2, with proxy and without. Browsing > seems to work fine with all variations (I didn't try to analyze the > packets), but I had a hard time trying to see the difference: I guess my > internet connection is too fast and my X server too slow. Since I have > a good internet connection, the images loaded faster without the patch, > but sometimes the ones at the top of a page appeared sooner when the patch > was applied. Yes, that's expected. The purpose of the patch is not to improve loading speed or user experience, but to play nice with servers and infrastructure. Modern sites especially big ones like flickr are well prepared to handle the load caused by the few dillo users out there. Nevertheless it's important to use network resources in a sensible and compliant way. > > Since dillo and the servers seem to work with unlimited connections, > perhaps defaulting to a limit of 6 wouldn't hurt? Yes, the current limit is certainly debatable. We could even make it configurable. Any opinions on this? > > Dillo crashed twice in the beginning of my testing, but I am not sure > anymore which build it was (because I've applied the gnutls-0.patch at > first - https seemed to work). I couldn't reproduce the crashes > afterwards. Please keep an eye on potential crashes. Those need to be ironed out of course. Thanks for reporting back, Johannes -- Haiti-Nothilfe! Helfen Sie per SMS: Sende UIHAITI an die Nummer 81190. Von 5 Euro je SMS (zzgl. SMS-Geb?hr) gehen 4,83 Euro an UNICEF.

Jan. 23, 2010

connection limit (was: display: inline)

by rogutes＠googlemail.com

Johannes Hofmann (2010-01-21 21:07): > Hi, > > I made some progress on the connection limit stuff. > > On Wed, Jan 20, 2010 at 02:43:17PM +0100, Johannes Hofmann wrote: > > On Wed, Jan 20, 2010 at 08:59:12AM -0300, Jorge Arellano Cid wrote: > > > > > > BTW, is the connection-number-limit patch in the next release? > > > > No, it's not ready yet - unfortunately. We urgently need a connection > > limit, but it needs to be per remote host, as otherwise dillo can > > have all of it's 4 connections open to a slow server and then there > > is no connection left for other servers, e.g. used for other windows > > or tabs. > > The limit is now per remote host, so slow servers don't block > connections to other (faster) ones. > Unfortunately the hash table implementation from lout can't be used, > as http.c is C file. > However the number of hosts to which we have open connections at a > time seems to be quite small (maximum about 4 in my tests), so the > simple list I'm using instead should be ok. > > > We alse might need to make a special case for connections to proxies. > > Connections to proxies are also limited - with the same limit as for > normal hosts (4 atm). > > Please give it a try. If it works out ok we might even consider it > for the release. I have skimmed over hundreds of photos on flickr.com with both, vanilla dillo-2.1.1 and dillo-hg+connlimit2, with proxy and without. Browsing seems to work fine with all variations (I didn't try to analyze the packets), but I had a hard time trying to see the difference: I guess my internet connection is too fast and my X server too slow. Since I have a good internet connection, the images loaded faster without the patch, but sometimes the ones at the top of a page appeared sooner when the patch was applied. Since dillo and the servers seem to work with unlimited connections, perhaps defaulting to a limit of 6 wouldn't hurt? Dillo crashed twice in the beginning of my testing, but I am not sure anymore which build it was (because I've applied the gnutls-0.patch at first - https seemed to work). I couldn't reproduce the crashes afterwards. -- -- Rogut?s Sparnuotos

Jan. 23, 2010

connection limit (was: display: inline)

by Johannes.Hofmann＠gmx.de

Hi, I made some progress on the connection limit stuff. On Wed, Jan 20, 2010 at 02:43:17PM +0100, Johannes Hofmann wrote: > On Wed, Jan 20, 2010 at 08:59:12AM -0300, Jorge Arellano Cid wrote: > > > > BTW, is the connection-number-limit patch in the next release? > > No, it's not ready yet - unfortunately. We urgently need a connection > limit, but it needs to be per remote host, as otherwise dillo can > have all of it's 4 connections open to a slow server and then there > is no connection left for other servers, e.g. used for other windows > or tabs. The limit is now per remote host, so slow servers don't block connections to other (faster) ones. Unfortunately the hash table implementation from lout can't be used, as http.c is C file. However the number of hosts to which we have open connections at a time seems to be quite small (maximum about 4 in my tests), so the simple list I'm using instead should be ok. > We alse might need to make a special case for connections to proxies. Connections to proxies are also limited - with the same limit as for normal hosts (4 atm). Please give it a try. If it works out ok we might even consider it for the release. Cheers, Johannes

Jan. 21, 2010

experimental patch: gnutls for the https dpi

by newman.x＠gmail.com

On Wed, Jan 20, 2010 at 9:01 PM, corvid <corvid(a)lavabit.com> wrote: > Michal wrote: >> On Wed, Jan 20, 2010 at 10:52 AM, corvid <corvid(a)lavabit.com> wrote: >> > A few hours ago, I started to wonder how much trouble it would be to >> > make https.c understand gnutls instead of openssl. >> > >> > The answer is: If you borrow liberally from the public domain code in >> > ? ? ? ? ? ? ? the gnutls manual, then not very much! >> > >> > So here's a toy for anyone who would like one: >> > ?http://www.dillo.org/test/gnutls.0.patch >> >> What about NSS? >> >> https://fedoraproject.org/wiki/FedoraCryptoConsolidation >> >> It might bring even more (FIPS) compared to gnutls. > > Does gnutls lack it because it requires paying $$ and/or infinite > bureaucratic hurdles, or is it a technical matter? Possibility #1: definitely Possibility #2: FWIH, crypto part is easy, mostly eliminating "unsafe" algorithms and it's settings but there might be others not that easy stuff, like centralized cert management, ... > The NSS documentation seems to say that applications would no longer > have to know where a file full of certificates is because that would all > be taken care of by NSS in some centralized thing. Is that the case? I guess so https://fedoraproject.org/wiki/CryptoConsolidationEval#NSS > _That_ would definitely appeal to me. NSS seems to be the future of crypto API, unless it's too heavy-weight for project like Dillo, consider it instead of gnutls. > > _______________________________________________ > Dillo-dev mailing list > Dillo-dev(a)dillo.org > http://lists.auriga.wearlab.de/cgi-bin/mailman/listinfo/dillo-dev >

Jan. 21, 2010

dpi send cmd error msg (Re: [Dillo-dev] Re: even more cookies SECURITY)

by corvid＠lavabit.com

I wrote: > Jorge wrote: > > On Mon, Jan 04, 2010 at 11:38:03PM +0000, corvid wrote: > > > Do we currently have anything that needs to be > > > fixed/implemented before a release would be sensible? > > > > Not that I remember. The only problem I recall was the error > > message you reported with cookies and later seemed to be working > > somehow. > > a_Dpi_send_blocking_cmd() gets upset when > Dpi_blocking_read() doesn't get anything. > > So in the cookie dpi's srv_parse_tok(), > after the Cookies_set(), I tried putting a > a_Dpip_dsh_write_str(sh, 1, "text"), and > that makes the error msgs go away for me. What if we do the following:

Jan. 20, 2010