[we've had this thread for a long time outside dillo-dev, it has some interesting information so I'm switching to dillo-dev] Ferdi, On Thu, 4 Dec 2003, Ferdi Franceschini wrote:

Hi Jorge,

On Tue, Dec 02, 2003 at 09:03:08AM -0300, Jorge Arellano Cid wrote:

...

Hi Ferdi,

...

...

[...] Unfortunately the dpid is showing some problems with some https urls (also with some ftp ones). It sometimes ends in a loop of endless forking of dpi programs. The https dpi appears to be implemented as a filter type plugin but it is installed as https.dpi and not https.filter.dpi.

Oh!

Does https work now?

It does partly! (with the dpi-cache patch I'm developing). I'm planning to develop https.dpi as a dpi server so it can directly link against SSL lib or GNU TLS, instead of using wget. This ends the forking and allows for an easy implemetation of connection caching. BTW, https pages render OK with my tree, it's just the lack of reuse of the same connection what I suspect keeps it from working fully. I need to test a bit more too. Note: is there anybody here interested in working on the https dpi. I can quickly make the base (dpi server instead of filter) and then is a matter of knowing SSL or TLS and start implementing connection caching (no need to know about dillo internals).

[...]

...

The idea is interesting. Perhaps looking at the Volume III of Douglas Comer's book is well worth a try. I skimmed it yesterday and it has a lot of theory about this kind of servers. Do you have that volume? I'm affraid not, I'll see if I can hunt down a cheap copy.

Worth a read (although I prefer Richard Stevens' books).

...

Now that filter dpis can run in parallel, they're also showing the fork hog problem. I'm attaching an example page; drop it inside an ftp directory of your local machine and try loading it from dillo.

WARNING: don't do it on a server, it can bring down the machine! (I've had to switch-off mine to recover).

...

Dpid keeps on forking new instances of the ftp dpi until the machine is full of new processes. It looks like when the first child can't accept the connection, dpid regards the old request as a new one and forks another child. But dpid accepts the connection from dillo on behalf of filter dpis so any connections seen by dpid must be new requests.

I'll double check that. (It could be that the service FD keeps showing in the select call because it was connected, but never serviced...)

I set up an ftp server on my old 486 and tried out the test page. And it killed my Xserver! It also really upset inetd on my 486 which was running ftpd, inetd reported looping ftp connections and terminated the service. It also refused further connections for some time afterwards, however you can get things back to normal by restarting inetd.

I redirected the messages to a file and tried again, I found multiple occurences of the following message: <q> Dpi_get_server_uds_name:: /tmp/ferdi-khv5us/ftp.filter.dpi </q>

this suggests that dillo is actually making multiple requests. I also found the following error messages from dpid: <q> main.c:153: get_command: dpid tag is NULL : main.c:323: main: get_command failed </q>

I still don't know what's going wrong, I'll have another look at this in the next couple of days, I hope the above information helps.

I'll check it too.

...

Some time ago I started a download with a graphical dpi, and the downloader was open twice! This hints towards the presence of a race condition. What do you mean by graphical dpi?

It's an idea I've been preaching on for a long time. But nobody seems to believe until they see! :-) --Blame St. Tomás.

Is there a shiny new downloads server that I'm missing out on?

Yes! MUUUAAHAHAHAHAHAHAA!! It needs polishing but illustrates the idea pretty good. You'd be surprised at how simple it is. BTW, it was developed by a complete newcomer to dillo I was advicing. I'll send it to you and Sebastian (to illustrate part of a response I owe him about preferences). List: I'll review the latest version and decide whether to share it in its current state for polishing. The required knowledge to work on it is minimal: just GTK+ (the server part is already done). More on this later... Anyone interested?

...

a race condition: if the father continues running before its child accepts, the request may be served twice or more times. Only if the child terminates before it accepts. This is because dpid clears the downloader's socket from the socket_set which 'select' is watching before forking and the socket is then set again by dpid.c:handle_sigchld() when the child terminates.

Time to review. Regards Jorge.-