Hello, Sorry for the huge mail. :)
# Moving the dns stuff to IO. # Move all the socket stuff into IO.c, the module that wants to handle opening the connection should also close it (all non-internet sockets, like files. Unix sockets may be added to IO.c.). Gzilla had a flat design, we've come a long way to layer dillo and to separate its code in modules.
It's not what you think, what I meant was to move dns.c|h to the IO directory, not to move the code from those files into IO.c, I would never do that. What I want to do is to move the code into the modules where it belongs. I see no reason why the IO module shouldn't handle all the socket related stuff, currently http.c makes a new socket and connects it, then it passes it more or less to the IO module. It seems more logical and modular to me when the IO module does those TCP/IP specific things when http calls IO_new. Let the IO module do the IO, and the http module do the http related things, that's my opinion. I don't see the http module doing anything special with the dns info, so it probably makes more sense to let the IO module use the dns module too. You seperated the code in modules and layers, what I want to do now is to move code to the place it belongs, making Dillo in my opinion more modular, overall simpler and more flexible. Currently this has the effect that the IO modules becomes bigger and more complex, but it also means that the http module will be smaller and simpler, and that it's easier to add more http like modules in the future. What's bad about the current ccc functions? The problem is that currently not all modules can communicate blindly with other modules via ccc functions, because different ccc functions expect different parameters. Also most of the time modules know exactly with which module they talk, they have no choice, otherwise they don't know what parameters to give and expect. Now you use ccc functions like a sort of bad used void pointers: You know what you get, you know what you call, but still tunnel it through a typeless function. You mix a general API with module specific API's, making things only very confusing and unclear. All I want is a clear API. I see how ccc functions are usefull, and what their point is, but that is not my problem with them, the idea of a sort of linked list of functions is nice, but the implementation isn't very well if you ask me. Also the API used internally in Dillo isn't consistent. For example: Why call a_IO_new and not IO_ccc with OpStart? Things are unnecessary complicated and inflexible now. It's not easy to unravel the spaghetti, but it's doable, and when you're done you've a nice clean internal API that makes implementing library plugins a piece of cake.
It serves to isolate the query process from the answer process. Also as dillo uses pipes to talk to itself, it makes sense to have read-only and write-only FDs. Of course it can be merged, but it may be not simple.
With pipes it makes sense yes, but in general all other connections are bidirectional. Dillo uses mostly TCP, then it's one socket, one connection, one fd. And one IOData structure if you ask me. If you use one IOData structure that is also bidirectional, then all you need to do with pipes is to still use two IOData structures, but only for pipes. One IOData structure for each fd makes most sense to me. Connection caching. You say that Dillo has it, but when I open a website, and look with netstat, Dillo has no open connections. Same experience with debug output. Maybe Dillo has it with dpi connections, I don't know, didn't look at that code yet, but in general Dillo has no connection caching, at least not the kind I mean. The one I would want to have is that keeps connections open until the server closes them, or until they time out (I'm not sure if all connections should be cached, or only the one to the last domain/current site).
In the past that approach was used. ~/.dillo ended having a splash file for each release. The point is that what's more memory consuming is the widget tree for rendering, not the source itself.
I do not know (nor want to know ;) how you first implemented it, what I was thinking about is opening by default the about html file. To have something that does exactly the same as now, all you have to do is open by default e.g. /etc/dillo_splash.html. It takes about 5 lines, patch is written within 15 minutes, most work is getting rid of the old code. And instead of keeping that 10K splash info all the time in memory, you now don't use any memory at all for it, except the one to render it, but that's gone after going to another site (if we can tell the cache to not cache it anyway, but that's the same for the current implementation). Putting an about file integral into Dillo, that's what I consider being monolithic. There is nothing wrong with a distributed design, but my problem with you pushing everything into dpi plugins is that most of those things can be done in seperate modules, meaning that they can be library plugins as wel. It becomes scary when a simple "hello" dpi plugin is 150 lines of code, while the whole ssl support to IO.c is about the same (and much less when there is one IOData/fd). Now you get much more code that all does more or less the same, you may have kept the Dillo core clean and mean, but you get a ridiculous complicated and big plugin system, moving most of the code and functionality outside of Dillo (but in the same time making it twice as big). Now it's very tempting to make a dpi library that implements most used functions, as someone already discovered. What's the next step? Moving all the useful code from Dillo to that library so that even more code is shared I guess (and let Dillo use that library too). What you get is a weird attempt to make something that more or less resembles library plugins, but in, if you ask me, a very awkward way. Of course there are plugins that behave more like daemons, like the bookmark plugin, but then make it a real daemon, so that it can be used by even other browsers. With a flexible and clear internal API most plugins can be done with relative small and simple library plugins, and a few can be implemented as real daemons.
# Add ftp support. It has.
Using wget for something as basic as ftp in a plugin isn't something that I would call ftp support. Dillo is not a graphical front end for wget, thus I do not think Dillo has ftp support. Wget has.
(this is old problem of sharing a single resource among several clients).
An old problem that is solved long ago. Just use inter process communication, like file locks (you can unlock you know). Only what you need to add is an extra fstat to see if the file is edited, and if so, reload the cookies. No big deal if you ask me.
Yes I agree with that reusing dillo's http may come handy.
the details of handling the possible errors should be solved by
What your mistake is, in my humble opinion, is that you treat https like a seperate protocol, instead of http over another kind of connection. My view is that http is just http over ssl, and that ssl should be a sort of sub-module of IO (or a module that's only used by IO, like dns would be in my way). If you take a look at my patch, you'll see that the changes in http.c and IO.c are minimal, most of the code is at the bottom of IO.c, it could be easily a seperate file (again, only if there is one IOdata per fd, but then the ssl code will be 50 lines or so). Currently proxy support has more impact on http than https support, and you should treat them in a similar way if you ask me. You don't want a seperate plugin or module for proxy http, why do you want one for https? If you mean with monolithic that it's part of the core code, then yes, my way is monolithic, but it's only very little code, modular, and much less monolithic than an embedded splash page. the client application depending on the protocol and what it is being used for.
So having a generic SSL/TLS gateway may look interesting but it can get tricky to implement.
Not at all. The openssl api supports error callback functions, so the SSL/TLS support in IO can be minimal, if modules need more control they can use the callback functions. All that is needed is a general behaviour that is sufficient in 90% of the cases, and the current implementation does exactly that. Because modules have access to the SSL structure they can have all control they want, if needed. There are more or less 3 types of errors: 1: Connection related errors. (handshake_failure etc) Not much choice how to handle this, retry or abort (all connection related errors, including ssl related things like unsupported something). 2: Bad use of the API. (illegal_parameter) Should never happen with a good implementation. 3: Certificate related errors. (6 out of the 8 errors you listed) Most certificate related things can be done with openssl callback functions, so it's very seperate of the rest, making it easy to add or change later, or to handle it as module. Maybe there's something needed to give the module a chance to do it's certificate (startup) things and setting up the error handling, but not much. All in all IO doesn't have to worry much about all those SSL errors.
It puzzles me: changing the IO engine,
Only what will chance is that it does more what it should do, handling IO.
integrating parts of the code that were separated on purpose,
That was a misunderstanding, I wouldn't do that, sorry for not being clear enough.
changing the internal API, getting rid of the CCC
The current CCC API is not clear or flexible, nor simple. Maybe a more consistent CCC API is possible, maybe no CCC functions at all, I don't know yet, but I'm not simply getting rid of the CCC.
and porting to FLTK, we'd be talking about another browser.
Well, porting to FLTK, FLTK has similar aims as Dillo (small and fast), so maybe it was best if it was implemented with FLTK in the first place, but it's too late for that now. I would think twice before porting Dillo to FLTK (making sure that FLTK is alive and actively developed is one thing I would do), but GTK 2 is currently just scary, so FLTK as long term option isn't that bad. Getting libraries smaller is always hard, and GTK 2 is 12 mb now? I also saw the benchmarks, all in all not very encouraging. There is also an existing UTF8 Dillo patch, so maybe it's an option to stick with GTK 1 for a while. I didn't make Dillo, nor do I have much GUI programming experience, so I'm the last one you should listen to when it comes to what widget set to use. But from the networking side, FLTK has a perfect API, only a thin layer on top of select/poll, making it possible to easily add, remove and change the poll events on fds. GTK has an undocumented way of removing fds, can only hope it cleans the io channels up too and so on, but it still has no way to change the events on a fd. GTK has a lot bloat in it's io channel implementation (it's bloat when you don't need all it's features). I looked a bit in GTK's online API, and found the "perfect" replacement for io channels, a bunch of GDK functions that do almost exactly what Dillo needs: http://developer.gnome.org/doc/API/gdk/gdk-input.html It gives us almost the same API as FLTK does, but instead of being a thin layer, it's implemented with io channels, so the bloat is still there, but no way around that with GTK (not without very ugly complicated stuff). Also I think that the hairy bug isn't there, nor will we get another by GTK caused hairy bug if Dillo uses this GDK API, because it does what we want and need, without adding extra complexity like io channels (sure, the bug wasn't GTK's fault you can say, but if the API was clear and simpler, it wouldn't happen). Also FLTK uses internally the same select/poll as for our fds, X communicates through unix sockets, so whole FLTK blocks when nothing happens, while it seems that GTK polls 50 times/second for events or something like that. It's just that I prefer FLTK's way of doing some things. :)
Making a web site (or page for it) requires time, if you're willing to do that, go ahead but beware that we do not support unofficial patches so you'd have to answer all the incoming questions too.
Hmm, a web page, mind if I use the code of the splash screen as a basis? I only want to make an installer so that it's easy for people who, for instance, don't know what cvs is, to get a patched Dillo version. The goal is to have some sort of centralized patch place, with patches that work with eachother and are up to date. It requires time, true, but I'm willing to spend time on Dillo. The installer has no priority though, would rather spend my time to unite the IOData structures, switch to the GDK api, and clean up the https patch for now. All the things I want to have different in Dillo I'm willing to change myself. I'm not saying you should change this or that so or so, I ask if you mind if I change or add some things a certain way, and if you agree that it's better that way. Greetings, Indan (I wonder how many people wil read this 13K monster :)