October 2010 - Dillo-dev - lists.mailman3.com

A Dillo-Win32 update

by obeythepenguin＠gmail.com

On Sun, 03 Oct 2010 14:36:14 -0400, Johannes Hofmann <Johannes.Hofmann(a)gmx.de> wrote: > Do you plan to get the windows port into dillo mainline, or do you > want to maintain it as a patch? Getting it into Dillo mainline was always my intention. I do plan to maintain the SourceForge site for binary releases and other Windows-specific components, similar to GIMP for Windows. > If you plan to get it in, I would suggest to focus on getting in > what you have before doing other stuff. I'm just trying to look ahead, that's all. In a nutshell, I'm faced with some difficult technical hurdles, and somewhat a conflict of interests. On the one hand, I personally am not fond of DPI, and (as you've seen) question its necessity, particularly for such basic functionality as non-HTTP protocol support. On the other, my goal is (and has always been) to get my code into Dillo mainline, since I have neither the interest nor resources to maintain a full-blown fork. > That would mean: > > * splitting up your patch into smaller chunks, > each with a description of what it does and why it's needed. > It should be obvious for each patch that it does what you > described and only that. I tried to do that already. The problem is that this is a very complicated patch, and all the individual "subpatches" are highly interdependent on one another. In order to effectively split the patch, I would essentially have to re-code each change manually, which I currently lack the time to do. > * Avoid any changes to the code base that are not necessary for the > windows port (e.g. the include path changes in src/table.cc, or > the fix for the race mentioned in html.cc - if you think there is a > bug, this should be discussed on the list independent from the win > port. The same for the change in decode.c). I have not made a single change that is not necessary for the Windows port. The include path changes are necessary because MinGW does not accept them as written (relative to the top-level source directory). Regarding the race condition, that already *has* been discussed on the list, and the fix was specifically necessary for the Windows port: http://lists.auriga.wearlab.de/pipermail/dillo-dev/2010-August/007652.html > * This iowatch kludge is nasty - as you also state in the > comment. This must be fixed in some other way. Did you do some > research on the fltk lists about this? Is this solved in fltk-1.3? Yes, I did do research on the fltk lists -- that's how I discovered its necessity. To wit: http://www.mail-archive.com/fltk-bugs@easysw.com/msg01341.html Unfortunately, WSAAsyncSelect() doesn't appear to be the only problem. I patched the fltk source to disable USE_ASYNC_SELECT, to no avail. That's when I decided to implement the timeout kludge, in the interest of having *something* working, which I considered better than having nothing at all. I dislike the timeout kludge as well, since it's slow and extremely unstable, but as of yet have not found a workable alternative considering the current implementation. I haven't been able to dig further into fltk itself, due to lack of time and familiarity with the codebase (though I intend to someday, as there are some other things in there that need work, like popup menus on Windows 95). By the way, this is one reason I was interested in an alternate implementation using libcurl -- it would be much simpler than trying to implement low-level, cross-platform I/O ourselves. > * I don't like the DSOCK_NETINET_IN_H and similar defines to include > the right files on each platform. Can't we always include > dsock.h which in turn includes all necessary files? I wasn't sure whether or not to do that, but I erred on the side of selectively including files since it seemed the more conservative option. If you don't like it, feel free to change it. > Cheers, > Johannes Cheers, ~Benjamin

Oct. 3, 2010

A Dillo-Win32 update

by Johannes.Hofmann＠gmx.de

On Sat, Oct 02, 2010 at 03:55:32PM -0400, Benjamin Johnson wrote: > On Sat, 02 Oct 2010 15:27:30 -0400, Johannes Hofmann > <Johannes.Hofmann(a)gmx.de> wrote: > > >I like them and actually there is no way around it. One great > >feature of dillo is that you can fire up multiple processes very > >fast and still have access to the same bookmarks and cookies. > > (1) Can't that be implemented without a separate server process? Sure, you can use flock() or something like that. I think we even had this for cookies some time ago. But you get stale locks and all that. A separate process is the cleaner solution imho. > > (2) Is there any reason to run multiple Dillo processes, anyway? Yes. There are lots of reasons actually. Here are some: * You have separate caches. Stopping one process frees the cache and deletes the history of that process. * If one process crashes the other one continues. That's what google seems to do now with chrome. Dillo had this feature much earlier. * You could have multiple dillo instances on separate X11 sessions. > > (3) How does that affect other protocols (FTP, HTTPS, local file browsing?) ? > > >>>>On a related note, is there any reason not to cut DPI out of the > >>>>download system, i.e. just invoke a separate downloader process > >>>>directly? I'm considering implementing that at least as a temporary > >>>>solution, using my own self-contained download tool. > >>> > >>> The default downloader is a separate process. Before it, we > >>>used a call to wget from a separate process too. You're welcomed > >>>to hook your own preferred application, that's the idea of dpi. > >> > >>Well, I meant that more in the sense of "since the downloader's a > >>separate process anyway, why go through DPI at all?" > > > >Well, we have all the infrastucture for DPI already, why use a new > >mechanism? > > What's the advantage to doing it through DPI, besides that the > infrastructure's already there? See above. > > Despite its perceived redundancy, my mechanism is ultimately > simpler, since there's no interprocess communication involved. It's > also a bit more efficient, since it cuts out the middleman (dpid). I think the efficency argument doesn't count here. If you start a download, the single IPC to the DPI is certainly not an issue. > > I think, by the way, there *are* clear advantages to a separate > downloader: among other things, it allows downloads to continue > after the user exits the browser. On the other hand, I don't see > why the user would need cookies after the browser has exited. Again, see above for reasons to have multiple dillo processes running. > > >>>>Or is something more passed through than just a URL? > >>> > >>> Maybe the target directory&savename. I don't remember well... > >>>You can make the dpip functions log the messages to a file > >>>without much hassle, and see what actually happens. > >> > >>I wrote my own downloader, and patched Dillo so it calls my program > >>directly, bypassing DPI. (It also bypasses the Dillo file dialogs, > >>since my program displays its own.) So far everything seems to be > >>working nicely. > >> > >>I've only implemented the patch for Windows so far, and it needs a > >>bit of cleanup, but if you're curious you can see it here: > >>http://dillo-win32.sourceforge.net/files/dillo-r1705-dlgui-20100930.diff > >> > >>and the downloader itself: > >>http://dillo-win32.sourceforge.net/dlgui.html > >> > >>I'm tempted to port Dillo to libcurl now, since it works so nicely > >>in my own program, and seems like it would be much more portable > >>(and stable, on Windows) than the current implementation. It would > >>also allow for easier implementation of other protocols, like HTTPS > >>and FTP. Any thoughts? > > > >Hm, sounds like a massive change in a core component, but feel free to > >experiment. > > Well, I've already done one of those with dsock, so why not another? :-) Do you plan to get the windows port into dillo mainline, or do you want to maintain it as a patch? If you plan to get it in, I would suggest to focus on getting in what you have before doing other stuff. That would mean: * splitting up your patch into smaller chunks, each with a description of what it does and why it's needed. It should be obvious for each patch that it does what you described and only that. * Avoid any changes to the code base that are not necessary for the windows port (e.g. the include path changes in src/table.cc, or the fix for the race mentioned in html.cc - if you think there is a bug, this should be discussed on the list independent from the win port. The same for the change in decode.c). * This iowatch kludge is nasty - as you also state in the comment. This must be fixed in some other way. Did you do some research on the fltk lists about this? Is this solved in fltk-1.3? * I don't like the DSOCK_NETINET_IN_H and similar defines to include the right files on each platform. Can't we always include dsock.h which in turn includes all necessary files? Cheers, Johannes

Oct. 3, 2010

A Dillo-Win32 update

by obeythepenguin＠gmail.com

On Sat, 02 Oct 2010 17:31:07 -0400, Diego . <darkspirit5000(a)gmail.com> wrote: > Your solution only works for downloads. Your downloader can be made a > DPI and you can chose wich to use. Actually downloads do not stop if > dillo close. I'm well aware how DPI is implemented, and that downloads continue when Dillo is exited. My solution is intended in part as a temporary measure, since I need downloads to function regardless of whether DPI is available or not. I do have concerns, however, with both the specific implementation and general (perceived) necessity of DPI. > If the problems are dup2 and sockets dpid can be change to work on > windows. > Dillo always ask the dpi that serves a service before conect to it. > dpid run the DPI so can send dillo the right fd, socket, file or do > other thing. I've been working on that, but with very limited success. dup2 and sockets are the most immediate problem, but there are larger problems beyond that (for example, packaging Dillo/DPI). > dpid avoid to load in every dillo code to handle DPIs or external > programs that will use only one and avoid the synchronize thing. I don't think basic functionality like local file browsing and HTTPS is unnecessary "load in every dillo code". If there's any situation where it's undesired, perhaps the code could be #ifdef'd out and a new autoconf option added (--disable-local-files, etc.) As far as sharing bookmarks, cookies, etc. between browser processes: most major browsers (IE, Firefox, Opera, probably Chrome) allow one browser process running, and if a second is started, it signals the first to take some action (create a new window, for instance). That would still require some amount of IPC, but it's less complicated than DPI, and certainly less wasteful since you don't have multiple instances loading the same UI, static libraries, etc. ~Benjamin

Oct. 3, 2010

A Dillo-Win32 update

by obeythepenguin＠gmail.com

On Sat, 02 Oct 2010 15:27:30 -0400, Johannes Hofmann <Johannes.Hofmann(a)gmx.de> wrote: > I like them and actually there is no way around it. One great > feature of dillo is that you can fire up multiple processes very > fast and still have access to the same bookmarks and cookies. (1) Can't that be implemented without a separate server process? (2) Is there any reason to run multiple Dillo processes, anyway? (3) How does that affect other protocols (FTP, HTTPS, local file browsing?) >> >>On a related note, is there any reason not to cut DPI out of the >> >>download system, i.e. just invoke a separate downloader process >> >>directly? I'm considering implementing that at least as a temporary >> >>solution, using my own self-contained download tool. >> > >> > The default downloader is a separate process. Before it, we >> >used a call to wget from a separate process too. You're welcomed >> >to hook your own preferred application, that's the idea of dpi. >> >> Well, I meant that more in the sense of "since the downloader's a >> separate process anyway, why go through DPI at all?" > > Well, we have all the infrastucture for DPI already, why use a new > mechanism? What's the advantage to doing it through DPI, besides that the infrastructure's already there? Despite its perceived redundancy, my mechanism is ultimately simpler, since there's no interprocess communication involved. It's also a bit more efficient, since it cuts out the middleman (dpid). I think, by the way, there *are* clear advantages to a separate downloader: among other things, it allows downloads to continue after the user exits the browser. On the other hand, I don't see why the user would need cookies after the browser has exited. >> >>Or is something more passed through than just a URL? >> > >> > Maybe the target directory&savename. I don't remember well... >> >You can make the dpip functions log the messages to a file >> >without much hassle, and see what actually happens. >> >> I wrote my own downloader, and patched Dillo so it calls my program >> directly, bypassing DPI. (It also bypasses the Dillo file dialogs, >> since my program displays its own.) So far everything seems to be >> working nicely. >> >> I've only implemented the patch for Windows so far, and it needs a >> bit of cleanup, but if you're curious you can see it here: >> http://dillo-win32.sourceforge.net/files/dillo-r1705-dlgui-20100930.diff >> >> and the downloader itself: >> http://dillo-win32.sourceforge.net/dlgui.html >> >> I'm tempted to port Dillo to libcurl now, since it works so nicely >> in my own program, and seems like it would be much more portable >> (and stable, on Windows) than the current implementation. It would >> also allow for easier implementation of other protocols, like HTTPS >> and FTP. Any thoughts? > > Hm, sounds like a massive change in a core component, but feel free to > experiment. Well, I've already done one of those with dsock, so why not another? :-) ~Benjamin

Oct. 2, 2010

A Dillo-Win32 update

by Johannes.Hofmann＠gmx.de

Hi Benjamin, On Sat, Oct 02, 2010 at 02:48:06PM -0400, Benjamin Johnson wrote: > I meant to reply sooner, and got sidetracked with other things... > > On Thu, 30 Sep 2010 07:28:39 -0400, Jorge Arellano Cid > <jcid(a)dillo.org> wrote: > > >On Wed, Sep 29, 2010 at 10:57:30PM -0400, Benjamin Johnson wrote: > >>On Wed, 29 Sep 2010 21:22:43 -0400, Jorge Arellano Cid > >><jcid(a)dillo.org> wrote: > >> > >>> The dpi framework works with isolated processes. A third option > >>>could be to make an alternative implementation that works in Win. > >>>For instance, is there a dbus port running in Win*? > >>> > >> > >>There is, although I have a couple reservations. I'm not sure how > >>cleanly it could be packaged, or whether it would run on Windows 9x > >>(one of my goals is to support all the way back to Windows 95). I > >>also have some personal dislike for it, after dealing with much > >>dbus-related breakage in my Linux development days. > >> > >>I do appreciate the suggestion, though, and if you think it could > >>work, I will certainly look into it. > > > > I haven't used dbus, but it's used in GNU/Linux by some big > >players so I guess it stabilized [1]. Any inter process message- > >passing lib will work, but implementing it well is a hard task, > >that's why I suggested dbus (it has undergone lots of testing). > >Probably there's a better candidate in the embedded area. > > > >[1] http://www.freedesktop.org/wiki/Software/DbusProjects > > My problem with dbus isn't so much stability, as complexity. It's > been a while since I've run Linux, but I vaguely remember dbus's > internal security/permissions were a nightmare. I also vote against dbus. It would be a new dependency and the size of the dbus tarball is currently more than twice the size of dillo-2.2.tar.bz2! > > I'll admit, I'm not the biggest fan of DPI anyway -- relying on > complex interprocess communication even for such simple things as > local file browsing seems like more trouble than it's worth. It's > better than the other way around, re-implementing operating system > functionality inside the browser (as a certain search giant has), > but I'm not yet convinced the advantages truly outweigh the > implementation hassles. I like them and actually there is no way around it. One great feature of dillo is that you can fire up multiple processes very fast and still have access to the same bookmarks and cookies. > > >>On a related note, is there any reason not to cut DPI out of the > >>download system, i.e. just invoke a separate downloader process > >>directly? I'm considering implementing that at least as a temporary > >>solution, using my own self-contained download tool. > > > > The default downloader is a separate process. Before it, we > >used a call to wget from a separate process too. You're welcomed > >to hook your own preferred application, that's the idea of dpi. > > Well, I meant that more in the sense of "since the downloader's a > separate process anyway, why go through DPI at all?" Well, we have all the infrastucture for DPI already, why use a new mechanism? > > >>Or is something more passed through than just a URL? > > > > Maybe the target directory&savename. I don't remember well... > >You can make the dpip functions log the messages to a file > >without much hassle, and see what actually happens. > > I wrote my own downloader, and patched Dillo so it calls my program > directly, bypassing DPI. (It also bypasses the Dillo file dialogs, > since my program displays its own.) So far everything seems to be > working nicely. > > I've only implemented the patch for Windows so far, and it needs a > bit of cleanup, but if you're curious you can see it here: > http://dillo-win32.sourceforge.net/files/dillo-r1705-dlgui-20100930.diff > > and the downloader itself: > http://dillo-win32.sourceforge.net/dlgui.html > > I'm tempted to port Dillo to libcurl now, since it works so nicely > in my own program, and seems like it would be much more portable > (and stable, on Windows) than the current implementation. It would > also allow for easier implementation of other protocols, like HTTPS > and FTP. Any thoughts? Hm, sounds like a massive change in a core component, but feel free to experiment. Cheers, Johannes

Oct. 2, 2010

A Dillo-Win32 update

by obeythepenguin＠gmail.com

I meant to reply sooner, and got sidetracked with other things... On Thu, 30 Sep 2010 07:28:39 -0400, Jorge Arellano Cid <jcid(a)dillo.org> wrote: > On Wed, Sep 29, 2010 at 10:57:30PM -0400, Benjamin Johnson wrote: >> On Wed, 29 Sep 2010 21:22:43 -0400, Jorge Arellano Cid >> <jcid(a)dillo.org> wrote: >> >> > The dpi framework works with isolated processes. A third option >> >could be to make an alternative implementation that works in Win. >> >For instance, is there a dbus port running in Win*? >> > >> >> There is, although I have a couple reservations. I'm not sure how >> cleanly it could be packaged, or whether it would run on Windows 9x >> (one of my goals is to support all the way back to Windows 95). I >> also have some personal dislike for it, after dealing with much >> dbus-related breakage in my Linux development days. >> >> I do appreciate the suggestion, though, and if you think it could >> work, I will certainly look into it. > > I haven't used dbus, but it's used in GNU/Linux by some big > players so I guess it stabilized [1]. Any inter process message- > passing lib will work, but implementing it well is a hard task, > that's why I suggested dbus (it has undergone lots of testing). > Probably there's a better candidate in the embedded area. > > [1] http://www.freedesktop.org/wiki/Software/DbusProjects My problem with dbus isn't so much stability, as complexity. It's been a while since I've run Linux, but I vaguely remember dbus's internal security/permissions were a nightmare. I'll admit, I'm not the biggest fan of DPI anyway -- relying on complex interprocess communication even for such simple things as local file browsing seems like more trouble than it's worth. It's better than the other way around, re-implementing operating system functionality inside the browser (as a certain search giant has), but I'm not yet convinced the advantages truly outweigh the implementation hassles. >> On a related note, is there any reason not to cut DPI out of the >> download system, i.e. just invoke a separate downloader process >> directly? I'm considering implementing that at least as a temporary >> solution, using my own self-contained download tool. > > The default downloader is a separate process. Before it, we > used a call to wget from a separate process too. You're welcomed > to hook your own preferred application, that's the idea of dpi. Well, I meant that more in the sense of "since the downloader's a separate process anyway, why go through DPI at all?" >> Or is something more passed through than just a URL? > > Maybe the target directory&savename. I don't remember well... > You can make the dpip functions log the messages to a file > without much hassle, and see what actually happens. I wrote my own downloader, and patched Dillo so it calls my program directly, bypassing DPI. (It also bypasses the Dillo file dialogs, since my program displays its own.) So far everything seems to be working nicely. I've only implemented the patch for Windows so far, and it needs a bit of cleanup, but if you're curious you can see it here: http://dillo-win32.sourceforge.net/files/dillo-r1705-dlgui-20100930.diff and the downloader itself: http://dillo-win32.sourceforge.net/dlgui.html I'm tempted to port Dillo to libcurl now, since it works so nicely in my own program, and seems like it would be much more portable (and stable, on Windows) than the current implementation. It would also allow for easier implementation of other protocols, like HTTPS and FTP. Any thoughts? ~Benjamin

Oct. 2, 2010