The browser patch is about as stable as I can get it, so I thought I'd send it in for your consideration. First the patch, then my comments: http://dillo-win32.sourceforge.net/files/dillo-r1705-win32-20100913-merge.di... I'd consider this "merge-ready" in terms of completeness, though I haven't much tested stability, and I'd need your opinion on coding style. This is a long patch (~1740 lines), with a lot of separate but closely-interconnected changes. I'd split it into smaller subpatches, but I'm not really sure how practical (or possible) that would be, so my apologies in advance for the length. I've run into a major impasse with DPI. Simply put, DPI is not portable to non-Unix systems. Specifically, there are two main problems as far as Windows is concerned: (1) lack of Unix signals, and (2) non-interchangeability of file and socket descriptors. Since I'm aiming for a native port, an emulation layer such as Cygwin is absolutely not an option. I'm faced with a bit of a dilemma here. The two options I see are to either redesign DPI, or ditch it altogether; and I don't think anyone is too keen on either idea. I've considered a fork, but I'm opposed to the idea on both practical and philosophical grounds (my goal's to help, not to hurt upstream). Any suggestions? Thanks & cheers, ~Benjamin
On Wed, Sep 29, 2010 at 04:51:27PM -0400, Benjamin Johnson wrote:
The browser patch is about as stable as I can get it, so I thought I'd send it in for your consideration. First the patch, then my comments:
http://dillo-win32.sourceforge.net/files/dillo-r1705-win32-20100913-merge.di...
I'd consider this "merge-ready" in terms of completeness, though I haven't much tested stability, and I'd need your opinion on coding style. This is a long patch (~1740 lines), with a lot of separate but closely-interconnected changes. I'd split it into smaller subpatches, but I'm not really sure how practical (or possible) that would be, so my apologies in advance for the length.
I've run into a major impasse with DPI. Simply put, DPI is not portable to non-Unix systems. Specifically, there are two main problems as far as Windows is concerned: (1) lack of Unix signals, and (2) non-interchangeability of file and socket descriptors. Since I'm aiming for a native port, an emulation layer such as Cygwin is absolutely not an option.
I'm faced with a bit of a dilemma here. The two options I see are to either redesign DPI, or ditch it altogether; and I don't think anyone is too keen on either idea. I've considered a fork, but I'm opposed to the idea on both practical and philosophical grounds (my goal's to help, not to hurt upstream). Any suggestions?
The dpi framework works with isolated processes. A third option could be to make an alternative implementation that works in Win. For instance, is there a dbus port running in Win*? -- Cheers Jorge.-
On Wed, 29 Sep 2010 21:22:43 -0400, Jorge Arellano Cid <jcid@dillo.org> wrote:
The dpi framework works with isolated processes. A third option could be to make an alternative implementation that works in Win. For instance, is there a dbus port running in Win*?
There is, although I have a couple reservations. I'm not sure how cleanly it could be packaged, or whether it would run on Windows 9x (one of my goals is to support all the way back to Windows 95). I also have some personal dislike for it, after dealing with much dbus-related breakage in my Linux development days. I do appreciate the suggestion, though, and if you think it could work, I will certainly look into it. On a related note, is there any reason not to cut DPI out of the download system, i.e. just invoke a separate downloader process directly? I'm considering implementing that at least as a temporary solution, using my own self-contained download tool. Or is something more passed through than just a URL? Thanks again, ~Benjamin
On Wed, Sep 29, 2010 at 10:57:30PM -0400, Benjamin Johnson wrote:
On Wed, 29 Sep 2010 21:22:43 -0400, Jorge Arellano Cid <jcid@dillo.org> wrote:
The dpi framework works with isolated processes. A third option could be to make an alternative implementation that works in Win. For instance, is there a dbus port running in Win*?
There is, although I have a couple reservations. I'm not sure how cleanly it could be packaged, or whether it would run on Windows 9x (one of my goals is to support all the way back to Windows 95). I also have some personal dislike for it, after dealing with much dbus-related breakage in my Linux development days.
I do appreciate the suggestion, though, and if you think it could work, I will certainly look into it.
I haven't used dbus, but it's used in GNU/Linux by some big players so I guess it stabilized [1]. Any inter process message- passing lib will work, but implementing it well is a hard task, that's why I suggested dbus (it has undergone lots of testing). Probably there's a better candidate in the embedded area. [1] http://www.freedesktop.org/wiki/Software/DbusProjects
On a related note, is there any reason not to cut DPI out of the download system, i.e. just invoke a separate downloader process directly? I'm considering implementing that at least as a temporary solution, using my own self-contained download tool.
The default downloader is a separate process. Before it, we used a call to wget from a separate process too. You're welcomed to hook your own preferred application, that's the idea of dpi.
Or is something more passed through than just a URL?
Maybe the target directory&savename. I don't remember well... You can make the dpip functions log the messages to a file without much hassle, and see what actually happens. -- Cheers Jorge.-
I meant to reply sooner, and got sidetracked with other things... On Thu, 30 Sep 2010 07:28:39 -0400, Jorge Arellano Cid <jcid@dillo.org> wrote:
On Wed, Sep 29, 2010 at 10:57:30PM -0400, Benjamin Johnson wrote:
On Wed, 29 Sep 2010 21:22:43 -0400, Jorge Arellano Cid <jcid@dillo.org> wrote:
The dpi framework works with isolated processes. A third option could be to make an alternative implementation that works in Win. For instance, is there a dbus port running in Win*?
There is, although I have a couple reservations. I'm not sure how cleanly it could be packaged, or whether it would run on Windows 9x (one of my goals is to support all the way back to Windows 95). I also have some personal dislike for it, after dealing with much dbus-related breakage in my Linux development days.
I do appreciate the suggestion, though, and if you think it could work, I will certainly look into it.
I haven't used dbus, but it's used in GNU/Linux by some big players so I guess it stabilized [1]. Any inter process message- passing lib will work, but implementing it well is a hard task, that's why I suggested dbus (it has undergone lots of testing). Probably there's a better candidate in the embedded area.
My problem with dbus isn't so much stability, as complexity. It's been a while since I've run Linux, but I vaguely remember dbus's internal security/permissions were a nightmare. I'll admit, I'm not the biggest fan of DPI anyway -- relying on complex interprocess communication even for such simple things as local file browsing seems like more trouble than it's worth. It's better than the other way around, re-implementing operating system functionality inside the browser (as a certain search giant has), but I'm not yet convinced the advantages truly outweigh the implementation hassles.
On a related note, is there any reason not to cut DPI out of the download system, i.e. just invoke a separate downloader process directly? I'm considering implementing that at least as a temporary solution, using my own self-contained download tool.
The default downloader is a separate process. Before it, we used a call to wget from a separate process too. You're welcomed to hook your own preferred application, that's the idea of dpi.
Well, I meant that more in the sense of "since the downloader's a separate process anyway, why go through DPI at all?"
Or is something more passed through than just a URL?
Maybe the target directory&savename. I don't remember well... You can make the dpip functions log the messages to a file without much hassle, and see what actually happens.
I wrote my own downloader, and patched Dillo so it calls my program directly, bypassing DPI. (It also bypasses the Dillo file dialogs, since my program displays its own.) So far everything seems to be working nicely. I've only implemented the patch for Windows so far, and it needs a bit of cleanup, but if you're curious you can see it here: http://dillo-win32.sourceforge.net/files/dillo-r1705-dlgui-20100930.diff and the downloader itself: http://dillo-win32.sourceforge.net/dlgui.html I'm tempted to port Dillo to libcurl now, since it works so nicely in my own program, and seems like it would be much more portable (and stable, on Windows) than the current implementation. It would also allow for easier implementation of other protocols, like HTTPS and FTP. Any thoughts? ~Benjamin
Hi Benjamin, On Sat, Oct 02, 2010 at 02:48:06PM -0400, Benjamin Johnson wrote:
I meant to reply sooner, and got sidetracked with other things...
On Thu, 30 Sep 2010 07:28:39 -0400, Jorge Arellano Cid <jcid@dillo.org> wrote:
On Wed, Sep 29, 2010 at 10:57:30PM -0400, Benjamin Johnson wrote:
On Wed, 29 Sep 2010 21:22:43 -0400, Jorge Arellano Cid <jcid@dillo.org> wrote:
The dpi framework works with isolated processes. A third option could be to make an alternative implementation that works in Win. For instance, is there a dbus port running in Win*?
There is, although I have a couple reservations. I'm not sure how cleanly it could be packaged, or whether it would run on Windows 9x (one of my goals is to support all the way back to Windows 95). I also have some personal dislike for it, after dealing with much dbus-related breakage in my Linux development days.
I do appreciate the suggestion, though, and if you think it could work, I will certainly look into it.
I haven't used dbus, but it's used in GNU/Linux by some big players so I guess it stabilized [1]. Any inter process message- passing lib will work, but implementing it well is a hard task, that's why I suggested dbus (it has undergone lots of testing). Probably there's a better candidate in the embedded area.
My problem with dbus isn't so much stability, as complexity. It's been a while since I've run Linux, but I vaguely remember dbus's internal security/permissions were a nightmare.
I also vote against dbus. It would be a new dependency and the size of the dbus tarball is currently more than twice the size of dillo-2.2.tar.bz2!
I'll admit, I'm not the biggest fan of DPI anyway -- relying on complex interprocess communication even for such simple things as local file browsing seems like more trouble than it's worth. It's better than the other way around, re-implementing operating system functionality inside the browser (as a certain search giant has), but I'm not yet convinced the advantages truly outweigh the implementation hassles.
I like them and actually there is no way around it. One great feature of dillo is that you can fire up multiple processes very fast and still have access to the same bookmarks and cookies.
On a related note, is there any reason not to cut DPI out of the download system, i.e. just invoke a separate downloader process directly? I'm considering implementing that at least as a temporary solution, using my own self-contained download tool.
The default downloader is a separate process. Before it, we used a call to wget from a separate process too. You're welcomed to hook your own preferred application, that's the idea of dpi.
Well, I meant that more in the sense of "since the downloader's a separate process anyway, why go through DPI at all?"
Well, we have all the infrastucture for DPI already, why use a new mechanism?
Or is something more passed through than just a URL?
Maybe the target directory&savename. I don't remember well... You can make the dpip functions log the messages to a file without much hassle, and see what actually happens.
I wrote my own downloader, and patched Dillo so it calls my program directly, bypassing DPI. (It also bypasses the Dillo file dialogs, since my program displays its own.) So far everything seems to be working nicely.
I've only implemented the patch for Windows so far, and it needs a bit of cleanup, but if you're curious you can see it here: http://dillo-win32.sourceforge.net/files/dillo-r1705-dlgui-20100930.diff
and the downloader itself: http://dillo-win32.sourceforge.net/dlgui.html
I'm tempted to port Dillo to libcurl now, since it works so nicely in my own program, and seems like it would be much more portable (and stable, on Windows) than the current implementation. It would also allow for easier implementation of other protocols, like HTTPS and FTP. Any thoughts?
Hm, sounds like a massive change in a core component, but feel free to experiment. Cheers, Johannes
On Sat, 02 Oct 2010 15:27:30 -0400, Johannes Hofmann <Johannes.Hofmann@gmx.de> wrote:
I like them and actually there is no way around it. One great feature of dillo is that you can fire up multiple processes very fast and still have access to the same bookmarks and cookies.
(1) Can't that be implemented without a separate server process? (2) Is there any reason to run multiple Dillo processes, anyway? (3) How does that affect other protocols (FTP, HTTPS, local file browsing?)
On a related note, is there any reason not to cut DPI out of the download system, i.e. just invoke a separate downloader process directly? I'm considering implementing that at least as a temporary solution, using my own self-contained download tool.
The default downloader is a separate process. Before it, we used a call to wget from a separate process too. You're welcomed to hook your own preferred application, that's the idea of dpi.
Well, I meant that more in the sense of "since the downloader's a separate process anyway, why go through DPI at all?"
Well, we have all the infrastucture for DPI already, why use a new mechanism?
What's the advantage to doing it through DPI, besides that the infrastructure's already there? Despite its perceived redundancy, my mechanism is ultimately simpler, since there's no interprocess communication involved. It's also a bit more efficient, since it cuts out the middleman (dpid). I think, by the way, there *are* clear advantages to a separate downloader: among other things, it allows downloads to continue after the user exits the browser. On the other hand, I don't see why the user would need cookies after the browser has exited.
Or is something more passed through than just a URL?
Maybe the target directory&savename. I don't remember well... You can make the dpip functions log the messages to a file without much hassle, and see what actually happens.
I wrote my own downloader, and patched Dillo so it calls my program directly, bypassing DPI. (It also bypasses the Dillo file dialogs, since my program displays its own.) So far everything seems to be working nicely.
I've only implemented the patch for Windows so far, and it needs a bit of cleanup, but if you're curious you can see it here: http://dillo-win32.sourceforge.net/files/dillo-r1705-dlgui-20100930.diff
and the downloader itself: http://dillo-win32.sourceforge.net/dlgui.html
I'm tempted to port Dillo to libcurl now, since it works so nicely in my own program, and seems like it would be much more portable (and stable, on Windows) than the current implementation. It would also allow for easier implementation of other protocols, like HTTPS and FTP. Any thoughts?
Hm, sounds like a massive change in a core component, but feel free to experiment.
Well, I've already done one of those with dsock, so why not another? :-) ~Benjamin
On Sat, Oct 02, 2010 at 03:55:32PM -0400, Benjamin Johnson wrote:
On Sat, 02 Oct 2010 15:27:30 -0400, Johannes Hofmann <Johannes.Hofmann@gmx.de> wrote:
I like them and actually there is no way around it. One great feature of dillo is that you can fire up multiple processes very fast and still have access to the same bookmarks and cookies.
(1) Can't that be implemented without a separate server process?
Sure, you can use flock() or something like that. I think we even had this for cookies some time ago. But you get stale locks and all that. A separate process is the cleaner solution imho.
(2) Is there any reason to run multiple Dillo processes, anyway?
Yes. There are lots of reasons actually. Here are some: * You have separate caches. Stopping one process frees the cache and deletes the history of that process. * If one process crashes the other one continues. That's what google seems to do now with chrome. Dillo had this feature much earlier. * You could have multiple dillo instances on separate X11 sessions.
(3) How does that affect other protocols (FTP, HTTPS, local file browsing?)
?
On a related note, is there any reason not to cut DPI out of the download system, i.e. just invoke a separate downloader process directly? I'm considering implementing that at least as a temporary solution, using my own self-contained download tool.
The default downloader is a separate process. Before it, we used a call to wget from a separate process too. You're welcomed to hook your own preferred application, that's the idea of dpi.
Well, I meant that more in the sense of "since the downloader's a separate process anyway, why go through DPI at all?"
Well, we have all the infrastucture for DPI already, why use a new mechanism?
What's the advantage to doing it through DPI, besides that the infrastructure's already there?
See above.
Despite its perceived redundancy, my mechanism is ultimately simpler, since there's no interprocess communication involved. It's also a bit more efficient, since it cuts out the middleman (dpid).
I think the efficency argument doesn't count here. If you start a download, the single IPC to the DPI is certainly not an issue.
I think, by the way, there *are* clear advantages to a separate downloader: among other things, it allows downloads to continue after the user exits the browser. On the other hand, I don't see why the user would need cookies after the browser has exited.
Again, see above for reasons to have multiple dillo processes running.
Or is something more passed through than just a URL?
Maybe the target directory&savename. I don't remember well... You can make the dpip functions log the messages to a file without much hassle, and see what actually happens.
I wrote my own downloader, and patched Dillo so it calls my program directly, bypassing DPI. (It also bypasses the Dillo file dialogs, since my program displays its own.) So far everything seems to be working nicely.
I've only implemented the patch for Windows so far, and it needs a bit of cleanup, but if you're curious you can see it here: http://dillo-win32.sourceforge.net/files/dillo-r1705-dlgui-20100930.diff
and the downloader itself: http://dillo-win32.sourceforge.net/dlgui.html
I'm tempted to port Dillo to libcurl now, since it works so nicely in my own program, and seems like it would be much more portable (and stable, on Windows) than the current implementation. It would also allow for easier implementation of other protocols, like HTTPS and FTP. Any thoughts?
Hm, sounds like a massive change in a core component, but feel free to experiment.
Well, I've already done one of those with dsock, so why not another? :-)
Do you plan to get the windows port into dillo mainline, or do you want to maintain it as a patch? If you plan to get it in, I would suggest to focus on getting in what you have before doing other stuff. That would mean: * splitting up your patch into smaller chunks, each with a description of what it does and why it's needed. It should be obvious for each patch that it does what you described and only that. * Avoid any changes to the code base that are not necessary for the windows port (e.g. the include path changes in src/table.cc, or the fix for the race mentioned in html.cc - if you think there is a bug, this should be discussed on the list independent from the win port. The same for the change in decode.c). * This iowatch kludge is nasty - as you also state in the comment. This must be fixed in some other way. Did you do some research on the fltk lists about this? Is this solved in fltk-1.3? * I don't like the DSOCK_NETINET_IN_H and similar defines to include the right files on each platform. Can't we always include dsock.h which in turn includes all necessary files? Cheers, Johannes
On Sun, 03 Oct 2010 14:36:14 -0400, Johannes Hofmann <Johannes.Hofmann@gmx.de> wrote:
Do you plan to get the windows port into dillo mainline, or do you want to maintain it as a patch?
Getting it into Dillo mainline was always my intention. I do plan to maintain the SourceForge site for binary releases and other Windows-specific components, similar to GIMP for Windows.
If you plan to get it in, I would suggest to focus on getting in what you have before doing other stuff.
I'm just trying to look ahead, that's all. In a nutshell, I'm faced with some difficult technical hurdles, and somewhat a conflict of interests. On the one hand, I personally am not fond of DPI, and (as you've seen) question its necessity, particularly for such basic functionality as non-HTTP protocol support. On the other, my goal is (and has always been) to get my code into Dillo mainline, since I have neither the interest nor resources to maintain a full-blown fork.
That would mean:
* splitting up your patch into smaller chunks, each with a description of what it does and why it's needed. It should be obvious for each patch that it does what you described and only that.
I tried to do that already. The problem is that this is a very complicated patch, and all the individual "subpatches" are highly interdependent on one another. In order to effectively split the patch, I would essentially have to re-code each change manually, which I currently lack the time to do.
* Avoid any changes to the code base that are not necessary for the windows port (e.g. the include path changes in src/table.cc, or the fix for the race mentioned in html.cc - if you think there is a bug, this should be discussed on the list independent from the win port. The same for the change in decode.c).
I have not made a single change that is not necessary for the Windows port. The include path changes are necessary because MinGW does not accept them as written (relative to the top-level source directory). Regarding the race condition, that already *has* been discussed on the list, and the fix was specifically necessary for the Windows port: http://lists.auriga.wearlab.de/pipermail/dillo-dev/2010-August/007652.html
* This iowatch kludge is nasty - as you also state in the comment. This must be fixed in some other way. Did you do some research on the fltk lists about this? Is this solved in fltk-1.3?
Yes, I did do research on the fltk lists -- that's how I discovered its necessity. To wit: http://www.mail-archive.com/fltk-bugs@easysw.com/msg01341.html Unfortunately, WSAAsyncSelect() doesn't appear to be the only problem. I patched the fltk source to disable USE_ASYNC_SELECT, to no avail. That's when I decided to implement the timeout kludge, in the interest of having *something* working, which I considered better than having nothing at all. I dislike the timeout kludge as well, since it's slow and extremely unstable, but as of yet have not found a workable alternative considering the current implementation. I haven't been able to dig further into fltk itself, due to lack of time and familiarity with the codebase (though I intend to someday, as there are some other things in there that need work, like popup menus on Windows 95). By the way, this is one reason I was interested in an alternate implementation using libcurl -- it would be much simpler than trying to implement low-level, cross-platform I/O ourselves.
* I don't like the DSOCK_NETINET_IN_H and similar defines to include the right files on each platform. Can't we always include dsock.h which in turn includes all necessary files?
I wasn't sure whether or not to do that, but I erred on the side of selectively including files since it seemed the more conservative option. If you don't like it, feel free to change it.
Cheers, Johannes
Cheers, ~Benjamin
On Sun, Oct 03, 2010 at 03:13:51PM -0400, Benjamin Johnson wrote:
On Sun, 03 Oct 2010 14:36:14 -0400, Johannes Hofmann <Johannes.Hofmann@gmx.de> wrote:
Do you plan to get the windows port into dillo mainline, or do you want to maintain it as a patch?
Getting it into Dillo mainline was always my intention. I do plan to maintain the SourceForge site for binary releases and other Windows-specific components, similar to GIMP for Windows.
If you plan to get it in, I would suggest to focus on getting in what you have before doing other stuff.
I'm just trying to look ahead, that's all.
In a nutshell, I'm faced with some difficult technical hurdles, and somewhat a conflict of interests. On the one hand, I personally am not fond of DPI, and (as you've seen) question its necessity, particularly for such basic functionality as non-HTTP protocol support. On the other, my goal is (and has always been) to get my code into Dillo mainline, since I have neither the interest nor resources to maintain a full-blown fork.
That would mean:
* splitting up your patch into smaller chunks, each with a description of what it does and why it's needed. It should be obvious for each patch that it does what you described and only that.
I tried to do that already. The problem is that this is a very complicated patch, and all the individual "subpatches" are highly interdependent on one another. In order to effectively split the patch, I would essentially have to re-code each change manually, which I currently lack the time to do.
That's fine, but I fear nobody else will do the work for you.
* Avoid any changes to the code base that are not necessary for the windows port (e.g. the include path changes in src/table.cc, or the fix for the race mentioned in html.cc - if you think there is a bug, this should be discussed on the list independent from the win port. The same for the change in decode.c).
I have not made a single change that is not necessary for the Windows port. The include path changes are necessary because MinGW does not accept them as written (relative to the top-level source directory). Regarding the race condition, that already *has* been discussed on the list, and the fix was specifically necessary for the Windows port:
http://lists.auriga.wearlab.de/pipermail/dillo-dev/2010-August/007652.html
I have missed that. But I still fail to see why we should need to wait until there is no more connection open - that could take ages.
* This iowatch kludge is nasty - as you also state in the comment. This must be fixed in some other way. Did you do some research on the fltk lists about this? Is this solved in fltk-1.3?
Yes, I did do research on the fltk lists -- that's how I discovered its necessity. To wit:
http://www.mail-archive.com/fltk-bugs@easysw.com/msg01341.html
Unfortunately, WSAAsyncSelect() doesn't appear to be the only problem. I patched the fltk source to disable USE_ASYNC_SELECT, to no avail. That's when I decided to implement the timeout kludge, in the interest of having *something* working, which I considered better than having nothing at all.
The bug is marked as resolved for fltk-1.3. I'd suggest to write a test program to check whether it works as you expect in fltk-1.3.
I dislike the timeout kludge as well, since it's slow and extremely unstable, but as of yet have not found a workable alternative considering the current implementation. I haven't been able to dig further into fltk itself, due to lack of time and familiarity with the codebase (though I intend to someday, as there are some other things in there that need work, like popup menus on Windows 95).
By the way, this is one reason I was interested in an alternate implementation using libcurl -- it would be much simpler than trying to implement low-level, cross-platform I/O ourselves.
I doubt that libcurl would be an option, but as I said, feel free to experiment. Cheers, Johannes
On Sun, 03 Oct 2010 16:22:27 -0400, Johannes Hofmann <Johannes.Hofmann@gmx.de> wrote:
I doubt that libcurl would be an option, but as I said, feel free to experiment.
Cheers, Johannes
The idea would be to provide libcurl as an *alternative to* Dillo's implementation, not a replacement for it. Cheers, ~Benjamin
participants (3)
-
jcid＠dillo.org -
Johannes.Hofmann＠gmx.de -
obeythepenguin＠gmail.com