- Dillo-dev - lists.mailman3.com

Re: dillo friendly site directory

by Kevin Koster

a1ex-J7K0XVabL0iELgA04lAiVw(a)public.gmane.org wrote: > I'm sure there are more that I missed, corrections & updates welcome! Here's another source of links: https://libredirect.github.io/ Some more from there that seem to work in Dillo: Urban Dictionary: https://codeberg.org/zortazert/rural-dictionary Instructables: https://git.vern.cc/cobra/Destructables/src/branch/main/instances.json https://indestructables.codeberg.page/ Wayback Machine: https://github.com/ticky/wayback-classic I haven't got time to check more now.

Sept. 15, 2024

Re: Feature idea: save tabs list

by a1ex＠dismail.de

Diego <darkspirit5000(a)gmail.com> wrote: > I have an idea with minimal changes in dillo and various scripts. > > - Activate and format some MSGs in dillo > - use a script to run dillo so saves and load tabs MSGs based That is a neat solution, thanks! Ideally this will all happen inside of Dillo, but until then, your idea should make it easier to start testing this feature. There are also possibilities to customize it, like excluding certain domains from being saved, or removing duplicate tabs, etc. -Alex

Sept. 15, 2024

Re: dillo friendly site directory

by a1ex＠dismail.de

Hi, On Sun, 15 Sep 2024 08:10:26 +1000 "Kevin Koster" <dillo(a)ombertech.com> wrote: > > Maybe we could list some basic service alternatives first, and then > > a few specific directory links? > > There are directory lists of "front-ends" as well: > https://codeberg.org/skynet2982/awesome-alternative-front-ends > https://github.com/mendel5/alternative-front-ends I went through these lists (and others), and pulled out most of the main service alternatives which I verified to work with Dillo: Search: https://searx.neocities.org/nojs https://tailsx.com/ https://github.com/Ahwxorg/librey/ https://lx.vern.cc/instances.php https://github.com/benbusby/whoogle-search#public-instances https://search.marginalia.nu/ https://frogfind.com https://4get.ca/instances Youtube: https://docs.invidious.io/instances/ https://tube.cadence.moe/ https://poketube.fun/ https://codemadness.org/idiotbox/ Reddit: https://github.com/libreddit/libreddit-instances/blob/master/instances.md https://eddrit.com/ RSS: https://rss-bridge.github.io/rss-bridge/General/Public_Hosts.html https://m.simplepie.org Twitter: https://github.com/zedeus/nitter/wiki/Instances Wikipedia: https://wiki.zaggy.nl/nojs https://wikiless.org News: https://neuters.de https://68k.news Imgur: https://codeberg.org/rimgo/instances Bandcamp: https://forgejo.sny.sh/sun/Tent StackOverflow: https://code.whatever.social/ Medium.com https://scribe.rip/ https://git.batsense.net/realaravinth/libmedium IMDB: https://github.com/zyachel/libremdb#instances Quora: https://github.com/zyachel/quetre#instances Odysee / LBRY: https://codeberg.org/librarian/librarian#instances Gitlab: https://lab.vern.cc I'm sure there are more that I missed, corrections & updates welcome! To be clear, my point here is to make a curated list of Dillo-compatible main service alternatives, not a list of all websites that render in Dillo. Also, I would like to focus on free, open-source projects, so I'm trying to avoid listing any commercial/proprietary sites. Regards, Alex

Sept. 15, 2024

Re: dillo friendly site directory

by Kevin Koster

a1ex-J7K0XVabL0iELgA04lAiVw(a)public.gmane.org wrote: > On Sat, 14 Sep 2024 14:17:38 +0200 > Rodrigo Arias <rodarima-Re5JQEeQqe8AvxtiuMwx3w(a)public.gmane.org> wrote: > >> I guess it would be easier to have a directory of directories, so we >> don't need to do a lot of maintenance. So instead of asking us to add >> your nice page/blog/whatever, just submit it to one of the >> directories we link to. > > Maybe we could list some basic service alternatives first, and then > a few specific directory links? There are directory lists of "front-ends" as well: https://codeberg.org/skynet2982/awesome-alternative-front-ends https://github.com/mendel5/alternative-front-ends But they both include local programs/apps, and lots of "Web apps" that don't work any better than the original websites in Dillo due to CSS or JS. One that seems to be missing in those lists is the Laboratory JS-free front-end for GitLab, which works in Dillo: https://lab.vern.cc Gopher front-ends should be more compatible, via Dillo's Gopher plug-in. Many were listed in this discussion on Usenet in the comp.infosystems.gopher newsgroup: https://comp.infosystems.gopher.narkive.com/GASoqqif/gopher-based-services Although that might lead to making a Web browser that suggests people don't use the Web! > Some of those directories contain a lot of random stuff that > doesn't even work, which might be discouraging to new users. For directories of websites that work in Dillo, there's No-JS.club and noCSS.club: https://no-js.club/ https://nocss.club/ Probably even more "random stuff" there though.

Sept. 14, 2024

Re: dillo friendly site directory

by a1ex＠dismail.de

Hi Rodrigo, On Sat, 14 Sep 2024 14:17:38 +0200 Rodrigo Arias <rodarima(a)gmail.com> wrote: > I guess it would be easier to have a directory of directories, so we > don't need to do a lot of maintenance. So instead of asking us to add > your nice page/blog/whatever, just submit it to one of the > directories we link to. Maybe we could list some basic service alternatives first, and then a few specific directory links? Some of those directories contain a lot of random stuff that doesn't even work, which might be discouraging to new users. Here is where I would start: Search: SearX instances https://searx.neocities.org/nojs Youtube: invidious instances https://api.invidious.io/ Reddit: libreddit instances https://github.com/libreddit/libreddit-instances/blob/master/instances.md RSS: rss-bridge instances https://rss-bridge.github.io/rss-bridge/General/Public_Hosts.html Social media: nitter instances https://status.d420.de/ Web Archive (sadly not foss): https://archive.is There are probably more I missed, but you get the idea. Perhaps then we could also mention the tildeverse: https://tildeverse.org It would take slightly more work to maintain a list like this rather than just having a list of directories, but I think it could be significantly more useful. Regards, Alex

Sept. 14, 2024

Re: Feature idea: save tabs list

by a1ex＠dismail.de

On Sat, 14 Sep 2024 11:29:36 +0042 yeti <yeti(a)tilde.institute> wrote: > SNAC2 can be styled to be usable via Dillo: > > <https://comam.es/snac/grunfink> > <https://yeti.tilde.institute/dillo-screenshots/20240914-103713__dillo__a_SN…> Thanks for the heads-up! I saw this project a while ago and thought it was quite interesting. Nice to see that its still moving forward :) -Alex

Sept. 14, 2024

'load_images' pref ineffective on embedded images

by a1ex＠dismail.de

Hi, The pref 'load_images=NO' only disables loading images from a remote img src, and not from an embedded[1] data:image src. If this behavior is intentional, would it be reasonable to implement an option to disable all images? It might be the case that the user does not want to see any images, no matter the img src. [1] <img src="data:image/png;base64,..."> Regards, Alex

Sept. 14, 2024

Re: Feature idea: save tabs list

by Diego

I have an idea with minimal changes in dillo and various scripts. - Activate and format some MSGs in dillo - use a script to run dillo so saves and load tabs MSGs based Example: patch with make a script run_dillo.sh fn load_tabs ... dillo $@ | while read line echo $line # process dillo MSGs # MSG("a_UIcmd_open_url_nw: open url in new window '%s' bw=%p new_bw=%p\n", URL_STR(url), (void *) bw, (void *) new_bw); # MSG("a_UIcmd_open_url_nt: open url in new tab '%s' bw=%p new_bw=%p\n", URL_STR(url), (void *) bw, (void *) new_bw); # MSG("a_UIcmd_close_bw bw=%p\n", (void *) bw); # MSG("Nav_open_url: old_url='%s' idx=%d bw=%p\n", URL_STR(old_url), idx, (void *) bw); # MSG("Nav_open_url: saved scroll of '%s' at x=%d y=%d bw=%p\n", URL_STR(old_url), x, y, (void *) bw); # MSG("Nav: expect_done scrolling to x=%d y=%d bw=%p\n", posx, posy, (void *) bw); selec $line a_UIcmd_open_url_nw) ;; open new window anote bw a_UIcmd_open_url_nt) ;; open new tab anote bw a_UIcmd_close_bw) ;; delete window or tab Nav_open_url: old_url) ;; change url for tab Nav_open_url: saved scroll) ;; annotate scroll pos Nav: expect_done scrolling to) ;; annotate scroll pos Dillo: normal exit) ;; annotate normal exit El jue, 12 sept 2024 a las 20:01, <a1ex(a)dismail.de> escribió: > > Hi, > > It would be nice to have the ability to save a list of open tabs. Often > I find myself shutting down my workstation and still have many tabs > open in Dillo. Its a pain to go through them all at that moment to > decide whats worth saving or bookmarking. > > An option to save all open tab urls to a file would be great. It would > be even nicer if we could later open all those urls in new tabs when > starting Dillo :) > > On first glance it doesn't seem very easy to implement, but also > maybe not super complicated either. I don't have any patches yet, just > wanted to put the idea out and see what you think about it. > > Regards, > Alex > > > > _______________________________________________ > Dillo-dev mailing list -- dillo-dev(a)mailman3.com > To unsubscribe send an email to dillo-dev-leave(a)mailman3.com

Sept. 14, 2024

Re: Feature idea: save tabs list

by Rodrigo Arias

Hi, >This is a valid concern. Maybe there could be a warning if opening more >than 20 tabs (or whatever number is reasonable). I generally don't have >many more than that open anyway, and usually they are a mixture of >sites, so it shouldn't be a big issue for me at least. Yeah, that sounds reasonable. >On a side note, I still sometimes get blank tab names with no title, >even after this commit: >https://github.com/dillo-browser/dillo/pull/194/commits > >Here is an example: >https://search.dojoro.de/ > >When I search for something, the address becomes: >https://search.dojoro.de/search > >and yet the tab name still remains blank. It would be nice to have the >domain name as the tab name in these cases. That is what their page says: <title></title> But yeah, we should fall back to the URL instead of the file name. >Here are a few examples: >Search - SearXNG instances, frogfind.com, marginalia >Mailing lists - marc.info, mail-archive.com >Proxy - MortyProxy instances, floodgap.com gopher, gopherpedia >News - neuters.de, m.simplepie.org rss, rss-bridge instances >Tech - hn (or hngopher), tildes.net, lobste.rs, bleepingcomputer.com >Reddit - libreddit instances, gopherddit.com >Youtube - invidious instances, https://codemadness.org/idiotbox/ >OpenBSD stuff - freshbsd.org, openports.pl, undeadly.org, etc >Social media - nope > >Obviously this list is rather privacy focused, but I'd be happy to help >put together a more general list of sites which are Dillo-friendly. Nice ones! I think it would be handy to do such a list. Here are some directories, but the CSS is pretty broken on some, at least until we implement CSS variables: https://sjmulder.nl/en/textonly.html https://jsfree.org/ https://blogroll.org/ https://ooh.directory/ https://curlie.org/en/ I guess it would be easier to have a directory of directories, so we don't need to do a lot of maintenance. So instead of asking us to add your nice page/blog/whatever, just submit it to one of the directories we link to. Best, Rodrigo.

Sept. 14, 2024

Re: Feature idea: save tabs list

by yeti

<a1ex(a)dismail.de> writes: > Social media - nope SNAC2 can be styled to be usable via Dillo: <https://comam.es/snac/grunfink> <https://yeti.tilde.institute/dillo-screenshots/20240914-103713__dillo__a_SN…> That does not mean, that every SNAC2 instance is "Dillo compatible". -- 1. Hitchhiker 5: (101) "You just come along with me and have a good time. The Galaxy's a fun place. You'll need to have this fish in your ear."

Sept. 14, 2024

Re: Feature idea: save tabs list

by a1ex＠dismail.de

Hi Rodrigo, On Sat, 14 Sep 2024 08:49:14 +0200 Rodrigo Arias <rodarima(a)gmail.com> wrote: > >An option to save all open tab urls to a file would be great. It > >would be even nicer if we could later open all those urls in new > >tabs when starting Dillo :) > > Maybe it is a good idea to add a button in the quit dialog to allow > the user to save the current "session" for later (and/or set in > dillorc what to do by default). > > Keep in mind this will cause a spike of work loading hundreds of > tabs, and it can potentially ban you if all those tabs point to the > same server, as they will all load at the same time. This is a valid concern. Maybe there could be a warning if opening more than 20 tabs (or whatever number is reasonable). I generally don't have many more than that open anyway, and usually they are a mixture of sites, so it shouldn't be a big issue for me at least. On a side note, I still sometimes get blank tab names with no title, even after this commit: https://github.com/dillo-browser/dillo/pull/194/commits Here is an example: https://search.dojoro.de/ When I search for something, the address becomes: https://search.dojoro.de/search and yet the tab name still remains blank. It would be nice to have the domain name as the tab name in these cases. > I'm wondering how you use Dillo regularly and what sites you use that > work good without JS. I'm thinking in writing a short introduction to > directories and other pages to let newcomers have something to > actually read, rather than opening youtube.com and seeing a blank > page. My use case for Dillo is mostly reading text (research, news, mailing lists, oss dev sites, etc). I browse with images and css off by default, for me they are usually an unnecessary distraction, and only turn them on as needed. As for sites, Here are a few examples: Search - SearXNG instances, frogfind.com, marginalia Mailing lists - marc.info, mail-archive.com Proxy - MortyProxy instances, floodgap.com gopher, gopherpedia News - neuters.de, m.simplepie.org rss, rss-bridge instances Tech - hn (or hngopher), tildes.net, lobste.rs, bleepingcomputer.com Reddit - libreddit instances, gopherddit.com Youtube - invidious instances, https://codemadness.org/idiotbox/ OpenBSD stuff - freshbsd.org, openports.pl, undeadly.org, etc Social media - nope Obviously this list is rather privacy focused, but I'd be happy to help put together a more general list of sites which are Dillo-friendly. Regards, Alex PS: Why don't I use reddit/google/wikipedia/news/etc directly instead of using a intermediary? Because I don't trust them. I have nothing to hide, but these corporations are evil, and I will go out of my way to avoid them.

Sept. 14, 2024

Re: Feature idea: save tabs list

by Rodrigo Arias

Hi Alex, On Thu, Sep 12, 2024 at 07:58:35PM +0200, a1ex(a)dismail.de wrote: >Hi, > >It would be nice to have the ability to save a list of open tabs. Often >I find myself shutting down my workstation and still have many tabs >open in Dillo. Its a pain to go through them all at that moment to >decide whats worth saving or bookmarking. I use an extension (Tab Wrangler) in Firefox to close unused tabs after some time, so it forces me to bookmark earlier. It kills the tab hoarding problem and keeps the memory usage low, so maybe a similar approach could be used in Dillo. >An option to save all open tab urls to a file would be great. It would >be even nicer if we could later open all those urls in new tabs when >starting Dillo :) Maybe it is a good idea to add a button in the quit dialog to allow the user to save the current "session" for later (and/or set in dillorc what to do by default). Keep in mind this will cause a spike of work loading hundreds of tabs, and it can potentially ban you if all those tabs point to the same server, as they will all load at the same time. >On first glance it doesn't seem very easy to implement, but also >maybe not super complicated either. I don't have any patches yet, just >wanted to put the idea out and see what you think about it. Sounds good for me. I'm wondering how you use Dillo regularly and what sites you use that work good without JS. I'm thinking in writing a short introduction to directories and other pages to let newcomers have something to actually read, rather than opening youtube.com and seeing a blank page. Best, Rodrigo.

Sept. 14, 2024

Feature idea: save tabs list

by a1ex＠dismail.de

Hi, It would be nice to have the ability to save a list of open tabs. Often I find myself shutting down my workstation and still have many tabs open in Dillo. Its a pain to go through them all at that moment to decide whats worth saving or bookmarking. An option to save all open tab urls to a file would be great. It would be even nicer if we could later open all those urls in new tabs when starting Dillo :) On first glance it doesn't seem very easy to implement, but also maybe not super complicated either. I don't have any patches yet, just wanted to put the idea out and see what you think about it. Regards, Alex

Sept. 12, 2024

Re: [WIP PATCH] Use SIGUSR1 to reload the page

by Rodrigo Arias

Hi Alex, >I didn't get much further with this yet, but in the meantime, here is a >very simple example script which attempts to achieve the same goal. > >It opens the specified file in a new Dillo window and then automatically >reloads the page whenever the file changes. Note that it only reloads >the active tab. > >This relies on the 'entr' and 'xdotool' programs, but 'entr' can be >easily replaced with a bit more scripting if desired. > >```` >#!/bin/sh ># dillomon.sh ># Monitor a local file for changes with Dillo ># usage: dillomon.sh filename > >dillo $1 & sleep 1 >echo $1 | entr xdotool key --window `xdotool search --classname Dillo | >tail -1` ctrl+r > >```` Thanks!, that should do it for now. Eventually I'll need to add a tool to control the browser for automated testing, so I will probably also add the ability to refresh the current page too. So not sure if we want to spend too much effort in this, provided there is a intermediate solution already. I began tagging easy issues with "good first issue", so far only one: https://github.com/dillo-browser/dillo/issues?q=is%3Aissue+is%3Aopen+label%… Best, Rodrigo.

Sept. 11, 2024

Re: [WIP PATCH] Use SIGUSR1 to reload the page

by a1ex＠dismail.de

Hi, On Mon, 2 Sep 2024 21:25:12 +0200 Rodrigo Arias <rodarima(a)gmail.com> wrote: > You need to find a pointer to BrowserWindow and pass it to > a_UIcmd_reload(). Check a_Bw_num() and a_Bw_get() in src/bw.c. > > We should reload the current page on all windows. > > I'm not sure if we can do this from the context of a signal, we > probably need to register a timeout in FLTK and handle it from there. > > Probably not a very easy issue. I didn't get much further with this yet, but in the meantime, here is a very simple example script which attempts to achieve the same goal. It opens the specified file in a new Dillo window and then automatically reloads the page whenever the file changes. Note that it only reloads the active tab. This relies on the 'entr' and 'xdotool' programs, but 'entr' can be easily replaced with a bit more scripting if desired. ```` #!/bin/sh # dillomon.sh # Monitor a local file for changes with Dillo # usage: dillomon.sh filename dillo $1 & sleep 1 echo $1 | entr xdotool key --window `xdotool search --classname Dillo | tail -1` ctrl+r ```` Regards, Alex

Sept. 11, 2024

Re: [PATCH] make downloads.dpi use http_user_agent from dillorc

by Rodrigo Arias

On Tue, Sep 03, 2024 at 01:51:18PM +0200, a1ex(a)dismail.de wrote: >Hi, > >Here is a patch which allows wget called from downloads.dpi to use the >user agent provided in dillorc, and fallback to the hardcoded >Dillo/version agent if not defined. Thanks Alex!, I'll queue it. Best, Rodrigo.

Sept. 9, 2024

Re: Queue Downloads

by a1ex＠dismail.de

> Also, if you want to make this portable so it works on other shells > than just bash, you can do something like this to get the port number: > > echo $a | cut -f4- -d"'" | rev | cut -c 5- | rev > > There are probably other ways to do it using sed, etc. Here is a sed equivalent, which might be a more conventional approach: $ echo $a <cmd='send_data' msg='12345' '> $ echo $a | sed "s/^.*msg=[']$[^']*$['].*$/\1/" 12345 -Alex

Sept. 6, 2024

Re: Queue Downloads

by a1ex＠dismail.de

On Fri, 6 Sep 2024 00:06:41 +0200 Diego <darkspirit5000(a)gmail.com> wrote: > I see same when do use filename without a path. I think download.dpi > needs a filename with one path-segment or more. > > Send script v0.2 with a workarround (prefix "./" if there is not a "/" > in filename) That works. It might be nice to set a default download location, and also automatically get the filename from the url with basename. Also, if you want to make this portable so it works on other shells than just bash, you can do something like this to get the port number: echo $a | cut -f4- -d"'" | rev | cut -c 5- | rev There are probably other ways to do it using sed, etc. I attached a quick and probably incomplete example which incorporates these ideas :) Regards, Alex

Sept. 6, 2024

Re: Queue Downloads

by Diego

I see same when do use filename without a path. I think download.dpi needs a filename with one path-segment or more. Send script v0.2 with a workarround (prefix "./" if there is not a "/" in filename) Best, Diego! El jue, 5 sept 2024 a las 15:17, <a1ex(a)dismail.de> escribió: > > Hi Diego, > > On Wed, 4 Sep 2024 18:17:42 +0200 > Diego <darkspirit5000(a)gmail.com> wrote: > > > Hello, maybe this script can be useful to someone. > > > > Requests downloads.dpi to download a URL to a file. He talks to dpid > > like dillo does. > > > > I think it's also a useful coding exercise. > > Nice, looks like a fun experiment! I think it's neat to be able to > interact with dpid outside of Dillo, and possibly could aid in > troubleshooting dpi development issues in future. > > The script seems to work fine, except the filename in the download > window just shows '??'. Do you get that as well? > > Regards, > Alex > > _______________________________________________ > Dillo-dev mailing list -- dillo-dev(a)mailman3.com > To unsubscribe send an email to dillo-dev-leave(a)mailman3.com

Sept. 5, 2024

Re: Queue Downloads

by a1ex＠dismail.de

Hi Diego, On Wed, 4 Sep 2024 18:17:42 +0200 Diego <darkspirit5000(a)gmail.com> wrote: > Hello, maybe this script can be useful to someone. > > Requests downloads.dpi to download a URL to a file. He talks to dpid > like dillo does. > > I think it's also a useful coding exercise. Nice, looks like a fun experiment! I think it's neat to be able to interact with dpid outside of Dillo, and possibly could aid in troubleshooting dpi development issues in future. The script seems to work fine, except the filename in the download window just shows '??'. Do you get that as well? Regards, Alex

Sept. 5, 2024

Queue Downloads

by Diego

Hello, maybe this script can be useful to someone. Requests downloads.dpi to download a URL to a file. He talks to dpid like dillo does. I think it's also a useful coding exercise. I'm bad at naming so if someone has a better name ^_^¡ Best, Diego!

Sept. 4, 2024

[PATCH] make downloads.dpi use http_user_agent from dillorc

by a1ex＠dismail.de

Hi, Here is a patch which allows wget called from downloads.dpi to use the user agent provided in dillorc, and fallback to the hardcoded Dillo/version agent if not defined. Regards, Alex diff -upr a/dlib/dlib.c b/dlib/dlib.c --- a/dlib/dlib.c Sun Aug 11 22:21:59 2024 +++ b/dlib/dlib.c Tue Sep 3 13:49:00 2024 @@ -922,6 +922,45 @@ char *dGethomedir (void) } /** + * Return the http_user_agent value in a static string + */ +char *dGetuseragent (void) +{ + static char *dillorc = NULL; + dillorc = dStrconcat(dGethomedir(), "/", ".dillo/dillorc", NULL); + FILE *In; + int len; + char *rcline = NULL, *value = NULL, *p; + if ((In = fopen(dillorc, "r")) == NULL) { + DLIB_MSG("dGetuseragent: unable to open dillorc.\n"); + return ("Dillo/" VERSION); + } + while ((rcline = dGetline(In)) != NULL) { + if (strncmp(rcline, "http_user_agent", 15) == 0) + break; + dFree(rcline); + } + fclose(In); + if (!rcline) { + value = ("Dillo/" VERSION); + DLIB_MSG("dGetuseragent: no 'http_user_agent' in dillorc.\n"); + } else { + len = (int) strlen(rcline); + if (len && rcline[len - 1] == '\n') + rcline[len - 1] = 0; + if ((p = strchr(rcline, '='))) { + while (*++p == ' '); + value = dStrdup(p); + } else { + value = ("Dillo/" VERSION); + DLIB_MSG("dGetuseragent: error parsing value in dillorc.\n"); + } + } + dFree(rcline); + return (value); +} + +/** * Get a line from a FILE stream. * Return value: read line on success, NULL on EOF. */ diff -upr a/dlib/dlib.h b/dlib/dlib.h --- a/dlib/dlib.h Sun Aug 11 22:21:59 2024 +++ b/dlib/dlib.h Tue Sep 3 13:49:00 2024 @@ -175,6 +175,7 @@ void dLib_show_messages(bool_t show); */ char *dGetcwd(void); char *dGethomedir(void); +char *dGetuseragent(void); char *dGetline(FILE *stream); int dClose(int fd); int dUsleep(unsigned long us); diff -upr a/dpi/downloads.cc b/dpi/downloads.cc --- a/dpi/downloads.cc Sun Aug 11 22:21:59 2024 +++ b/dpi/downloads.cc Tue Sep 3 13:49:06 2024 @@ -45,6 +45,7 @@ #include "config.h" #include "dpiutil.h" #include "../dpip/dpip.h" +#include "../dlib/dlib.h" /* * Debugging macros @@ -324,7 +325,7 @@ DLItem::DLItem(const char *full_filename, const char * cookies_path = dStrconcat(dGethomedir(), "/.dillo/cookies.txt", NULL); dl_argv = new char*[10]; int i = 0; - const char *user_agent = "Dillo/" VERSION; + const char *user_agent = dGetuseragent(); dl_argv[i++] = (char*)"wget"; if (stat(fullname, &ss) == 0) init_bytesize = (int)ss.st_size;

Sept. 3, 2024

Re: [WIP PATCH] Use SIGUSR1 to reload the page

by Rodrigo Arias

Hi, On Mon, Sep 02, 2024 at 04:04:30PM +0200, a1ex(a)dismail.de wrote: >Hi Rodrigo, > >I find your recent idea[0] to refresh the page on local file changes >interesting, and it could be a quite useful feature. > >[0] https://github.com/dillo-browser/dillo/issues/255 > >I imagine it can be triggered with a tool like entr[1]: > >ls /tmp/test.html | entr 'pkill -SIGUSR1 dillo' > >[1] http://eradman.com/entrproject/ Yeah, that was the idea. >So far, I have a working signal handler which will act upon receipt of >a SIGUSR1 signal. It just prints the signal # to the console right now. >I am trying to make it perform an 'a_UIcmd_reload', but haven't been >successful yet: > >dillo.cc:389:19: error: use of undeclared identifier 'vbw' > a_UIcmd_reload(*vbw); > ^ >1 error generated You need to find a pointer to BrowserWindow and pass it to a_UIcmd_reload(). Check a_Bw_num() and a_Bw_get() in src/bw.c. We should reload the current page on all windows. I'm not sure if we can do this from the context of a signal, we probably need to register a timeout in FLTK and handle it from there. Probably not a very easy issue. I should start labeling the ones that are easy. Best, Rodrigo.

Sept. 2, 2024

[WIP PATCH] Use SIGUSR1 to reload the page

by a1ex＠dismail.de

Hi Rodrigo, I find your recent idea[0] to refresh the page on local file changes interesting, and it could be a quite useful feature. [0] https://github.com/dillo-browser/dillo/issues/255 I imagine it can be triggered with a tool like entr[1]: ls /tmp/test.html | entr 'pkill -SIGUSR1 dillo' [1] http://eradman.com/entrproject/ So far, I have a working signal handler which will act upon receipt of a SIGUSR1 signal. It just prints the signal # to the console right now. I am trying to make it perform an 'a_UIcmd_reload', but haven't been successful yet: dillo.cc:389:19: error: use of undeclared identifier 'vbw' a_UIcmd_reload(*vbw); ^ 1 error generated bw.h and uicmd.hh are defined in dillo.cc, so its probably a skill issue on my end :) Any hints would be appreciated! Regards, Alex --- a/src/dillo.cc Sun Aug 11 22:21:59 2024 +++ b/src/dillo.cc Mon Sep 2 14:21:08 2024 @@ -381,6 +381,14 @@ static DilloUrl *makeStartUrl(char *str, bool local) } /* + * Use SIGUSR1 to reload the page + */ +static void reload_handler(int signum) +{ + MSG("signum=%d\n\n", signum); +} + +/* * MAIN */ int main(int argc, char **argv) @@ -579,6 +587,10 @@ int main(int argc, char **argv) a_Url_free(start_url); } } + + // Reload page on SIGUSR1 + if (signal (SIGUSR1, reload_handler) == SIG_IGN) + signal (SIGUSR1, SIG_IGN); Fl::run();

Sept. 2, 2024

Re: Issues with HTTP multipart/form-data file upload

by Rodrigo Arias

Hi Xavier, On Sun, Sep 01, 2024 at 11:17:53PM +0200, Xavier Del Campo Romero wrote: >> This other method has a slightly not >> uniform distribution, but only runs rand() 70 times: >> >> Which is probably okay for this case. > >I also think it is good enough. > >> Then, I think using our own set may be the most readable solution here, >> which also allows the quick module method. > >I agree. Please, take a look at this PR: https://github.com/dillo-browser/dillo/pull/256 Here is the raw patch series for "git am": https://github.com/dillo-browser/dillo/pull/256.patch It would be nice if you can test it with slcl to know it works fine. Best, Rodrigo.

Sept. 1, 2024

Re: Issues with HTTP multipart/form-data file upload

by Xavier Del Campo Romero

Hi Rodrigo, > Not sure what you mean with "unnecessary use of the heap". I meant > something like this: You are right: I was too fixated with strcspn(3), when in fact strchr(3) would already do. Also, this use of dStr_append_c() looks good to me. > The variable c is stored in a register, as you can see in the > disassembly (built with -Og): Thanks for the detailed explanation. Anyway, I do not expect this function to become a performance bottleneck. > This other method has a slightly not > uniform distribution, but only runs rand() 70 times: > > Which is probably okay for this case. I also think it is good enough. > Then, I think using our own set may be the most readable solution here, > which also allows the quick module method. I agree. Best regards, Xavi On 1/9/24 16:12, Rodrigo Arias wrote: > Hi Xavier, > >> Thank you. I am still unfamiliar with that part of Dillo, so please let >> me know about any progress. > > For now I'm still writing the RFC and doing some proof of concepts. I > can see that it will take a while. > >> Limiting ourselves to a-z, A-Z and 0-9 would still account for 62 out of >> the 75 possible characters, so roughly 82% of the set. I think that >> removing the quoting in favour of the limited set reduce the risk for >> broken implementations, yet still provide a good amount of randomness. > > Yes, I think so too. > >> I am not sure whether this was an intentional modification from your >> side. My patch is adding a <space> as defined by POSIX.1-2017 [1], so >> that sizeof " " would always return 2. Was it your intention to flag >> this potential confusion? > > Yes, my point was that it is that is not easy to determine the length of > a UTF-8 string by just looking at it. > >> Also, there was not strict reason to use sizeof " ". Any other >> character would do e.g.: sizeof "x", sizeof "A", etc. > > Same problem: > > ᕁ 𝓍 𝙭 х 𝐱 𝗑 ⤫ 𝑥 𝘅 ⤬ ᙮ ⨯ 𝕩 𝖝 × 𝔁 𝚡 x ｘ ⅹ 𝔵 ᕽ 𝒙 𝘹 > > 𝙰 A 𝐴 ᗅ 𝑨 𝚨 𝕬 𝖠 А Α 𝛢 𝝖 𝒜 𝜜 𖽀 𝓐 𝞐 Ａ 𝗔 𝘈 Ꭺ ꓮ 𝐀 𝔄 𝔸 𐊠 𝘼 > > Check: https://util.unicode.org/UnicodeJsps/confusables.jsp > > It is generally safer to use the explicit length. > >>> You can also use dStr_append_c() to only append one character, so you >>> only need a single character. >> >> That would be an unnecessary use of the heap, because the size is static. > > Not sure what you mean with "unnecessary use of the heap". I meant > something like this: > > static void generate_boundary(Dstr *boundary) > { > for (int i = 0; i < 70; i++) { > /* Extracted from RFC 2046, section 5.1.1. */ > static const char set[] = "abcdefghijklmnopqrstuvwxyz" > "ABCDEFGHIJKLMNOPQRSTUVWXYZ" > "0123456789"; > int c; > > do { > c = rand() & 0xff; > } while (!strchr(set, c)); > > dStr_append_c(boundary, c); > } > } > > The variable c is stored in a register, as you can see in the > disassembly (built with -Og): > > hop% r2 build/src/dillo > WARN: Relocs has not been applied. Please use `-e bin.relocs.apply=true` > or `-e bin.cache=true` next time > [0x000e3860]> s sym.generate_boundary_Dstr_ > [0x0012cf99]> af > [0x0012cf99]> pdf > ┌ 85: sym.generate_boundary_Dstr_ (int64_t arg1); > │ ; arg int64_t arg1 @ rdi > │ 0x0012cf99 55 push rbp ; > Fl_Pixmap.H:1250 ; generate_boundary(Dstr*) > │ 0x0012cf9a 4889e5 mov rbp, rsp > │ 0x0012cf9d 4155 push r13 > │ 0x0012cf9f 4154 push r12 > │ 0x0012cfa1 53 push rbx > │ 0x0012cfa2 4883ec08 sub rsp, 8 > │ 0x0012cfa6 4989fd mov r13, rdi ; > arg1 > │ 0x0012cfa9 41bc00000000 mov r12d, 0 ; > Fl_Pixmap.H:1251 > │ ┌─< 0x0012cfaf eb2c jmp 0x12cfdd > │ ┌┌──> 0x0012cfb1 ff1509cd1e00 call qword [reloc.rand] ; > Fl_Pixmap.H:1253 ; [0x319cc0:8]=0 > │ ╎╎│ 0x0012cfb7 0fb6d8 movzx ebx, al ; > <--- Perform the "& 0xff" by zero-extending the lowest byte in eax > │ ╎╎│ 0x0012cfba 89de mov esi, ebx ; > Fl_Pixmap.H:1260 > │ ╎╎│ 0x0012cfbc 488d3dbd5b.. lea rdi, > obj.generate_boundary_Dstr_::set ; 0x2a2b80 ; > "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789" > │ ╎╎│ 0x0012cfc3 ff15afc51e00 call qword [reloc.strchr] ; > [0x319578:8]=0 > │ ╎╎│ 0x0012cfc9 4885c0 test rax, rax > │ └───< 0x0012cfcc 74e3 je 0x12cfb1 > │ ╎│ 0x0012cfce 89de mov esi, ebx ; > Fl_Pixmap.H:1262 > │ ╎│ 0x0012cfd0 4c89ef mov rdi, r13 > │ ╎│ 0x0012cfd3 67e830570800 call sym.dStr_append_c > │ ╎│ 0x0012cfd9 4183c401 add r12d, 1 ; > Fl_Pixmap.H:1251 > │ ╎│ ; CODE XREF from generate_boundary(Dstr*) @ 0x12cfaf(x) > │ ╎└─> 0x0012cfdd 4183fc45 cmp r12d, 0x45 ; > 'E' > │ └──< 0x0012cfe1 7ece jle 0x12cfb1 > │ 0x0012cfe3 4883c408 add rsp, 8 ; > Fl_Pixmap.H:1264 > │ 0x0012cfe7 5b pop rbx > │ 0x0012cfe8 415c pop r12 > │ 0x0012cfea 415d pop r13 > │ 0x0012cfec 5d pop rbp > └ 0x0012cfed c3 ret > > This method gives us a perfect uniform distribution when RAND_MAX is a > power of 2, at the cost of executing rand() more times than required > (70/(62/256) ≈ 289 on average). This other method has a slightly not > uniform distribution, but only runs rand() 70 times: > > static void generate_boundary(Dstr *boundary) > { > /* Extracted from RFC 2046, section 5.1.1. */ > static const char set[] = "abcdefghijklmnopqrstuvwxyz" > "ABCDEFGHIJKLMNOPQRSTUVWXYZ" > "0123456789"; > static const int n = strlen(set); > > for (int i = 0; i < 70; i++) { > int c = (unsigned char) set[rand() % n]; > dStr_append_c(boundary, c); > } > } > > Which is probably okay for this case. > > Notice I haven't tested any of these methods yet, I'll need to add a > form upload test case to be able to see them in action (or a unit test). > >>> If we only use alphanumeric characters, we can just use isalnum() right? >> >> According to POSIX.1-2017 [2], isalnum(3) depends on the current locale >> configured by the system. For example, characters such as Ä or ú could >> return non-zero. > > Oh right, for some reason I was thinking this was the other way around, > and isalnum only worked with ASCII. I think we should review the other > uses of isalnum and friends as I think there may be used under similar > assumptions. > >> To avoid this, there are two possible solutions: >> >> 1. Use isalnum_l(3) to specify a locale_t object corresponding to the >> "POSIX" locale (equivalent to "C" [3]), which must be previously >> allocated by the newlocale(3) function [3] and released by the >> freelocal(3) function [4]. A minimalist example is shown below: >> >> locale_t l = newlocale(LC_CTYPE, "POSIX", NULL); >> >> for (unsigned char i = 0; i < 255; i++) >> printf("hhu=%hhu, c=%c, isalnum=%d\n", i, i, >> isalnum_l(i, l)); >> >> freelocale(l); >> >> 2. Define a known subset from the portable character set defined by >> POSIX.1-2017 [5] and use strspn(3), as already suggested by the patch. >> IMHO this approach is better because: >> - It does not deal with locales, so developers not familiar with them >> would understand the code better. >> - It is also portable outside a POSIX environment (not sure if this a >> requirement, though). >> - It does not require dynamic allication via newlocale(3). >> - It is the only possible option if non-alnum characters, such as ':' >> or '/', are appended to the boundary string. > > Then, I think using our own set may be the most readable solution here, > which also allows the quick module method. > > Best, > Rodrigo > _______________________________________________ > Dillo-dev mailing list -- dillo-dev(a)mailman3.com > To unsubscribe send an email to dillo-dev-leave(a)mailman3.com

Sept. 1, 2024

Re: Issues with HTTP multipart/form-data file upload

by Rodrigo Arias

Hi Xavier, >Thank you. I am still unfamiliar with that part of Dillo, so please let >me know about any progress. For now I'm still writing the RFC and doing some proof of concepts. I can see that it will take a while. >Limiting ourselves to a-z, A-Z and 0-9 would still account for 62 out of >the 75 possible characters, so roughly 82% of the set. I think that >removing the quoting in favour of the limited set reduce the risk for >broken implementations, yet still provide a good amount of randomness. Yes, I think so too. >I am not sure whether this was an intentional modification from your >side. My patch is adding a <space> as defined by POSIX.1-2017 [1], so >that sizeof " " would always return 2. Was it your intention to flag >this potential confusion? Yes, my point was that it is that is not easy to determine the length of a UTF-8 string by just looking at it. >Also, there was not strict reason to use sizeof " ". Any other >character would do e.g.: sizeof "x", sizeof "A", etc. Same problem: ᕁ 𝓍 𝙭 х 𝐱 𝗑 ⤫ 𝑥 𝘅 ⤬ ᙮ ⨯ 𝕩 𝖝 × 𝔁 𝚡 x ｘ ⅹ 𝔵 ᕽ 𝒙 𝘹 𝙰 A 𝐴 ᗅ 𝑨 𝚨 𝕬 𝖠 А Α 𝛢 𝝖 𝒜 𝜜 𖽀 𝓐 𝞐 Ａ 𝗔 𝘈 Ꭺ ꓮ 𝐀 𝔄 𝔸 𐊠 𝘼 Check: https://util.unicode.org/UnicodeJsps/confusables.jsp It is generally safer to use the explicit length. >> You can also use dStr_append_c() to only append one character, so you >> only need a single character. > >That would be an unnecessary use of the heap, because the size is static. Not sure what you mean with "unnecessary use of the heap". I meant something like this: static void generate_boundary(Dstr *boundary) { for (int i = 0; i < 70; i++) { /* Extracted from RFC 2046, section 5.1.1. */ static const char set[] = "abcdefghijklmnopqrstuvwxyz" "ABCDEFGHIJKLMNOPQRSTUVWXYZ" "0123456789"; int c; do { c = rand() & 0xff; } while (!strchr(set, c)); dStr_append_c(boundary, c); } } The variable c is stored in a register, as you can see in the disassembly (built with -Og): hop% r2 build/src/dillo WARN: Relocs has not been applied. Please use `-e bin.relocs.apply=true` or `-e bin.cache=true` next time [0x000e3860]> s sym.generate_boundary_Dstr_ [0x0012cf99]> af [0x0012cf99]> pdf ┌ 85: sym.generate_boundary_Dstr_ (int64_t arg1); │ ; arg int64_t arg1 @ rdi │ 0x0012cf99 55 push rbp ; Fl_Pixmap.H:1250 ; generate_boundary(Dstr*) │ 0x0012cf9a 4889e5 mov rbp, rsp │ 0x0012cf9d 4155 push r13 │ 0x0012cf9f 4154 push r12 │ 0x0012cfa1 53 push rbx │ 0x0012cfa2 4883ec08 sub rsp, 8 │ 0x0012cfa6 4989fd mov r13, rdi ; arg1 │ 0x0012cfa9 41bc00000000 mov r12d, 0 ; Fl_Pixmap.H:1251 │ ┌─< 0x0012cfaf eb2c jmp 0x12cfdd │ ┌┌──> 0x0012cfb1 ff1509cd1e00 call qword [reloc.rand] ; Fl_Pixmap.H:1253 ; [0x319cc0:8]=0 │ ╎╎│ 0x0012cfb7 0fb6d8 movzx ebx, al ; <--- Perform the "& 0xff" by zero-extending the lowest byte in eax │ ╎╎│ 0x0012cfba 89de mov esi, ebx ; Fl_Pixmap.H:1260 │ ╎╎│ 0x0012cfbc 488d3dbd5b.. lea rdi, obj.generate_boundary_Dstr_::set ; 0x2a2b80 ; "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789" │ ╎╎│ 0x0012cfc3 ff15afc51e00 call qword [reloc.strchr] ; [0x319578:8]=0 │ ╎╎│ 0x0012cfc9 4885c0 test rax, rax │ └───< 0x0012cfcc 74e3 je 0x12cfb1 │ ╎│ 0x0012cfce 89de mov esi, ebx ; Fl_Pixmap.H:1262 │ ╎│ 0x0012cfd0 4c89ef mov rdi, r13 │ ╎│ 0x0012cfd3 67e830570800 call sym.dStr_append_c │ ╎│ 0x0012cfd9 4183c401 add r12d, 1 ; Fl_Pixmap.H:1251 │ ╎│ ; CODE XREF from generate_boundary(Dstr*) @ 0x12cfaf(x) │ ╎└─> 0x0012cfdd 4183fc45 cmp r12d, 0x45 ; 'E' │ └──< 0x0012cfe1 7ece jle 0x12cfb1 │ 0x0012cfe3 4883c408 add rsp, 8 ; Fl_Pixmap.H:1264 │ 0x0012cfe7 5b pop rbx │ 0x0012cfe8 415c pop r12 │ 0x0012cfea 415d pop r13 │ 0x0012cfec 5d pop rbp └ 0x0012cfed c3 ret This method gives us a perfect uniform distribution when RAND_MAX is a power of 2, at the cost of executing rand() more times than required (70/(62/256) ≈ 289 on average). This other method has a slightly not uniform distribution, but only runs rand() 70 times: static void generate_boundary(Dstr *boundary) { /* Extracted from RFC 2046, section 5.1.1. */ static const char set[] = "abcdefghijklmnopqrstuvwxyz" "ABCDEFGHIJKLMNOPQRSTUVWXYZ" "0123456789"; static const int n = strlen(set); for (int i = 0; i < 70; i++) { int c = (unsigned char) set[rand() % n]; dStr_append_c(boundary, c); } } Which is probably okay for this case. Notice I haven't tested any of these methods yet, I'll need to add a form upload test case to be able to see them in action (or a unit test). >> If we only use alphanumeric characters, we can just use isalnum() right? > >According to POSIX.1-2017 [2], isalnum(3) depends on the current locale >configured by the system. For example, characters such as Ä or ú could >return non-zero. Oh right, for some reason I was thinking this was the other way around, and isalnum only worked with ASCII. I think we should review the other uses of isalnum and friends as I think there may be used under similar assumptions. > To avoid this, there are two possible solutions: > >1. Use isalnum_l(3) to specify a locale_t object corresponding to the >"POSIX" locale (equivalent to "C" [3]), which must be previously >allocated by the newlocale(3) function [3] and released by the >freelocal(3) function [4]. A minimalist example is shown below: > > locale_t l = newlocale(LC_CTYPE, "POSIX", NULL); > > for (unsigned char i = 0; i < 255; i++) > printf("hhu=%hhu, c=%c, isalnum=%d\n", i, i, >isalnum_l(i, l)); > > freelocale(l); > >2. Define a known subset from the portable character set defined by >POSIX.1-2017 [5] and use strspn(3), as already suggested by the patch. >IMHO this approach is better because: > - It does not deal with locales, so developers not familiar with them >would understand the code better. > - It is also portable outside a POSIX environment (not sure if this a >requirement, though). > - It does not require dynamic allication via newlocale(3). > - It is the only possible option if non-alnum characters, such as ':' >or '/', are appended to the boundary string. Then, I think using our own set may be the most readable solution here, which also allows the quick module method. Best, Rodrigo

Sept. 1, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by Rodrigo Arias

Hi Alex, On Thu, Aug 29, 2024 at 12:04:12PM +0200, a1ex(a)dismail.de wrote: >Hi Rodrigo, > >On Wed, 28 Aug 2024 23:02:38 +0200 >Rodrigo Arias <rodarima(a)gmail.com> wrote: > >> I think your IP may be banned from narkive. > >Yeah, that does sound like it could be the case. Funny, because I've >never even used that site before. I have a static IP address, so that >might have something to do with it. I'm thinking if you ever ran an exit Tor node that could be a plausible explanation. >On the plus side, without this issue we wouldn't have found the crash >in Dillo, so, thank you narkive for inadvertently helping Dillo :) Yes, this is more a symptom that we don't have enough tests. I'm thinking how to improve the test infrastructure so we can include more complex tests like bad TLS responses from a https server, among others. Best, Rodrigo.

Sept. 1, 2024

Re: Firefox bookmarks plugin

by a1ex＠dismail.de

Hi, On Fri, 30 Aug 2024 10:58:57 +0200 Diego <darkspirit5000(a)gmail.com> wrote: > > It doesn't require describing the available dpis in the file. Not > > sure what would be the advantage of doing so. > > For reference. To explain to the user how to switch between a built-in > DPI and another DPI for the same service. I think this part in the docs[0] covers it: "Plugins are searched in the ~/.dillo/dpi/ directory first, and then in the system directory if not found. The dpi_dir option sets the system dpi directory and must be specified once." [0] https://dillo-browser.github.io/user_help.html#dpidrc It seems reasonable to me, but if you have a better way of wording it, that might be the best place to start. Regards, Alex

Aug. 30, 2024

Re: Issues with HTTP multipart/form-data file upload

by Xavier Del Campo Romero

Hi Rodrigo, > Dillo has a mechanism to read chunks of data from different sources as they are arriving and pass them to the next stage for processing. However, AFAIK it always reads a chunk and appends it to a large buffer. It doesn't free the processed part until is done with the whole thing. > > This would require a change in the way Dillo processes data, but I think it would be required for large files. There are more details in the > devdoc/CCCwork.txt file and in src/chain.c if you want to take a closer look. > > As I'm planning to change the design of the CCC, I think I can take this into account too so it would be doable. I'll add it to the list of shortcomings of the current design. Thank you. I am still unfamiliar with that part of Dillo, so please let me know about any progress. > Okay, I'll focus on the boundary patch first, which is the easiest to merge and then I'll take a closer look at the others. > > Yeah, I would assume a lot of implementations are broken, so we want to try to minimize the chances we run into problems. Limiting ourselves to a-z, A-Z and 0-9 would still account for 62 out of the 75 possible characters, so roughly 82% of the set. I think that removing the quoting in favour of the limited set reduce the risk for broken implementations, yet still provide a good amount of randomness. > Check sizeof " ": https://godbolt.org/z/7Tso8ooYz Interestingly, the " " character on your last email is not really a <space> (<U0020>): $ printf "%s" " " | hd 00000000 e2 80 88 |...| 00000003 Compared to an ASCII whitespace: $ printf "%s" " " | hd 00000000 20 | | 00000001 Both Godbolt and my editor also flag that multi-byte character with a yellow rectangle around it because it would be highly confusing otherwise. For example: printf("len=%zu\n", strlen(" ")); Confusingly returns "len=3". I am not sure whether this was an intentional modification from your side. My patch is adding a <space> as defined by POSIX.1-2017 [1], so that sizeof " " would always return 2. Was it your intention to flag this potential confusion? Also, there was not strict reason to use sizeof " ". Any other character would do e.g.: sizeof "x", sizeof "A", etc. > You can also use dStr_append_c() to only append one character, so you only need a single character. That would be an unnecessary use of the heap, because the size is static. > If we only use alphanumeric characters, we can just use isalnum() right? According to POSIX.1-2017 [2], isalnum(3) depends on the current locale configured by the system. For example, characters such as Ä or ú could return non-zero. To avoid this, there are two possible solutions: 1. Use isalnum_l(3) to specify a locale_t object corresponding to the "POSIX" locale (equivalent to "C" [3]), which must be previously allocated by the newlocale(3) function [3] and released by the freelocal(3) function [4]. A minimalist example is shown below: locale_t l = newlocale(LC_CTYPE, "POSIX", NULL); for (unsigned char i = 0; i < 255; i++) printf("hhu=%hhu, c=%c, isalnum=%d\n", i, i, isalnum_l(i, l)); freelocale(l); 2. Define a known subset from the portable character set defined by POSIX.1-2017 [5] and use strspn(3), as already suggested by the patch. IMHO this approach is better because: - It does not deal with locales, so developers not familiar with them would understand the code better. - It is also portable outside a POSIX environment (not sure if this a requirement, though). - It does not require dynamic allication via newlocale(3). - It is the only possible option if non-alnum characters, such as ':' or '/', are appended to the boundary string. [1]: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap06.html [2]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/isalnum.html [3]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/newlocale.html [4]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/freelocale.html [5]: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap06.html > I meant when is the next KoVoꓘ concert :-) No gigs ahead, but I will keep you informed. :) Best regards, Xavi On 28/8/24 22:47, Rodrigo Arias wrote: > Hi Xavier, > > On Wed, Aug 28, 2024 at 01:04:04AM +0200, Xavier Del Campo Romero wrote: >> Hi Rodrigo, >> >>> Glad to read that you also consider Dillo for slcl, and thanks for >>> preparing the patches :-) >> >> Thank you! I want slcl to be useful to anyone, including users who care >> about minimalist software like Dillo. The web is already too crowded >> with bloated "webapps" and other terrible things. :) > > Agreed! > >>> Sounds good, not sure how complicated it would be to do this. >> >> I still need to investigate this further, but I assume this would >> require Dillo to at least implement a sink callback. >> >> In other words, the component responsible for transmitting the data >> (probably src/IO/IO.c) should trigger a user-defined callback with an >> arbitrarily-sized buffer (typically, of BUFSIZ bytes, as defined by >> stdio.h) that must filled with file data. Then, the user-defined >> callback can fill from zero up to BUFSIZ bytes, which are eventually >> trasmitted to the server. > > Dillo has a mechanism to read chunks of data from different sources as > they are arriving and pass them to the next stage for processing. > However, AFAIK it always reads a chunk and appends it to a large buffer. > It doesn't free the processed part until is done with the whole thing. > > This would require a change in the way Dillo processes data, but I think > it would be required for large files. There are more details in the > devdoc/CCCwork.txt file and in src/chain.c if you want to take a closer > look. > > As I'm planning to change the design of the CCC, I think I can take this > into account too so it would be doable. I'll add it to the list of > shortcomings of the current design. > >> That said, I am still not sure how much actual effort this would take. >> But I am glad to receive positive feedback so far - I will then continue >> to find a solution. >> >>> However, being able to upload multiple files at the same time sounds >>> reasonable, so feel free to try on your own in the meanwhile. >> >> Uploading multiple files at once seems doable - the patches I sent on my >> previous email are probably already doing most of the required work. >> Again, the trickiest task is to send data on-the-fly for each selected >> file. > > Okay, I'll focus on the boundary patch first, which is the easiest to > merge and then I'll take a closer look at the others. > >> >>> Shouldn't it be 68 then? >> >> I understand the opposite: the boundary string with the two leading >> dashes ("--") included can be up to 72 bytes long, and 74 bytes long for >> the ending boundary (which includes two more dashes after the boundary >> string). This is confirmed by reading the BNF defined by RFC 2046 (some >> bits omitted for simplicity), section 5.1.1 [1]: >> >>> boundary := 0*69<bchars> bcharsnospace >>> bchars := bcharsnospace / " " >>> bcharsnospace := DIGIT / ALPHA / "'" / "(" / ")" / >>> "+" / "_" / "," / "-" / "." / >>> "/" / ":" / "=" / "?" >>> dash-boundary := "--" boundary >>> ; boundary taken from the value of >>> ; boundary parameter of the >>> ; Content-Type field> multipart-body := >>> [preamble CRLF] >>> dash-boundary transport-padding CRLF >>> body-part *encapsulation >>> close-delimiter transport-padding >>> [CRLF epilogue] >>> delimiter := CRLF dash-boundary >>> close-delimiter := delimiter "--" > > Oh right! I see that we are already using 70 characters anyway. > >> Note: even if the specification tells receivers to handle transport >> padding, for the time being I am assuming "transport-padding" as zero >> length since composers must not generate non-zero length transport >> padding. I am still not sure where transport padding would apply, >> anyway. Probably outside web browsers? >> >>> I would leave out all the symbols to avoid quoting and only use A-Z >>> a-z and 0-9. >> >> Interestingly, Dillo would always quote boundary strings [2], even if >> only using A-Z, a-z and 0-9. In fact, this is one of the wrong >> assumptions I spotted when testing slcl against Dillo. > > Yeah, I would assume a lot of implementations are broken, so we want to > try to minimize the chances we run into problems. > > Apart from slcl we should also test this with some sites and see if they > continue to work okay. > > This will also increase the fingerprinting information to distinguish > Dillo among other browsers, but I think it is not more information that > the already leaked by the user agent. > >>> Which, if I computed it correctly, is still too small to worry about. >> >> Not only it is too small of a chance: if we really wanted to do "the >> right thing" and make Dillo absolutely sure the boundary string is not >> contained within the selected files, this would imply a noticeable >> performance impact when dealing with large files, much likely for a >> near-zero benefit. >> >> I have not inspected their source code yet (and I do not want to), but I >> understand both Gecko and Chromium are also making that assumption, >> because otherwise it would take them a lot of CPU time to upload large >> files. > > But then they would be doing such assumption with a "much larger" > probability it hits the file. > > Skipping it with 70 characters is safe for one file, but also probably > safe for all files ever uploaded with Dillo. > > Maybe curl or other small codebases are easier to read, but not really > needed. > >> >>> Why sizeof " " instead of just 2? >> >> Because, to my eyes, sizeof " " has more meaningful semantics, compared >> to a magic integer constant such as 2. However, for this simple >> scenario, I would still consider both acceptable. > > Check sizeof " ": https://godbolt.org/z/7Tso8ooYz > > You can also use dStr_append_c() to only append one character, so you > only need a single character. > > If we only use alphanumeric characters, we can just use isalnum() right? > >> I can replace it with 2 if you find the other construct unacceptable. >> >>> PS: When are you playing? >> >> Sorry, I did not understand your last sentence. Could you please give a >> bit more context? :) > > I meant when is the next KoVoꓘ concert :-) > > Best, > Rodrigo. > _______________________________________________ > Dillo-dev mailing list -- dillo-dev(a)mailman3.com > To unsubscribe send an email to dillo-dev-leave(a)mailman3.com

Aug. 30, 2024

Re: Firefox bookmarks plugin

by Diego

El mar, 27 ago 2024 a las 23:17, Rodrigo Arias (<rodarima(a)gmail.com>) escribió: > > Hi, > > On Mon, Aug 26, 2024 at 01:31:13AM +0200, Diego wrote: > >Good work and very interesting idea. > > Thanks! > > >If I can suggest a change I would add this lines to dpidrc at Makefile > >the installation > > > ># Default dillo bookmarks dpi > >#bookmarks=bookmarks/bookmarks.dpi > ># ffbm: A Dillo plugin to sync bookmarks with Firefox > >bookmarks=$(DPI_DIR)/$(BIN) > > > > > >maybe it's time to patch dpidrc to make it more descriptive > > > >I sugest this > >dpi_dir=/usr/local/lib/dillo/dpi > > > ># standard dillo distributed DPIs > >#bookmarks=bookmarks/bookmarks.dpi > >#cookies=cookies/cookies.dpi > >#downloads=downloads/downloads.dpi > >#vsource=vsource/vsource.filter.dpi > > The file dpidrc only registers the protocols (ftp: -> ftp dpi), the dpis > are taken from the dpi_dir or ~/.dillo/dpi automatically. > I know. > It doesn't require describing the available dpis in the file. Not sure > what would be the advantage of doing so. For reference. To explain to the user how to switch between a built-in DPI and another DPI for the same service. Maybe a documentation patch better? I can try to write some documentation about it Diego.

Aug. 30, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by a1ex＠dismail.de

Hi Rodrigo, On Wed, 28 Aug 2024 23:02:38 +0200 Rodrigo Arias <rodarima(a)gmail.com> wrote: > I think your IP may be banned from narkive. Yeah, that does sound like it could be the case. Funny, because I've never even used that site before. I have a static IP address, so that might have something to do with it. On the plus side, without this issue we wouldn't have found the crash in Dillo, so, thank you narkive for inadvertently helping Dillo :) Regards, Alex

Aug. 29, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by Rodrigo Arias

Hi, >I have confirmed that this patch prevents Dillo from crashing. >Thanks! Nice! >> However, it doesn't explain the EOF you are getting in narkive. Are >> you using tor or some proxy? > >No vpn/tor/proxy. I wouldn't be surprised if my network provider was >doing some weird stuff though... Maybe thats an area for further study. Look at what I just found: hop% torsocks curl -v https://narkive.com * Host narkive.com:443 was resolved. * IPv6: (none) * IPv4: 149.248.211.108 * Trying 149.248.211.108:443... * Connected to narkive.com (149.248.211.108) port 443 * ALPN: curl offers h2,http/1.1 * TLSv1.3 (OUT), TLS handshake, Client hello (1): * CAfile: /etc/ssl/certs/ca-certificates.crt * CApath: none * OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to narkive.com:443 * Closing connection curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to narkive.com:443 Same if I pass Dillo traffic via tor. I think your IP may be banned from narkive. Try in some days or reset your router if you have a dynamic IP. Or mail them if persists. Best, Rodrigo.

Aug. 28, 2024

Re: Issues with HTTP multipart/form-data file upload

by Rodrigo Arias

Hi Xavier, On Wed, Aug 28, 2024 at 01:04:04AM +0200, Xavier Del Campo Romero wrote: >Hi Rodrigo, > >> Glad to read that you also consider Dillo for slcl, and thanks for preparing the patches :-) > >Thank you! I want slcl to be useful to anyone, including users who care >about minimalist software like Dillo. The web is already too crowded >with bloated "webapps" and other terrible things. :) Agreed! >> Sounds good, not sure how complicated it would be to do this. > >I still need to investigate this further, but I assume this would >require Dillo to at least implement a sink callback. > >In other words, the component responsible for transmitting the data >(probably src/IO/IO.c) should trigger a user-defined callback with an >arbitrarily-sized buffer (typically, of BUFSIZ bytes, as defined by >stdio.h) that must filled with file data. Then, the user-defined >callback can fill from zero up to BUFSIZ bytes, which are eventually >trasmitted to the server. Dillo has a mechanism to read chunks of data from different sources as they are arriving and pass them to the next stage for processing. However, AFAIK it always reads a chunk and appends it to a large buffer. It doesn't free the processed part until is done with the whole thing. This would require a change in the way Dillo processes data, but I think it would be required for large files. There are more details in the devdoc/CCCwork.txt file and in src/chain.c if you want to take a closer look. As I'm planning to change the design of the CCC, I think I can take this into account too so it would be doable. I'll add it to the list of shortcomings of the current design. >That said, I am still not sure how much actual effort this would take. >But I am glad to receive positive feedback so far - I will then continue >to find a solution. > >> However, being able to upload multiple files at the same time sounds reasonable, so feel free to try on your own in the meanwhile. > >Uploading multiple files at once seems doable - the patches I sent on my >previous email are probably already doing most of the required work. >Again, the trickiest task is to send data on-the-fly for each selected file. Okay, I'll focus on the boundary patch first, which is the easiest to merge and then I'll take a closer look at the others. > >> Shouldn't it be 68 then? > >I understand the opposite: the boundary string with the two leading >dashes ("--") included can be up to 72 bytes long, and 74 bytes long for >the ending boundary (which includes two more dashes after the boundary >string). This is confirmed by reading the BNF defined by RFC 2046 (some >bits omitted for simplicity), section 5.1.1 [1]: > >> boundary := 0*69<bchars> bcharsnospace >> bchars := bcharsnospace / " " >> bcharsnospace := DIGIT / ALPHA / "'" / "(" / ")" / >> "+" / "_" / "," / "-" / "." / >> "/" / ":" / "=" / "?" >> dash-boundary := "--" boundary >> ; boundary taken from the value of >> ; boundary parameter of the >> ; Content-Type field> multipart-body := [preamble CRLF] >> dash-boundary transport-padding CRLF >> body-part *encapsulation >> close-delimiter transport-padding >> [CRLF epilogue] >> delimiter := CRLF dash-boundary >> close-delimiter := delimiter "--" Oh right! I see that we are already using 70 characters anyway. >Note: even if the specification tells receivers to handle transport >padding, for the time being I am assuming "transport-padding" as zero >length since composers must not generate non-zero length transport >padding. I am still not sure where transport padding would apply, >anyway. Probably outside web browsers? > >> I would leave out all the symbols to avoid quoting and only use A-Z a-z and 0-9. > >Interestingly, Dillo would always quote boundary strings [2], even if >only using A-Z, a-z and 0-9. In fact, this is one of the wrong >assumptions I spotted when testing slcl against Dillo. Yeah, I would assume a lot of implementations are broken, so we want to try to minimize the chances we run into problems. Apart from slcl we should also test this with some sites and see if they continue to work okay. This will also increase the fingerprinting information to distinguish Dillo among other browsers, but I think it is not more information that the already leaked by the user agent. >> Which, if I computed it correctly, is still too small to worry about. > >Not only it is too small of a chance: if we really wanted to do "the >right thing" and make Dillo absolutely sure the boundary string is not >contained within the selected files, this would imply a noticeable >performance impact when dealing with large files, much likely for a >near-zero benefit. > >I have not inspected their source code yet (and I do not want to), but I >understand both Gecko and Chromium are also making that assumption, >because otherwise it would take them a lot of CPU time to upload large >files. But then they would be doing such assumption with a "much larger" probability it hits the file. Skipping it with 70 characters is safe for one file, but also probably safe for all files ever uploaded with Dillo. Maybe curl or other small codebases are easier to read, but not really needed. > >> Why sizeof " " instead of just 2? > >Because, to my eyes, sizeof " " has more meaningful semantics, compared >to a magic integer constant such as 2. However, for this simple >scenario, I would still consider both acceptable. Check sizeof " ": https://godbolt.org/z/7Tso8ooYz You can also use dStr_append_c() to only append one character, so you only need a single character. If we only use alphanumeric characters, we can just use isalnum() right? >I can replace it with 2 if you find the other construct unacceptable. > >> PS: When are you playing? > >Sorry, I did not understand your last sentence. Could you please give a >bit more context? :) I meant when is the next KoVoꓘ concert :-) Best, Rodrigo.

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by Theo Buehler

Thanks to both of you for tracking this down. I pulled the fix into the OpenBSD port for now.

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by a1ex＠dismail.de

On Wed, 28 Aug 2024 21:40:20 +0200 Rodrigo Arias <rodarima(a)gmail.com> wrote: > Hi, > > >This is enough for me to at least fix this case on Dillo side. > > I see, I was trying to set the conn->in_connect = FALSE after conn > was freed. Try this patch: > > diff --git a/src/IO/tls_openssl.c b/src/IO/tls_openssl.c > index 3e68e928..5ad12b8c 100644 > --- a/src/IO/tls_openssl.c > +++ b/src/IO/tls_openssl.c > @@ -1186,7 +1186,10 @@ static void Tls_connect(int fd, int connkey) > if (a_Klist_get_data(conn_list, connkey)) { > conn->connecting = FALSE; > if (failed) { > + conn->in_connect = FALSE; > Tls_close_by_key(connkey); > + /* conn is freed now */ > + conn = NULL; > } > a_IOwatch_remove_fd(fd, DIO_READ|DIO_WRITE); > a_Http_connect_done(fd, failed ? FALSE : TRUE); > @@ -1195,7 +1198,8 @@ static void Tls_connect(int fd, int connkey) > } > } > > - conn->in_connect = FALSE; > + if (conn) > + conn->in_connect = FALSE; > } > > static void Tls_connect_cb(int fd, void *vconnkey) I have confirmed that this patch prevents Dillo from crashing. Thanks! > However, it doesn't explain the EOF you are getting in narkive. Are > you using tor or some proxy? No vpn/tor/proxy. I wouldn't be surprised if my network provider was doing some weird stuff though... Maybe thats an area for further study. -Alex

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by Rodrigo Arias

Hi, >This is enough for me to at least fix this case on Dillo side. I see, I was trying to set the conn->in_connect = FALSE after conn was freed. Try this patch: diff --git a/src/IO/tls_openssl.c b/src/IO/tls_openssl.c index 3e68e928..5ad12b8c 100644 --- a/src/IO/tls_openssl.c +++ b/src/IO/tls_openssl.c @@ -1186,7 +1186,10 @@ static void Tls_connect(int fd, int connkey) if (a_Klist_get_data(conn_list, connkey)) { conn->connecting = FALSE; if (failed) { + conn->in_connect = FALSE; Tls_close_by_key(connkey); + /* conn is freed now */ + conn = NULL; } a_IOwatch_remove_fd(fd, DIO_READ|DIO_WRITE); a_Http_connect_done(fd, failed ? FALSE : TRUE); @@ -1195,7 +1198,8 @@ static void Tls_connect(int fd, int connkey) } } - conn->in_connect = FALSE; + if (conn) + conn->in_connect = FALSE; } static void Tls_connect_cb(int fd, void *vconnkey)

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by Rodrigo Arias

Hi, On Wed, Aug 28, 2024 at 08:14:20PM +0200, a1ex(a)dismail.de wrote: >Hi, > >On Wed, 28 Aug 2024 19:12:43 +0200 >Rodrigo Arias <rodarima(a)gmail.com> wrote: > >> You may be able to get some info on where it is failing by building >> Dillo with the address sanitizer enabled: >> >> $ git clone https://github.com/dillo-browser/dillo >> $ cd dillo >> $ ./autogen.sh >> $ mkdir build >> $ cd build >> $ ../configure 'CFLAGS=-Og -g -fsanitize=address' 'CXXFLAGS=-Og -g >> -fsanitize=address' $ make >> $ src/dillo https://narkive.com > >That doesn't seem to work: > >$ ../configure 'CFLAGS=-Og -g -fsanitize=address' 'CXXFLAGS=-Og -g >-fsanitize=address' >checking build system type... x86_64-unknown-openbsd7.6 >[...] >configure: error: C compiler cannot create executables >See `config.log' for more details Oh, I didn't knew it was not supported in OpenBSD. Well, I added a simple patch to reproduce your problem, and I got a heap use after free with asan: hop% LD_LIBRARY_PATH=/usr/lib/libressl src/dillo https://narkive.com dillo_dns_init: Here we go! (threaded) TLS library: LibreSSL 3.9.2 Enabling cookies as from cookiesrc... Nav_open_url: new url='https://narkive.com' Dns_server [0]: narkive.com is 149.248.211.108 Connecting to 149.248.211.108:443 TLS connect error: "an EOF was observed that violates the protocol" Tls_close_by_key: Avoiding SSL shutdown for: https://narkive.com fd 6 is done and failed ================================================================= ==1646866==ERROR: AddressSanitizer: heap-use-after-free on address 0x5030002c8649 at pc 0x64733a019b37 bp 0x7ffd6d8bfe00 sp 0x7ffd6d8bfdf0 WRITE of size 1 at 0x5030002c8649 thread T0 #0 0x64733a019b36 in Tls_connect IO/../../../src/IO/tls_openssl.c:1201 #1 0x64733a019d88 in a_Tls_openssl_connect IO/../../../src/IO/tls_openssl.c:1266 #2 0x64733a019e9a in a_Tls_connect IO/../../../src/IO/tls.c:133 #3 0x64733a01a10d in Http_connect_tls IO/../../../src/IO/http.c:521 #4 0x64733a01b7c4 in Http_connect_socket_cb IO/../../../src/IO/http.c:555 #5 0x769620b7c1f3 in fl_wait(double) (/usr/lib/libfltk.so.1.3+0xa51f3) (BuildId: 80ecc607ef6caa03d5591ba84ff58945ff1be889) #6 0x769620b1b09a in Fl::wait(double) (/usr/lib/libfltk.so.1.3+0x4409a) (BuildId: 80ecc607ef6caa03d5591ba84ff58945ff1be889) #7 0x769620b1b15d in Fl::run() (/usr/lib/libfltk.so.1.3+0x4415d) (BuildId: 80ecc607ef6caa03d5591ba84ff58945ff1be889) #8 0x647339fd912c in main ../../src/dillo.cc:583 #9 0x769620039c87 (/usr/lib/libc.so.6+0x25c87) (BuildId: 32a656aa5562eece8c59a585f5eacd6cf5e2307b) #10 0x769620039d4b in __libc_start_main (/usr/lib/libc.so.6+0x25d4b) (BuildId: 32a656aa5562eece8c59a585f5eacd6cf5e2307b) #11 0x647339fb8894 in _start (/home/ram/dev/dillo/git/build-libressl/src/dillo+0xe1894) (BuildId: 83bcb2b57d527264c53219ed2204e01b019e3e34) 0x5030002c8649 is located 25 bytes inside of 32-byte region [0x5030002c8630,0x5030002c8650) freed by thread T0 here: #0 0x769620cfb422 in free /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_malloc_linux.cpp:52 #1 0x64733a083ca3 in dFree ../../dlib/dlib.c:70 #2 0x64733a019782 in Tls_connect IO/../../../src/IO/tls_openssl.c:1192 #3 0x64733a019d88 in a_Tls_openssl_connect IO/../../../src/IO/tls_openssl.c:1266 #4 0x64733a019e9a in a_Tls_connect IO/../../../src/IO/tls.c:133 #5 0x64733a01a10d in Http_connect_tls IO/../../../src/IO/http.c:521 #6 0x64733a01b7c4 in Http_connect_socket_cb IO/../../../src/IO/http.c:555 #7 0x769620b7c1f3 in fl_wait(double) (/usr/lib/libfltk.so.1.3+0xa51f3) (BuildId: 80ecc607ef6caa03d5591ba84ff58945ff1be889) #8 0x769620b1b09a in Fl::wait(double) (/usr/lib/libfltk.so.1.3+0x4409a) (BuildId: 80ecc607ef6caa03d5591ba84ff58945ff1be889) #9 0x769620b1b15d in Fl::run() (/usr/lib/libfltk.so.1.3+0x4415d) (BuildId: 80ecc607ef6caa03d5591ba84ff58945ff1be889) #10 0x647339fd912c in main ../../src/dillo.cc:583 #11 0x769620039c87 (/usr/lib/libc.so.6+0x25c87) (BuildId: 32a656aa5562eece8c59a585f5eacd6cf5e2307b) #12 0x769620039d4b in __libc_start_main (/usr/lib/libc.so.6+0x25d4b) (BuildId: 32a656aa5562eece8c59a585f5eacd6cf5e2307b) #13 0x647339fb8894 in _start (/home/ram/dev/dillo/git/build-libressl/src/dillo+0xe1894) (BuildId: 83bcb2b57d527264c53219ed2204e01b019e3e34) previously allocated by thread T0 here: #0 0x769620cfca31 in malloc /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_malloc_linux.cpp:69 #1 0x64733a083c3a in dMalloc ../../dlib/dlib.c:47 SUMMARY: AddressSanitizer: heap-use-after-free IO/../../../src/IO/tls_openssl.c:1201 in Tls_connect Shadow bytes around the buggy address: 0x5030002c8380: fa fa fd fd fd fa fa fa fd fd fd fd fa fa fd fd 0x5030002c8400: fd fd fa fa 00 00 00 00 fa fa fd fd fd fd fa fa 0x5030002c8480: fd fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd 0x5030002c8500: fa fa 00 00 00 00 fa fa 00 00 00 00 fa fa 00 00 0x5030002c8580: 00 fa fa fa 00 00 00 00 fa fa 00 00 00 00 fa fa =>0x5030002c8600: 00 00 00 fa fa fa fd fd fd[fd]fa fa fd fd fd fa 0x5030002c8680: fa fa fd fd fd fa fa fa 00 00 00 fa fa fa 00 00 0x5030002c8700: 00 00 fa fa 00 00 00 00 fa fa 00 00 00 fa fa fa 0x5030002c8780: fd fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd 0x5030002c8800: fa fa fd fd fd fd fa fa fd fd fd fd fa fa fd fd 0x5030002c8880: fd fd fa fa fd fd fd fd fa fa fd fd fd fd fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==1646866==ABORTING Hopefully is the same you are seeing. Here is the patch I'm using: diff --git a/src/IO/tls_openssl.c b/src/IO/tls_openssl.c index 3e68e928..05d795a5 100644 --- a/src/IO/tls_openssl.c +++ b/src/IO/tls_openssl.c @@ -1108,8 +1108,11 @@ static void Tls_connect(int fd, int connkey) a_IOwatch_remove_fd(fd, -1); + ret = 0; + if (ret <= 0) { int err1_ret = SSL_get_error(conn->ssl, ret); + err1_ret = SSL_ERROR_SYSCALL; if (err1_ret == SSL_ERROR_WANT_READ || err1_ret == SSL_ERROR_WANT_WRITE) { int want = err1_ret == SSL_ERROR_WANT_READ ? DIO_READ : DIO_WRITE; This is enough for me to at least fix this case on Dillo side. However, it doesn't explain the EOF you are getting in narkive. Are you using tor or some proxy? Best, Rodrigo.

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by a1ex＠dismail.de

On Wed, 28 Aug 2024 20:22:37 +0200 Theo Buehler <tb(a)theobuehler.org> wrote: > > dillo(76315) in malloc(): write to free chunk > > 0x1957a5e82c0[24..31]@32 allocated at /usr/lib/libc++.so.10.0 > > 0x8b918 (preceding chunk 0x1957a5e82a0 allocated at > > /usr/local/lib/libcrypto.so.53.0 0x1a7f51) > > what's the output of > > addr2line -e /usr/lib/libc++.so.10.0 0x8b918 /usr/src/gnu/lib/libcxx/../../../gnu/llvm/libcxx/src/new.cpp:67 > and > > addr2line -e /usr/local/lib/libcrypto.so.53.0 0x1a7f51 stack/stack.c:128 I am also including part of a ktrace of the crash, just in case that would be useful: 57506 dillo GIO fd 1 wrote 34 bytes "Connecting to 149.248.211.108:443 " 57506 dillo RET write 34/0x22 57506 dillo CALL connect(6,0x759b98292280,16) 57506 dillo STRU struct sockaddr { AF_INET, 149.248.211.108:443 } 57506 dillo RET connect -1 errno 36 Operation now in progress 57506 dillo CALL futex(0xc4d868bc00,0x82<FUTEX_WAKE|FUTEX_PRIVATE_FLAG>,2147483647,0,0) 57506 dillo RET futex 0 57506 dillo CALL sigprocmask(SIG_BLOCK,0<>) 57506 dillo RET sigprocmask 0<> 57506 dillo CALL poll(0x759b982923c8,1,INFTIM) 57506 dillo STRU struct pollfd { fd=5, events=0x5<POLLIN|POLLOUT>, revents=0x5<POLLIN|POLLOUT> } 57506 dillo RET poll 1 57506 dillo CALL recvmsg(5,0x759b98292280,0) 57506 dillo GIO fd 5 read 32 bytes "\^N\0l\b\^N\0\M^@\^B\0\0>\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0" 57506 dillo STRU struct msghdr { name=0x0, namelen=0, iov=0x759b98292270, iovlen=1, control=0x759b98292220, controllen=0, flags=0x80<MSG_DONTWAIT> } 57506 dillo STRU struct iovec { base=0xc554d1a04c, len=4064 } 57506 dillo RET recvmsg 32/0x20 57506 dillo CALL futex(0xc4d86b3ef0,0x82<FUTEX_WAKE|FUTEX_PRIVATE_FLAG>,1,0,0) 57506 dillo RET futex 0 57506 dillo CALL writev(5,0x759b98292470,3) 57506 dillo STRU struct iovec [3] { base=0xc53d099000, len=504 } { base=0x0, len=0 } { base=0xc504b50156, len=0 } 57506 dillo GIO fd 5 wrote 504 bytes "8\^B\^E\0\^E\0\M^@\^B\^D\0\b\0\M^Y\M^Y\M^Y\0\0\0\0\0;\^C\^E\0\^E\0\M^@\^B\0\0\0\0\ \0\0\M-1\^C\M-k\^D\^T\0;\^C\^E\0\^E\0\M^@\^B\0\0\0\0\^A\0\M-2\^C\M-i\^D\^R\0;\^C\ \^E\0\^E\0\M^@\^B\0\0\0\0\^A\0\M-2\^C\M-i\^D\^R\08\0\^D\0\^E\0\M^@\^B\^D\0\0\0qqq\ \0B\0\a\0\^N\0\M^@\^B\^E\0\M^@\^B\0\0\M-D\^C\M-j\^D\M-D\^C\M-j\^D\M-C\^C\M-j\^D\ \M-1\^C8\M^@\^D\0\^E\0\M^@\^B\^D\0\0\0\0\0\0\0B\0\a\0\^N\0\M^@\^B\^E\0\M^@\^B\0\0\ \M-1\^C\M-i\^D\M-1\^C\0\0\M-C\^C\0\0\M-2\^C8C\^D\0\^E\0\M^@\^B\^D\0\0\0\^A\^A\^A\0\ F\0\^E\0\^N\0\M^@\^B\^E\0\M^@\^B\^A\0\M-2\^C\M-i\^D\^R\0\M^K\a\^B\0\a\^A\M^@\^B\ \M^K\^D\^E\0\b\^A\M^@\^B\^N\0\M^@\^B)\0\0\0\0\0\0\0\M^K\^F\^E\0\b\^A\M^@\^B\0\0\0\ \0\^A\0\M-2\^C\M-i\^D\^R\0\M^K\^T\^_\0\^B\0\M^@\^B\^A\0\0\0&\0\0\0\b\0\f\0\0\0\v\0\ \0\0\0\0\0\0\0\^E\0\0\0\0\0i\M-]\M-~\M-fz\^B\0d\M-nY W\M-pn\0\M-\o\0\0\0z\ \M-L\b\M-~0\0\0\0\^XI\^_\M^?\^\\0\0\0\0\0\^_\M^?\^\\0\0\0\0\0\r\M^?+\0\0\0\0\0\0\ \M-g_\0\0\0`\M-T\0z\M-`0\0%\M-X\M^I\0\^F\M^R\M-}\M-v\M^?\M-# \0\0\0 ,\^O\0\0\ \M^K\^X\^R\0\^C\M-U\M-b\0(\0\M^@\^B\b\^A\M^@\^B$\0\0\0\^B\0\M^@\^B\0\0\0\0\^R\0\0\ \0\^B\0\M-@\^C&\0t\^Bj\^B\M-2\^B\M-d\^A\^F\^B\M-2\^BC\^Bj\^B1\^B\^C\09\^Bt\^B\M-'\ \^B\M-2\^BP\^DP\^DP\^D;\^C\^E\0\^E\0\M^@\^B\0\0\0\0\^A\0\M-2\^C\M-i\^D\^R\0;\^C\^E\ \0\^E\0\M^@\^B\0\0\0\0\0\0\M-1\^C\M-k\^D\^T\0\M^Q\^C\^D\0\^A\0\0\0\^D\0\M^@\^B\^C\ \0\0\0" 57506 dillo RET writev 504/0x1f8 57506 dillo CALL futex(0xc4d86ad1a0,0x82<FUTEX_WAKE|FUTEX_PRIVATE_FLAG>,2147483647,0,0) 57506 dillo RET futex 0 57506 dillo CALL futex(0xc4d86b3ef0,0x82<FUTEX_WAKE|FUTEX_PRIVATE_FLAG>,1,0,0) 57506 dillo RET futex 0 57506 dillo CALL recvmsg(5,0x759b98292370,0) 57506 dillo RET recvmsg -1 errno 35 Resource temporarily unavailable 57506 dillo CALL recvmsg(5,0x759b982920d0,0) 57506 dillo RET recvmsg -1 errno 35 Resource temporarily unavailable 57506 dillo CALL recvmsg(5,0x759b98292370,0) 57506 dillo RET recvmsg -1 errno 35 Resource temporarily unavailable 57506 dillo CALL select(7,0x759b98292390,0x759b98292410,0x759b98292490,0) 57506 dillo STRU struct fd_set { 3,5 } 57506 dillo STRU struct fd_set { 6 } 57506 dillo STRU struct fd_set { } 57506 dillo STRU struct fd_set { } 57506 dillo STRU struct fd_set { 6 } 57506 dillo STRU struct fd_set { } 57506 dillo RET select 1 57506 dillo CALL kbind(0x759b98292288,24,0xb28d566424727b4) 57506 dillo RET kbind 0 57506 dillo CALL getsockopt(6,SOL_SOCKET,SO_ERROR,0x759b98292358,0x759b9829235c) 57506 dillo RET getsockopt 0 57506 dillo CALL kbind(0x759b98292248,24,0xb28d566424727b4) 57506 dillo RET kbind 0 57506 dillo CALL kbind(0x759b98292248,24,0xb28d566424727b4) 57506 dillo RET kbind 0 57506 dillo CALL kbind(0x759b98292248,24,0xb28d566424727b4) 57506 dillo RET kbind 0 57506 dillo CALL kbind(0x759b98292218,24,0xb28d566424727b4) 57506 dillo RET kbind 0 57506 dillo CALL kbind(0x759b98292248,24,0xb28d566424727b4) 57506 dillo RET kbind 0 57506 dillo CALL kbind(0x759b982911d8,24,0xb28d566424727b4) 57506 dillo RET kbind 0 57506 dillo CALL mmap(0,0x5000,0x3<PROT_READ|PROT_WRITE>,0x1002<MAP_PRIVATE|MAP_ANON>,-1,0) 57506 dillo RET mmap 846576910336/0xc51bea8000 57506 dillo CALL write(6,0xc48c3b0640,0x123) 57506 dillo GIO fd 6 wrote 291 bytes "\^V\^C\^C\^A\^^\^A\0\^A\^Z\^C\^C\M-=\^[j 2\M-r_\M-5\M-Z*|^Bm\M-+\M-I\M-LD\M^G]-e\^Xa\M^H\M-0\^]6\M-J'g\M-k \M^^\M-n\M-S\M-D\ \M-B\M-}\^QWyqV\M-d,\M^K\M^L\M-/\M-l\f< \M--~\M-ax\^V,^`\M^Y\M-FV\M-]\0V\^S\^B\^S\^C\^S\^A\M-@0\M-@,\M-@(\M-@$\M-@\^T\M-@ \0\M^_\0k\09\M-L\M-)\M-L\M-(\M-L\M-*\0\M-D\0\M^H\0\M^]\0=\0005\0\M-@\0\M^D\M-@/\ \M-@+\M-@'\M-@#\M-@\^S\M-@ \0\M^^\0g\0003\0\M->\0E\0\M^\\0<\0/\0\M-:\0A\M-@\ \^R\M-@\b\0\^V\0 \0\M^?\^A\0\0{\0\0\0\^P\0\^N\0\0\vnarkive.com\0\v\0\^B\^A\0\0\r\0\^X\0\^V\b\^F\^F\ \^A\^F\^C\b\^E\^E\^A\^E\^C\b\^D\^D\^A\^D\^C\^B\^A\^B\^C\0 \0 \0\b\0\^]\0\^W\0\^X\0\^Y\0003\0&\0$\0\^]\0 \^O\M-+\r\M-k$\M-f\^\\f]K\M-Fr\M->&\M-s\ a \M^Y\M-0\M-]F`NU\M^X(\^XT\M-MTQ5\0+\0\^E\^D\^C\^D\^C\^C\0#\0\0" 57506 dillo RET write 291/0x123 57506 dillo CALL read(6,0xc48c3a58b0,0x5) 57506 dillo RET read -1 errno 35 Resource temporarily unavailable 57506 dillo CALL kbind(0x759b982911d8,24,0xb28d566424727b4) 57506 dillo RET kbind 0 57506 dillo CALL futex(0xc4d868bc00,0x82<FUTEX_WAKE|FUTEX_PRIVATE_FLAG>,2147483647,0,0) 57506 dillo RET futex 0 57506 dillo CALL sigprocmask(SIG_BLOCK,0<>) 57506 dillo RET sigprocmask 0<> 57506 dillo CALL poll(0x759b982923c8,1,INFTIM) 57506 dillo STRU struct pollfd { fd=5, events=0x5<POLLIN|POLLOUT>, revents=0x4<POLLOUT> } 57506 dillo RET poll 1 57506 dillo CALL writev(5,0x759b98292470,3) 57506 dillo STRU struct iovec [3] { base=0xc53d099000, len=508 } { base=0x0, len=0 } { base=0xc504b50156, len=0 } 57506 dillo GIO fd 5 wrote 508 bytes "8\^B\^E\0\^E\0\M^@\^B\^D\0\b\0\M^Y\M^Y\M^Y\0\0\0\0\0;\^C\^E\0\^E\0\M^@\^B\0\0\0\0\ \0\0\M-1\^C\M-k\^D\^T\0;\^C\^E\0\^E\0\M^@\^B\0\0\0\0\^A\0\M-2\^C\M-i\^D\^R\0;\^C\ \^E\0\^E\0\M^@\^B\0\0\0\0\^A\0\M-2\^C\M-i\^D\^R\08\0\^D\0\^E\0\M^@\^B\^D\0\0\0qqq\ \0B\0\a\0\^N\0\M^@\^B\^E\0\M^@\^B\0\0\M-D\^C\M-j\^D\M-D\^C\M-j\^D\M-C\^C\M-j\^D\ \M-1\^C8\M^@\^D\0\^E\0\M^@\^B\^D\0\0\0\0\0\0\0B\0\a\0\^N\0\M^@\^B\^E\0\M^@\^B\0\0\ \M-1\^C\M-i\^D\M-1\^C\0\0\M-C\^C\0\0\M-2\^C8C\^D\0\^E\0\M^@\^B\^D\0\0\0\^A\^A\^A\0\ F\0\^E\0\^N\0\M^@\^B\^E\0\M^@\^B\^A\0\M-2\^C\M-i\^D\^R\0\M^K\a\^B\0\b\^A\M^@\^B\ \M^K\^D\^E\0 \^A\M^@\^B\^N\0\M^@\^B)\0\0\0\0\0\0\0\M^K\^F\^E\0 \^A\M^@\^B\ \0\0\0\0\^A\0\M-2\^C\M-i\^D\^R\0\M^K\^T\^Y\0\^B\0\M^@\^B\^A\0\0\0\M-<\^B\0\0\a\0\ \0\M^?\M^?\b\0 \0\0\0J/\0\0\0T%\0\M-7s\0\0\0\M-O[\0\M-7s\0\0\0\M-O[\0\M-7\ s\0\0\0\M-O[\0\M-7s\0\0\0\M-P[\0\M-2\^?\0\0\^A\M-i[\0\M^H\M-S*\^Dy\M-q[\0\^W\M-O\ \M^?\M-kp\M-C[\0\0\0\^U\^C\0\0\0\0\M^K\^X\^Y\0\^C%\M-X\M^I(\0\M^@\^B \^A\M^@\^B\ $\0\0\0\^B\0\M^@\^B\0\0\0\0 \0\0\0\^B\0\M-@\^C\M-F\0\^U\^B\^F\^B\M-<\^B\M^]\^B\^U\ \^B\^C\0\^F\^Bt\^Bj\^Bj\^B\^U\^B\^F\^B\M-2\^BC\^Bt\^Bj\^B\^C\0j\^B\^U\^B1\^Bt\^B\ \M-2\^BC\^B\M-d\^A\M-2\^BC\^Bt\^Bj\^BP\^DP\^DP\^D;\^C\^E\0\^E\0\M^@\^B\0\0\0\0\^A\ \0\M-2\^C\M-i\^D\^R\0;\^C\^E\0\^E\0\M^@\^B\0\0\0\0\0\0\M-1\^C\M-k\^D\^T\0\M^Q\^C\ \^D\0\^A\0\0\0\^D\0\M^@\^B\^C\0\0\0" 57506 dillo RET writev 508/0x1fc 57506 dillo CALL futex(0xc4d86ad1a0,0x82<FUTEX_WAKE|FUTEX_PRIVATE_FLAG>,2147483647,0,0) 57506 dillo RET futex 0 57506 dillo CALL futex(0xc4d86b3ef0,0x82<FUTEX_WAKE|FUTEX_PRIVATE_FLAG>,1,0,0) 57506 dillo RET futex 0 57506 dillo CALL recvmsg(5,0x759b98292370,0) 57506 dillo RET recvmsg -1 errno 35 Resource temporarily unavailable 57506 dillo CALL select(7,0x759b98292390,0x759b98292410,0x759b98292490,0) 57506 dillo STRU struct fd_set { 3,5,6 } 57506 dillo STRU struct fd_set { } 57506 dillo STRU struct fd_set { } 57506 dillo STRU struct fd_set { 5 } 57506 dillo STRU struct fd_set { } 57506 dillo STRU struct fd_set { } 57506 dillo RET select 1 57506 dillo CALL recvmsg(5,0x759b982920d0,0) 57506 dillo GIO fd 5 read 32 bytes "\^C$\M-/\bZ\^C\0\^A\M-G\^B\0\0\^D\0\M^@\^B\0\0\0\0j\^Fi\0 \^A\^R\0\0\0\^A\0" 57506 dillo STRU struct msghdr { name=0x0, namelen=0, iov=0x759b982920c0, iovlen=1, control=0x759b98292070, controllen=0, flags=0x80<MSG_DONTWAIT> } 57506 dillo STRU struct iovec { base=0xc554d1a04c, len=4064 } 57506 dillo RET recvmsg 32/0x20 57506 dillo CALL futex(0xc4d86b3ef0,0x82<FUTEX_WAKE|FUTEX_PRIVATE_FLAG>,1,0,0) 57506 dillo RET futex 0 57506 dillo CALL recvmsg(5,0x759b982920d0,0) 57506 dillo RET recvmsg -1 errno 35 Resource temporarily unavailable 57506 dillo CALL recvmsg(5,0x759b98291ed0,0) 57506 dillo RET recvmsg -1 errno 35 Resource temporarily unavailable 57506 dillo CALL recvmsg(5,0x759b982920d0,0) 57506 dillo RET recvmsg -1 errno 35 Resource temporarily unavailable 57506 dillo CALL recvmsg(5,0x759b98292370,0) 57506 dillo RET recvmsg -1 errno 35 Resource temporarily unavailable 57506 dillo CALL select(7,0x759b98292390,0x759b98292410,0x759b98292490,0) 57506 dillo STRU struct fd_set { 3,5,6 } 57506 dillo STRU struct fd_set { } 57506 dillo STRU struct fd_set { } 57506 dillo STRU struct fd_set { 6 } 57506 dillo STRU struct fd_set { } 57506 dillo STRU struct fd_set { } 57506 dillo RET select 1 57506 dillo CALL read(6,0xc48c3a58b0,0x5) 57506 dillo RET read 0 57506 dillo CALL write(1,0xc4e11d5000,0x44) 57506 dillo GIO fd 1 wrote 68 bytes "TLS connect error: "an EOF was observed that violates the protocol" " 57506 dillo RET write 68/0x44 57506 dillo CALL write(1,0xc4e11d5000,0x41) 57506 dillo GIO fd 1 wrote 65 bytes "Tls_close_by_key: Avoiding SSL shutdown for: https://narkive.com " 57506 dillo RET write 65/0x41 57506 dillo CALL kbind(0x759b982911d8,24,0xb28d566424727b4) 57506 dillo RET kbind 0 57506 dillo CALL write(1,0xc4e11d5000,0x18) 57506 dillo GIO fd 1 wrote 24 bytes "fd 6 is done and failed " 57506 dillo RET write 24/0x18 57506 dillo CALL close(6) 57506 dillo RET close 0 57506 dillo CALL getpid() 57506 dillo RET getpid 57506/0xe0a2 57506 dillo CALL write(2,0x759b98291bf0,0x18) 57506 dillo GIO fd 2 wrote 24 bytes "dillo(57506) in free(): " 57506 dillo RET write 24/0x18 57506 dillo CALL write(2,0x759b98291cd0,0x29) 57506 dillo GIO fd 2 wrote 41 bytes "write to free mem 0xc48c3dbd80[24..31]@32" 57506 dillo RET write 41/0x29 57506 dillo CALL write(2,0x759b98291bf0,0x1) 57506 dillo GIO fd 2 wrote 1 bytes " " 57506 dillo RET write 1 57506 dillo CALL sigprocmask(SIG_SETMASK,~0x20<SIGABRT>) 57506 dillo RET sigprocmask 0<> 57506 dillo CALL thrkill(0,SIGABRT,0) 57506 dillo PSIG SIGABRT SIG_DFL 57506 dillo NAMI "dillo.core"

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by Theo Buehler

On Wed, Aug 28, 2024 at 08:14:20PM +0200, a1ex(a)dismail.de wrote: > Hi, > > On Wed, 28 Aug 2024 19:12:43 +0200 > Rodrigo Arias <rodarima(a)gmail.com> wrote: > > > You may be able to get some info on where it is failing by building > > Dillo with the address sanitizer enabled: > > > > $ git clone https://github.com/dillo-browser/dillo > > $ cd dillo > > $ ./autogen.sh > > $ mkdir build > > $ cd build > > $ ../configure 'CFLAGS=-Og -g -fsanitize=address' 'CXXFLAGS=-Og -g > > -fsanitize=address' $ make > > $ src/dillo https://narkive.com > > That doesn't seem to work: > > $ ../configure 'CFLAGS=-Og -g -fsanitize=address' 'CXXFLAGS=-Og -g > -fsanitize=address' > checking build system type... x86_64-unknown-openbsd7.6 > [...] > configure: error: C compiler cannot create executables > See `config.log' for more details yes, asan isn't supported on openbsd. > dillo(76315) in malloc(): write to free chunk 0x1957a5e82c0[24..31]@32 > allocated at /usr/lib/libc++.so.10.0 0x8b918 (preceding chunk > 0x1957a5e82a0 allocated at /usr/local/lib/libcrypto.so.53.0 0x1a7f51) what's the output of addr2line -e /usr/lib/libc++.so.10.0 0x8b918 and addr2line -e /usr/local/lib/libcrypto.so.53.0 0x1a7f51 on your system?

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by a1ex＠dismail.de

Hi, On Wed, 28 Aug 2024 19:12:43 +0200 Rodrigo Arias <rodarima(a)gmail.com> wrote: > You may be able to get some info on where it is failing by building > Dillo with the address sanitizer enabled: > > $ git clone https://github.com/dillo-browser/dillo > $ cd dillo > $ ./autogen.sh > $ mkdir build > $ cd build > $ ../configure 'CFLAGS=-Og -g -fsanitize=address' 'CXXFLAGS=-Og -g > -fsanitize=address' $ make > $ src/dillo https://narkive.com That doesn't seem to work: $ ../configure 'CFLAGS=-Og -g -fsanitize=address' 'CXXFLAGS=-Og -g -fsanitize=address' checking build system type... x86_64-unknown-openbsd7.6 [...] configure: error: C compiler cannot create executables See `config.log' for more details config.log: [...] configure:4670: checking whether the C compiler works configure:4692: cc -Og -g -fsanitize=address -I/usr/local/include -I/usr/X11R6/include -L/usr/local/lib -L/usr/X11R6/lib conftest.c >&5 cc: error: unsupported option '-fsanitize=address' for target 'amd64-unknown-openbsd7.6' [...] I also tried it with gcc, but also no luck: configure:4670: checking whether the C compiler works configure:4692: egcc -Og -g -fsanitize=address -I/usr/local/include -I/usr/X11R6/include -L/usr/local/lib -L/usr/X11R6/lib conftest.c >&5 ld: error: unable to find library -lasan Then I tried this: $ MALLOC_OPTIONS=D dillo result: Nav_open_url: new url='https://narkive.com' Dns_server [0]: narkive.com is 149.248.211.108 Connecting to 149.248.211.108:443 TLS connect error: "an EOF was observed that violates the protocol" Tls_close_by_key: Avoiding SSL shutdown for: https://narkive.com fd 6 is done and failed dillo(76315) in malloc(): write to free chunk 0x1957a5e82c0[24..31]@32 allocated at /usr/lib/libc++.so.10.0 0x8b918 (preceding chunk 0x1957a5e82a0 allocated at /usr/local/lib/libcrypto.so.53.0 0x1a7f51) Abort trap (core dumped) So now we have a reference to libcrypto. Here is the backtrace, also with some new info: #0 thrkill () at /tmp/-:2 No locals. #1 0xf3ab3815f99febfa in ?? () No symbol table info available. #2 0x000001956d60a5ab in _libc_abort () at /usr/src/lib/libc/stdlib/abort.c:51 sa = {__sigaction_u = {__sa_handler = 0x3000000010, __sa_sigaction = 0x3000000010}, sa_mask = 3661176352, sa_flags = 31051} mask = 4294967263 #3 0x000001956d5d93e4 in wrterror (d=0x195325a97a8, msg=0x1956d53c0ad "write to free chunk %p[%zu..%zu]@%zu allocated at %s (preceding chunk %p allocated at %s%s)") at /usr/src/lib/libc/stdlib/malloc.c:378 ap = {{gp_offset = 48, fp_offset = 48, overflow_arg_area = 0x794bda391f40, reg_save_area = 0x794bda391e20}} saved_errno = 35 #4 0x000001956d5df052 in print_chunk_details (pool=0x195325a97a8, p=0x1957a5e82c0, sz=32, index=<optimized out>) at /usr/src/lib/libc/stdlib/malloc.c:2433 buf1 = "/usr/lib/libc++.so.10.0 0x8b918\000\310pam\225\001\000\000\300\337Z1\225\001\000\000\310pam\225\001\000\000\000\000\000\000\000\000\000\000x\322\020\027\225\001\000\000\240\371`z\225\001\000\000\240\371`z\225\001\000\000\250\227Z2\225\001\000\000\000\000\000\000\000\000\000\000IT\327|\370V\347W0 9\332Ky\000\000\000\000\000\000\000\000\000" buf2 = "/usr/local/lib/libcrypto.so.53.0 0x1a7f51", '\000' <repeats 16 times>, "\220c=\225\001\000\000\300\364`z\225\001\000\000\b\000\000\000\000\000\000\000@h=\366\224\001\000\000\272\fе\224\351\327]\310pam\225\001\000\000\253\305Sm\225\001\000\000\270tam\225\001\0-00\000\000\000\000\000\000\000\000" msg = <optimized out> r = <optimized out> chunkinfo = 0x195576c5f88 chunknum = <optimized out> btnode = <optimized out> frame = 0 #5 0x000001956d5da1dd in validate_junk (pool=0x195325a97a8, p=0x1957a5e82c0, argsz=0) at /usr/src/lib/libc/stdlib/malloc.c:777 step = <optimized out> lp = 0x1957a5e82c0 sz = <optimized out> i = 1741296686955 #6 malloc_bytes (d=0x195325a97a8, size=32) at /usr/src/lib/libc/stdlib/malloc.c:1237 bucket = <optimized out> r = <optimized out> listnum = <optimized out> bp = 0x195576c5f88 i = <optimized out> j = <optimized out> lp = <optimized out> k = <optimized out> p = 0x1957a5e82c0 r = <optimized out> #7 omalloc (pool=0x195325a97a8, sz=32, zero_fill=0) at /usr/src/lib/libc/stdlib/malloc.c:1391 caller = <error reading variable caller (Cannot access memory at address 0x0)> psz = <optimized out> p = <optimized out> #8 0x000001956d5d9689 in _libc_malloc (size=32) at /usr/src/lib/libc/stdlib/malloc.c:1545 saved_errno = 35 d = 0x195325a97a8 r = <optimized out> #9 0x000001956bbc0918 in operator new (size=32) at /usr/src/gnu/lib/libcxx/../../../gnu/llvm/libcxx/src/new.cpp:67 p = <optimized out> #10 0x00000192ef94bfb1 in lout::container::typed::HashSet<lout::object::TypedPointer<dw::core::Widget> >::HashSet(bool, int) () No symbol table info available. #11 >0x00000192ef94bec5 in >dw::core::StackingProcessingContext::StackingProcessingContext() () No symbol table info available. #12 0x00000192ef94bb13 in dw::core::GettingWidgetAtPointContext::GettingWidgetAtPointContext() () No symbol table info available. #13 0x00000192ef94a6c2 in dw::core::Layout::getWidgetAtPoint(int, int) () No symbol table info available. #14 0x00000192ef94b989 in dw::core::Layout::moveToWidgetAtPoint(int, int, dw::core::ButtonState) () No symbol table info available. #15 0x00000192ef94a4fe in dw::core::Layout::motionNotify(dw::core::View*, int, int, dw::core::ButtonState) () No symbol table info available. #16 0x00000192ef93fa0d in dw::fltk::FltkViewBase::handle(int) () No symbol table info available. #17 0x00000192ef8b792c in UI::handle(int) () No symbol table info available. #18 0x0000019517063404 in send(int, Fl_Widget*, Fl_Window*) () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #19 0x00000195170630da in Fl::handle_(int, Fl_Window*) () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #20 0x00000195170d1f7c in do_queued_events() () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #21 0x00000195170d1e88 in fl_wait(double) () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #22 0x0000019517061a6e in Fl::wait(double) () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #23 0x0000019517061c4d in Fl::run() () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #24 0x00000192ef8b3f95 in main () No symbol table info available. Hope this helps! Thanks, Alex

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by Rodrigo Arias

Hi, I think there are two problems here: 1) The narkive server is sending you an EOF but not us. 2) When that happens Dillo crashes. You may be able to get some info on where it is failing by building Dillo with the address sanitizer enabled: $ git clone https://github.com/dillo-browser/dillo $ cd dillo $ ./autogen.sh $ mkdir build $ cd build $ ../configure 'CFLAGS=-Og -g -fsanitize=address' 'CXXFLAGS=-Og -g -fsanitize=address' $ make $ src/dillo https://narkive.com Then send *all the lines of the output*. I may be able to emulate an EOF and crash my own Dillo just to reproduce the issue with this information. To debug the TLS traffic you may want to get a network trace with Wireshark so we can compare it. Best, Rodrigo. On Wed, Aug 28, 2024 at 05:45:37PM +0200, a1ex(a)dismail.de wrote: >Hi, > >On Wed, 28 Aug 2024 16:09:25 +0200 >Theo Buehler <tb(a)theobuehler.org> wrote: > >> > Here is the output of 'bt full': >> >> Thanks, that doesn't help us further as such. >> >> I have enabled DEBUG_PACKAGES in dillo, which unfortunately wasn't >> done before. Once the debug-dillo package is available (will be a few >> days), we might get some more insight from a backtrace with symbols... >> >> If you have a ports tree set up (with doas, etc) you can build and >> install it yourself: > >I have built the debug package as advised, here is the result: > >#0 thrkill () at /tmp/-:2 >No locals. >#1 0x4ef74ca4d7368dd1 in ?? () >No symbol table info available. >#2 0x000001049f2745ab in _libc_abort () at /usr/src/lib/libc/stdlib/abort.c:51 > sa = {__sigaction_u = {__sa_handler = 0x3000000010, > __sa_sigaction = 0x3000000010}, sa_mask = 90789216, sa_flags = 29661} > mask = 4294967263 >#3 0x000001049f2433e4 in wrterror (d=0x1054e1c1000, > msg=0x1049f1a7f67 "write to free mem %p[%zu..%zu]@%zu") > at /usr/src/lib/libc/stdlib/malloc.c:378 > ap = {{gp_offset = 48, fp_offset = 48, overflow_arg_area = 0x73dd05695660, > reg_save_area = 0x73dd05695560}} > saved_errno = 35 >#4 0x000001049f244206 in validate_junk (pool=0x1054e1c1000, p=0x1050ccc5980, argsz=0) > at /usr/src/lib/libc/stdlib/malloc.c:780 > step = <optimized out> > lp = 0x1050ccc5980 > sz = <optimized out> > i = <optimized out> >#5 malloc_bytes (d=0x1054e1c1000, size=32) at /usr/src/lib/libc/stdlib/malloc.c:1237 > bucket = <optimized out> > r = <optimized out> > listnum = <optimized out> > bp = 0x1050957bf18 > i = <optimized out> > j = <optimized out> > lp = <optimized out> > k = <optimized out> > p = 0x1050ccc5980 > r = <optimized out> >#6 omalloc (pool=0x1054e1c1000, sz=32, zero_fill=0) > at /usr/src/lib/libc/stdlib/malloc.c:1391 > caller = <error reading variable caller (Cannot access memory at address 0x0)> > psz = <optimized out> > p = <optimized out> >#7 0x000001049f243689 in _libc_malloc (size=32) at /usr/src/lib/libc/stdlib/malloc.c:1545 > saved_errno = 35 > d = 0x1054e1c1000 > r = <optimized out> >#8 0x000001047d040251 in XCreateRegion () from /usr/X11R6/lib/libX11.so.18.2 >No symbol table info available. >#9 0x00000104c101d2b2 in Fl_Graphics_Driver::clip_box(int, int, int, int, int&, int&, int&, int&) () from /usr/local/lib/libfltk.so.8.0 >No symbol table info available. >#10 0x00000102794e10fe in dw::fltk::FltkViewBase::draw(dw::core::Rectangle const*, dw::fltk::FltkViewBase::DrawType) () >No symbol table info available. >#11 0x00000102794e0fdd in dw::fltk::FltkViewBase::draw() () >No symbol table info available. >#12 0x00000102794e33f0 in dw::fltk::FltkViewport::draw_area(void*, int, int, int, int) () >No symbol table info available. >#13 0x00000104c1020bdc in fl_scroll(int, int, int, int, int, int, void (*)(void*, int, int, int, int), void*) () from /usr/local/lib/libfltk.so.8.0 >No symbol table info available. >#14 0x00000102794e3635 in dw::fltk::FltkViewport::draw() () >No symbol table info available. >#15 0x00000104c0fad2cb in Fl_Group::draw_children() () from /usr/local/lib/libfltk.so.8.0 >No symbol table info available. >#16 0x00000104c0fad62d in Fl_Group::update_child(Fl_Widget&) const () > from /usr/local/lib/libfltk.so.8.0 >No symbol table info available. >#17 0x00000104c0fad2cb in Fl_Group::draw_children() () from /usr/local/lib/libfltk.so.8.0 >No symbol table info available. >#18 0x00000104c0fa557f in Fl_Double_Window::flush(int) () > from /usr/local/lib/libfltk.so.8.0 >No symbol table info available. >#19 0x00000104c0f91bac in Fl::flush() () from /usr/local/lib/libfltk.so.8.0 >No symbol table info available. >#20 0x00000104c0f91a49 in Fl::wait(double) () from /usr/local/lib/libfltk.so.8.0 >No symbol table info available. >#21 0x00000104c0f91c4d in Fl::run() () from /usr/local/lib/libfltk.so.8.0 >No symbol table info available. >#22 0x0000010279455f95 in main () >No symbol table info available. > >That is from a 'bt full' like before. If there is something else that >needs to be done, please let me know. > >Regarding what you said before: >>> dillo(84242) in free(): write to free mem 0xb5efb754c40[24..31]@32 > >> If you manage figure out what was at 0xb5efb754c40 that would help a >> lot. > >Is there a practical way to do that? I would try, but not sure how. > >Thanks! > >-Alex >_______________________________________________ >Dillo-dev mailing list -- dillo-dev(a)mailman3.com >To unsubscribe send an email to dillo-dev-leave(a)mailman3.com

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by a1ex＠dismail.de

Hi, On Wed, 28 Aug 2024 16:09:25 +0200 Theo Buehler <tb(a)theobuehler.org> wrote: > > Here is the output of 'bt full': > > Thanks, that doesn't help us further as such. > > I have enabled DEBUG_PACKAGES in dillo, which unfortunately wasn't > done before. Once the debug-dillo package is available (will be a few > days), we might get some more insight from a backtrace with symbols... > > If you have a ports tree set up (with doas, etc) you can build and > install it yourself: I have built the debug package as advised, here is the result: #0 thrkill () at /tmp/-:2 No locals. #1 0x4ef74ca4d7368dd1 in ?? () No symbol table info available. #2 0x000001049f2745ab in _libc_abort () at /usr/src/lib/libc/stdlib/abort.c:51 sa = {__sigaction_u = {__sa_handler = 0x3000000010, __sa_sigaction = 0x3000000010}, sa_mask = 90789216, sa_flags = 29661} mask = 4294967263 #3 0x000001049f2433e4 in wrterror (d=0x1054e1c1000, msg=0x1049f1a7f67 "write to free mem %p[%zu..%zu]@%zu") at /usr/src/lib/libc/stdlib/malloc.c:378 ap = {{gp_offset = 48, fp_offset = 48, overflow_arg_area = 0x73dd05695660, reg_save_area = 0x73dd05695560}} saved_errno = 35 #4 0x000001049f244206 in validate_junk (pool=0x1054e1c1000, p=0x1050ccc5980, argsz=0) at /usr/src/lib/libc/stdlib/malloc.c:780 step = <optimized out> lp = 0x1050ccc5980 sz = <optimized out> i = <optimized out> #5 malloc_bytes (d=0x1054e1c1000, size=32) at /usr/src/lib/libc/stdlib/malloc.c:1237 bucket = <optimized out> r = <optimized out> listnum = <optimized out> bp = 0x1050957bf18 i = <optimized out> j = <optimized out> lp = <optimized out> k = <optimized out> p = 0x1050ccc5980 r = <optimized out> #6 omalloc (pool=0x1054e1c1000, sz=32, zero_fill=0) at /usr/src/lib/libc/stdlib/malloc.c:1391 caller = <error reading variable caller (Cannot access memory at address 0x0)> psz = <optimized out> p = <optimized out> #7 0x000001049f243689 in _libc_malloc (size=32) at /usr/src/lib/libc/stdlib/malloc.c:1545 saved_errno = 35 d = 0x1054e1c1000 r = <optimized out> #8 0x000001047d040251 in XCreateRegion () from /usr/X11R6/lib/libX11.so.18.2 No symbol table info available. #9 0x00000104c101d2b2 in Fl_Graphics_Driver::clip_box(int, int, int, int, int&, int&, int&, int&) () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #10 0x00000102794e10fe in dw::fltk::FltkViewBase::draw(dw::core::Rectangle const*, dw::fltk::FltkViewBase::DrawType) () No symbol table info available. #11 0x00000102794e0fdd in dw::fltk::FltkViewBase::draw() () No symbol table info available. #12 0x00000102794e33f0 in dw::fltk::FltkViewport::draw_area(void*, int, int, int, int) () No symbol table info available. #13 0x00000104c1020bdc in fl_scroll(int, int, int, int, int, int, void (*)(void*, int, int, int, int), void*) () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #14 0x00000102794e3635 in dw::fltk::FltkViewport::draw() () No symbol table info available. #15 0x00000104c0fad2cb in Fl_Group::draw_children() () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #16 0x00000104c0fad62d in Fl_Group::update_child(Fl_Widget&) const () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #17 0x00000104c0fad2cb in Fl_Group::draw_children() () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #18 0x00000104c0fa557f in Fl_Double_Window::flush(int) () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #19 0x00000104c0f91bac in Fl::flush() () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #20 0x00000104c0f91a49 in Fl::wait(double) () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #21 0x00000104c0f91c4d in Fl::run() () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #22 0x0000010279455f95 in main () No symbol table info available. That is from a 'bt full' like before. If there is something else that needs to be done, please let me know. Regarding what you said before: >> dillo(84242) in free(): write to free mem 0xb5efb754c40[24..31]@32 > If you manage figure out what was at 0xb5efb754c40 that would help a > lot. Is there a practical way to do that? I would try, but not sure how. Thanks! -Alex

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by Theo Buehler

On Wed, Aug 28, 2024 at 03:56:55PM +0200, a1ex(a)dismail.de wrote: > On Wed, 28 Aug 2024 15:29:57 +0200 > Theo Buehler <tb(a)theobuehler.org> wrote: > > > > gdb: > > > Program received signal SIGABRT, Aborted. > > > thrkill () at /tmp/-:2 > > > 2 /tmp/-: No such file or directory. > > > in /tmp/- > > > > Please pkg_add gdb, use 'egdb dillo dillo.core and show the output of > > 'bt full', that might give a clue. > > Here is the output of 'bt full': Thanks, that doesn't help us further as such. I have enabled DEBUG_PACKAGES in dillo, which unfortunately wasn't done before. Once the debug-dillo package is available (will be a few days), we might get some more insight from a backtrace with symbols... If you have a ports tree set up (with doas, etc) you can build and install it yourself: $ cd /usr/ports/www/dillo $ grep DEBUG Makefile # ensure that your tree's up-to-date DEBUG_PACKAGES = ${BUILD_PACKAGES} $ make FETCH_PACKAGES= prepare $ make package $ make INSTALL_DEBUG_PACKAGES=yes install

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by a1ex＠dismail.de

On Wed, 28 Aug 2024 15:29:57 +0200 Theo Buehler <tb(a)theobuehler.org> wrote: > > gdb: > > Program received signal SIGABRT, Aborted. > > thrkill () at /tmp/-:2 > > 2 /tmp/-: No such file or directory. > > in /tmp/- > > Please pkg_add gdb, use 'egdb dillo dillo.core and show the output of > 'bt full', that might give a clue. Here is the output of 'bt full': #0 thrkill () at /tmp/-:2 No locals. #1 0x93e8d49f8f1828f2 in ?? () No symbol table info available. #2 0x00000675d78d25ab in _libc_abort () at /usr/src/lib/libc/stdlib/abort.c:51 sa = {__sigaction_u = {__sa_handler = 0x3000000010, __sa_sigaction = 0x3000000010}, sa_mask = 2205690992, sa_flags = 29926} mask = 4294967263 #3 0x00000675d78a13e4 in wrterror (d=0x675dc6e7678, msg=0x675d7805f67 "write to free mem %p[%zu..%zu]@%zu") at /usr/src/lib/libc/stdlib/malloc.c:378 ap = {{gp_offset = 48, fp_offset = 48, overflow_arg_area = 0x74e683782d70, reg_save_area = 0x74e683782c70}} saved_errno = 35 #4 0x00000675d78a2206 in validate_junk (pool=0x675dc6e7678, p=0x675d0e8d1c0, argsz=0) at /usr/src/lib/libc/stdlib/malloc.c:780 step = <optimized out> lp = 0x675d0e8d1c0 sz = <optimized out> i = <optimized out> #5 malloc_bytes (d=0x675dc6e7678, size=32) at /usr/src/lib/libc/stdlib/malloc.c:1237 bucket = <optimized out> r = <optimized out> listnum = <optimized out> bp = 0x675ba1e5f88 i = <optimized out> j = <optimized out> lp = <optimized out> k = <optimized out> p = 0x675d0e8d1c0 r = <optimized out> #6 omalloc (pool=0x675dc6e7678, sz=32, zero_fill=0) at /usr/src/lib/libc/stdlib/malloc.c:1391 caller = <error reading variable caller (Cannot access memory at address 0x0)> psz = <optimized out> p = <optimized out> #7 0x00000675d78a1689 in _libc_malloc (size=32) at /usr/src/lib/libc/stdlib/malloc.c:1545 saved_errno = 35 d = 0x675dc6e7678 r = <optimized out> #8 0x000006761b3d9918 in operator new (size=32) at /usr/src/gnu/lib/libcxx/../../../gnu/llvm/libcxx/src/new.cpp:67 p = <optimized out> #9 0x000006734c32cdd1 in ?? () No symbol table info available. #10 0x000006734c32cce5 in ?? () No symbol table info available. #11 0x000006734c32c933 in ?? () No symbol table info available. #12 0x000006734c32b4e2 in ?? () No symbol table info available. #13 0x000006734c32c7a9 in ?? () No symbol table info available. #14 0x000006734c32b31e in ?? () No symbol table info available. #15 0x000006734c32082d in ?? () No symbol table info available. #16 0x000006734c297b5c in ?? () No symbol table info available. #17 0x000006754d2f179b in fl_fix_focus() () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #18 0x000006754d2f1ca7 in Fl::handle_(int, Fl_Window*) () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #19 0x000006754d36483b in fl_handle(_XEvent const&) () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #20 0x000006754d360f40 in do_queued_events() () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #21 0x000006754d360e88 in fl_wait(double) () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #23 0x000006754d2f0c4d in Fl::run() () from /usr/local/lib/libfltk.so.8.0 No symbol table info available. #24 0x000006734c2941d5 in ?? () No symbol table info available. #25 0x000006734c2932bb in ?? () No symbol table info available. #26 0x0000000000000000 in ?? () No symbol table info available.

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by Theo Buehler

> Nav_open_url: new url='https://narkive.com' > Dns_server [0]: narkive.com is 149.248.211.108 > Connecting to 149.248.211.108:443 > TLS connect error: "an EOF was observed that violates the protocol" > Tls_close_by_key: Avoiding SSL shutdown for: https://narkive.com > fd 9 is done and failed > dillo(84242) in free(): write to free mem 0xb5efb754c40[24..31]@32 ok, that's a use after free, perhaps triggered by the unexpected EOF... Can't repro even with vm.malloc_conf flags set. $ dillo https://narkive.com paths: Cannot open file '/home/tb/.dillo/dillorc': No such file or directory paths: Using /etc/dillo/dillorc paths: Cannot open file '/home/tb/.dillo/keysrc': No such file or directory paths: Using /etc/dillo/keysrc paths: Cannot open file '/home/tb/.dillo/domainrc': No such file or directory paths: Using /etc/dillo/domainrc Domain: Default accept. dillo_dns_init: Here we go! (threaded) TLS library: LibreSSL 3.9.0 Disabling cookies. paths: Cannot open file '/home/tb/.dillo/hsts_preload': No such file or directory paths: Cannot open file '/etc/dillo/hsts_preload': No such file or directory paths: Using internal defaults... Nav_open_url: new url='https://narkive.com' Dns_server [0]: narkive.com is 149.248.211.108 Connecting to 149.248.211.108:443 narkive.com: TLSv1.3, cipher TLS_AES_128_GCM_SHA256 sha256 2048-bit RSA: /CN=narkive.com sha256 2048-bit RSA: /C=US/O=Let's Encrypt/CN=R11 root: /C=US/O=Internet Security Research Group/CN=ISRG Root X1 NumPendingStyleSheets=1 Dns_server [0]: narkive.net is 188.114.96.12 188.114.97.12 2a06:98c1:3120::c 2a06:98c1:3121::c narkive.net: TLSv1.3, cipher TLS_AES_256_GCM_SHA384 ecdsa-with-SHA256 256-bit EC: /CN=narkive.net ecdsa-with-SHA384 256-bit EC: /C=US/O=Google Trust Services/CN=WE1 sha256 384-bit EC: /C=US/O=Google Trust Services LLC/CN=GTS Root R4 root: /C=BE/O=GlobalSign nv-sa/OU=Root CA/CN=GlobalSign Root CA >>>> a_Nav_repush <<<< Nav_open_url: new url='https://narkive.com' a_Nav_expect_done: repush! Dillo: normal exit! > gdb: > Program received signal SIGABRT, Aborted. > thrkill () at /tmp/-:2 > 2 /tmp/-: No such file or directory. > in /tmp/- Please pkg_add gdb, use 'egdb dillo dillo.core and show the output of 'bt full', that might give a clue. > dillo(84242) in free(): write to free mem 0xb5efb754c40[24..31]@32 If you manage figure out what was at 0xb5efb754c40 that would help a lot. > My nc output matches yours, but ftp still gets the handshake > failure. Maybe it's something network specific on my end? But like I > mentioned, an older OpenBSD system on my network with LibreSSL 3.6.0 > does not have an issue with that site. I disabled pf just in case, but > to no avail. No real idea at this point - sorry.

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by a1ex＠dismail.de

Hi, On Wed, 28 Aug 2024 13:45:04 +0200 Theo Buehler <tb(a)theobuehler.org> wrote: > > Certainly. Ideally this should not crash Dillo, no matter how > > obscure. > > What crashes dillo? Opening https://narkive.com crashes Dillo for me, It doesn't happen on any other sites: Nav_open_url: new url='https://narkive.com' Dns_server [0]: narkive.com is 149.248.211.108 Connecting to 149.248.211.108:443 TLS connect error: "an EOF was observed that violates the protocol" Tls_close_by_key: Avoiding SSL shutdown for: https://narkive.com fd 9 is done and failed dillo(84242) in free(): write to free mem 0xb5efb754c40[24..31]@32 gdb: Program received signal SIGABRT, Aborted. thrkill () at /tmp/-:2 2 /tmp/-: No such file or directory. in /tmp/- > > I don't know why it shows that version number instead of the latest > > one, this is a fresh install of a recent snapshot. > > It's a bit unfortunate, but the development release on current openbsd > snapshots still has 3.9.0 as the version number. It's complicated... > The short story is that we can't crank to 4.0.0 because it would break > the build of some important ports (e.g., rust). Thank you for clarifying this. > > $ ftp https://narkive.com/test > > Trying 149.248.211.108... > > TLS handshake failure: handshake failed: unexpected EOF > > I don't see this: > > $ ftp https://narkive.com/test > Trying 149.248.211.108... > Requesting https://narkive.com/test > ftp: Error retrieving https://narkive.com/test: 404 Not Found > $ nc -cvz narkive.com 443 > Connection to narkive.com (149.248.211.108) 443 port [tcp/https] > succeeded! TLS handshake negotiated TLSv1.3/TLS_AES_128_GCM_SHA256 > with host narkive.com Peer name: narkive.com > Subject: /CN=narkive.com > Issuer: /C=US/O=Let's Encrypt/CN=R11 > Valid From: Fri Jul 26 06:13:58 2024 > Valid Until: Thu Oct 24 06:13:57 2024 > Cert Hash: > SHA256:456275146f4b2e65c16d2432ff87917a4501ddb41dcb158195255c5995abbbf1 > OCSP URL: http://r11.o.lencr.org > > This looks all good from here. My nc output matches yours, but ftp still gets the handshake failure. Maybe it's something network specific on my end? But like I mentioned, an older OpenBSD system on my network with LibreSSL 3.6.0 does not have an issue with that site. I disabled pf just in case, but to no avail. I'm open to any clues on how to diagnose this! Regards, Alex

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by Theo Buehler

On Wed, Aug 28, 2024 at 01:25:21PM +0200, a1ex(a)dismail.de wrote: > Hi Rodrigo, > > On Tue, 27 Aug 2024 23:09:42 +0200 > Rodrigo Arias <rodarima(a)gmail.com> wrote: > > > >> >dillo-dev.auriga.wearlab.narkive.com:443 > > >> > > > >> >Is this site just broken/misconfigured? > > >> > > >> Cannot reproduce with LibreSSL 3.9.2 on Linux. > > >> ... > > >> Can you test with the latest LibreSSL 3.9.2? > > >> > > >> Is this it happening with the proxy enabled? Also, which user agent > > >> are you using in curl and Dillo? > > > > > >It happens with the stock user agent and no proxy, same with curl. > > >I'm running the latest snapshot of OpenBSD, which would have the > > >latest version of LibreSSL. > > > > > >I don't care about that site, my only worry is that the error can > > >crash Dillo. > > > > If it happens it is a bug on Dillo side as it is not handling all > > errors, regardless of the site. I would like to reproduce it to fix > > it. > > Certainly. Ideally this should not crash Dillo, no matter how obscure. What crashes dillo? > > >This could be an OpenBSD specific issue which wouldn't show up on > > >Linux. > > > > > It could be, but I would first reject that is not due to mismatch of > > versions. > > > > The last LibreSSL as per https://www.libressl.org/ is: > > > > > The latest stable release is 3.9.2 > > > > Which should be printed in the first lines when starting Dillo: > > > > > TLS library: LibreSSL 3.9.2 > > > > If it says 3.9.0, then Dillo is not using the last release. > > I don't know why it shows that version number instead of the latest > one, this is a fresh install of a recent snapshot. It's a bit unfortunate, but the development release on current openbsd snapshots still has 3.9.0 as the version number. It's complicated... The short story is that we can't crank to 4.0.0 because it would break the build of some important ports (e.g., rust). > Anyway, I installed 3.9.2 from source and built Dillo against that. > Now it reports the correct version, but the crash still happens the > same. > > I guess we would need to hear from some other OpenBSD users to confirm > if this a real issue, or if its something whacky on my end. > > I do have an older OpenBSD system which uses LibreSSL 3.6.0, and it does > not exhibit the problem. But on 3 newer systems here the problem occurs. > > Here is an easy way to confirm using only OpenBSD base tools: > > $ ftp https://narkive.com/test > Trying 149.248.211.108... > TLS handshake failure: handshake failed: unexpected EOF I don't see this: $ ftp https://narkive.com/test Trying 149.248.211.108... Requesting https://narkive.com/test ftp: Error retrieving https://narkive.com/test: 404 Not Found $ nc -cvz narkive.com 443 Connection to narkive.com (149.248.211.108) 443 port [tcp/https] succeeded! TLS handshake negotiated TLSv1.3/TLS_AES_128_GCM_SHA256 with host narkive.com Peer name: narkive.com Subject: /CN=narkive.com Issuer: /C=US/O=Let's Encrypt/CN=R11 Valid From: Fri Jul 26 06:13:58 2024 Valid Until: Thu Oct 24 06:13:57 2024 Cert Hash: SHA256:456275146f4b2e65c16d2432ff87917a4501ddb41dcb158195255c5995abbbf1 OCSP URL: http://r11.o.lencr.org This looks all good from here. > This doesn't happen on any other site that I have seen. > > Maybe I should report this to the OpenBSD/LibreSSL people as well, so > I'm CC'ing tb@ What is an example of a site that you can't connect to or even crashes?

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by a1ex＠dismail.de

Hi Rodrigo, On Tue, 27 Aug 2024 23:09:42 +0200 Rodrigo Arias <rodarima(a)gmail.com> wrote: > >> >dillo-dev.auriga.wearlab.narkive.com:443 > >> > > >> >Is this site just broken/misconfigured? > >> > >> Cannot reproduce with LibreSSL 3.9.2 on Linux. > >> ... > >> Can you test with the latest LibreSSL 3.9.2? > >> > >> Is this it happening with the proxy enabled? Also, which user agent > >> are you using in curl and Dillo? > > > >It happens with the stock user agent and no proxy, same with curl. > >I'm running the latest snapshot of OpenBSD, which would have the > >latest version of LibreSSL. > > > >I don't care about that site, my only worry is that the error can > >crash Dillo. > > If it happens it is a bug on Dillo side as it is not handling all > errors, regardless of the site. I would like to reproduce it to fix > it. Certainly. Ideally this should not crash Dillo, no matter how obscure. > >This could be an OpenBSD specific issue which wouldn't show up on > >Linux. > > > It could be, but I would first reject that is not due to mismatch of > versions. > > The last LibreSSL as per https://www.libressl.org/ is: > > > The latest stable release is 3.9.2 > > Which should be printed in the first lines when starting Dillo: > > > TLS library: LibreSSL 3.9.2 > > If it says 3.9.0, then Dillo is not using the last release. I don't know why it shows that version number instead of the latest one, this is a fresh install of a recent snapshot. Anyway, I installed 3.9.2 from source and built Dillo against that. Now it reports the correct version, but the crash still happens the same. I guess we would need to hear from some other OpenBSD users to confirm if this a real issue, or if its something whacky on my end. I do have an older OpenBSD system which uses LibreSSL 3.6.0, and it does not exhibit the problem. But on 3 newer systems here the problem occurs. Here is an easy way to confirm using only OpenBSD base tools: $ ftp https://narkive.com/test Trying 149.248.211.108... TLS handshake failure: handshake failed: unexpected EOF This doesn't happen on any other site that I have seen. Maybe I should report this to the OpenBSD/LibreSSL people as well, so I'm CC'ing tb@ Regards, Alex

Aug. 28, 2024

Re: Issues with HTTP multipart/form-data file upload

by Xavier Del Campo Romero

Hi Rodrigo, > Glad to read that you also consider Dillo for slcl, and thanks for preparing the patches :-) Thank you! I want slcl to be useful to anyone, including users who care about minimalist software like Dillo. The web is already too crowded with bloated "webapps" and other terrible things. :) > Sounds good, not sure how complicated it would be to do this. I still need to investigate this further, but I assume this would require Dillo to at least implement a sink callback. In other words, the component responsible for transmitting the data (probably src/IO/IO.c) should trigger a user-defined callback with an arbitrarily-sized buffer (typically, of BUFSIZ bytes, as defined by stdio.h) that must filled with file data. Then, the user-defined callback can fill from zero up to BUFSIZ bytes, which are eventually trasmitted to the server. That said, I am still not sure how much actual effort this would take. But I am glad to receive positive feedback so far - I will then continue to find a solution. > However, being able to upload multiple files at the same time sounds reasonable, so feel free to try on your own in the meanwhile. Uploading multiple files at once seems doable - the patches I sent on my previous email are probably already doing most of the required work. Again, the trickiest task is to send data on-the-fly for each selected file. > Shouldn't it be 68 then? I understand the opposite: the boundary string with the two leading dashes ("--") included can be up to 72 bytes long, and 74 bytes long for the ending boundary (which includes two more dashes after the boundary string). This is confirmed by reading the BNF defined by RFC 2046 (some bits omitted for simplicity), section 5.1.1 [1]: > boundary := 0*69<bchars> bcharsnospace > bchars := bcharsnospace / " " > bcharsnospace := DIGIT / ALPHA / "'" / "(" / ")" / > "+" / "_" / "," / "-" / "." / > "/" / ":" / "=" / "?" > dash-boundary := "--" boundary > ; boundary taken from the value of > ; boundary parameter of the > ; Content-Type field> multipart-body := [preamble CRLF] > dash-boundary transport-padding CRLF > body-part *encapsulation > close-delimiter transport-padding > [CRLF epilogue] > delimiter := CRLF dash-boundary > close-delimiter := delimiter "--" Note: even if the specification tells receivers to handle transport padding, for the time being I am assuming "transport-padding" as zero length since composers must not generate non-zero length transport padding. I am still not sure where transport padding would apply, anyway. Probably outside web browsers? > I would leave out all the symbols to avoid quoting and only use A-Z a-z and 0-9. Interestingly, Dillo would always quote boundary strings [2], even if only using A-Z, a-z and 0-9. In fact, this is one of the wrong assumptions I spotted when testing slcl against Dillo. > Which, if I computed it correctly, is still too small to worry about. Not only it is too small of a chance: if we really wanted to do "the right thing" and make Dillo absolutely sure the boundary string is not contained within the selected files, this would imply a noticeable performance impact when dealing with large files, much likely for a near-zero benefit. I have not inspected their source code yet (and I do not want to), but I understand both Gecko and Chromium are also making that assumption, because otherwise it would take them a lot of CPU time to upload large files. > Why sizeof " " instead of just 2? Because, to my eyes, sizeof " " has more meaningful semantics, compared to a magic integer constant such as 2. However, for this simple scenario, I would still consider both acceptable. I can replace it with 2 if you find the other construct unacceptable. > PS: When are you playing? Sorry, I did not understand your last sentence. Could you please give a bit more context? :) Best regards, Xavi [1]: https://www.rfc-editor.org/rfc/rfc2046.html#section-5.1.1 [2]: https://github.com/dillo-browser/dillo/blob/8a360e32ac3136494a494379a6dbbac… On 27/8/24 22:59, Rodrigo Arias wrote: > Hi Xavier, > > On Mon, Aug 26, 2024 at 01:27:29AM +0200, Xavier Del Campo Romero wrote: >> Hello Dillo dev community, >> >> I am testing support among web browsers for slcl [1], a JS-less >> minimalist web storage solution. slcl relies on >> "multipart/form-data"-encoded HTTP requests to upload files to a server. >> Whereas Dillo helped me to uncover a few wrong assumptions on my code, I >> have realised a few issues on Dillo itself related to filue uploads that >> should be considered. > > Glad to read that you also consider Dillo for slcl, and thanks for > preparing the patches :-) > >> Issue #1: >> >> While Dillo is fine uploading small files (up to a few MiB), things go >> wrong with larger files, so much that memory usage increases up to >> multiple GiB and can even lock the system up. This is because Dillo is >> designed to send requests always from memory, which means file contents >> must be dumped into memory first, and this might be unfeasible for large >> files. >> >> Suggestion: >> >> Instead, Dillo should ideally send file contents on-the-fly, so that >> memory usage is kept to a minimum regardless the file size. > > Sounds good, not sure how complicated it would be to do this. > >> Issue #2: >> >> Even if Dillo generates a 70-byte, random boundary string (yet mostly >> filled with '-', similarly to Firefox [2]), it ensures it is not found >> anywhere inside the file contents. Again, this can be a serious >> bottleneck in the case of large files, as it requires to scan the whole >> file for a match. >> >> Suggestion: >> >> Define *all* of the 70 bytes in the boundary string as random, and >> assume they would never be found inside a file. The chance of accidental >> collision is so low that it is not worth the effort into checking them. >> >> Suggested patches: >> >> - 0001-dialog.cc-Generate-more-random-boundaries.patch) > > Yes, I think is a good idea. > > @@ -1246,6 +1246,24 @@ Dstr *DilloHtmlForm::buildQueryData(DilloHtmlInput > *active_submit) > return DataStr; > } > > +static void generate_boundary(Dstr *boundary) > +{ > + for (int i = 0; i < 70; i++) { > > I think this is too long: > >> Boundary delimiters must not appear within the encapsulated material, >> and must be no longer than 70 characters, not counting the two >> leading hyphens. > > Shouldn't it be 68 then? > > + /* Extracted from RFC 2046, section 5.1.1. */ > + static const char set[] = "abcdefghijklmnopqrstuvwxyz" > + "ABCDEFGHIJKLMNOPQRSTUVWXYZ" > + "0123456789" > + "'()+_,-./:=? "; > > I would leave out all the symbols to avoid quoting and only use A-Z a-z > and 0-9. > > Assuming you are left with 62 symbols (26*2 + 10), the probability of > finding a random 70 character string in a random sample of 70 characters > is (using ** as ^): > > p = 1/62**70 = 3.4086e-126 > > But as you will be dealing with large files, each line of the input > could match it. We are interested in the probability that a given > boundary will mach any of the N lines. So we compute the probability > that it will match none of the lines (1-p)**N and then the opposite of > that (matches at least one line): > > 1 - (1 - p) ** N > > For example, a 1 TB encoded file, with 2**40/70 lines will have: > > p = 1 - (1 - 1/62**70) ** (2**40 / 70) > > Of occurring, which is around 5.354e-116. > > Which, if I computed it correctly, is still too small to worry about. > > + char s[sizeof " "] = {0}; > > Why sizeof " " instead of just 2? > > + > + do { > + *s = rand(); > + } while (!strspn(s, set)); > + > + dStr_append(boundary, s); > + } > +} > + > /** > * Generate a boundary string for use in separating the parts of a > * multipart/form-data submission. > >> Issue #3: >> >> Dillo only supports uploading 1 file at a time. This is mostly because >> it relies (probably temporarily?) on the a_Dialog_save_file function >> [3]. However, this is not a limitation on the HTTP protocol, and other >> implementation such as Firefox or Chromium-based browsers support this. >> >> Suggested patches: >> >> - 0002-dialog-Add-a_Dialog_select_files.patch >> - 0003-WIP-multi-file-uploads.patch >> >> Conclusions: >> >> Dillo seems designed to always send requests from memory, so it is not >> straightforward to break this assumption in order to support large file >> uploads. 0003-WIP-multi-file-uploads.patch is an incomplete first step >> into fixing this, but it surely needs deeper design changes. >> >> I did not put more effort into these patches for the time being because, >> after seeing the potential complexity behind this task, I thought it was >> a better idea to ask the community for feedback and guidelines. > > I'm not very familiar with this part of Dillo, so I'll have to check a > bit more before providing more feedback. > > However, being able to upload multiple files at the same time sounds > reasonable, so feel free to try on your own in the meanwhile. > > PS: When are you playing? > > Best, > Rodrigo. > _______________________________________________ > Dillo-dev mailing list -- dillo-dev(a)mailman3.com > To unsubscribe send an email to dillo-dev-leave(a)mailman3.com

Aug. 27, 2024