- Dillo-dev - lists.mailman3.com

Queue Downloads

by Diego

Hello, maybe this script can be useful to someone. Requests downloads.dpi to download a URL to a file. He talks to dpid like dillo does. I think it's also a useful coding exercise. I'm bad at naming so if someone has a better name ^_^¡ Best, Diego!

Sept. 4, 2024

[PATCH] make downloads.dpi use http_user_agent from dillorc

by a1ex＠dismail.de

Hi, Here is a patch which allows wget called from downloads.dpi to use the user agent provided in dillorc, and fallback to the hardcoded Dillo/version agent if not defined. Regards, Alex diff -upr a/dlib/dlib.c b/dlib/dlib.c --- a/dlib/dlib.c Sun Aug 11 22:21:59 2024 +++ b/dlib/dlib.c Tue Sep 3 13:49:00 2024 @@ -922,6 +922,45 @@ char *dGethomedir (void) } /** + * Return the http_user_agent value in a static string + */ +char *dGetuseragent (void) +{ + static char *dillorc = NULL; + dillorc = dStrconcat(dGethomedir(), "/", ".dillo/dillorc", NULL); + FILE *In; + int len; + char *rcline = NULL, *value = NULL, *p; + if ((In = fopen(dillorc, "r")) == NULL) { + DLIB_MSG("dGetuseragent: unable to open dillorc.\n"); + return ("Dillo/" VERSION); + } + while ((rcline = dGetline(In)) != NULL) { + if (strncmp(rcline, "http_user_agent", 15) == 0) + break; + dFree(rcline); + } + fclose(In); + if (!rcline) { + value = ("Dillo/" VERSION); + DLIB_MSG("dGetuseragent: no 'http_user_agent' in dillorc.\n"); + } else { + len = (int) strlen(rcline); + if (len && rcline[len - 1] == '\n') + rcline[len - 1] = 0; + if ((p = strchr(rcline, '='))) { + while (*++p == ' '); + value = dStrdup(p); + } else { + value = ("Dillo/" VERSION); + DLIB_MSG("dGetuseragent: error parsing value in dillorc.\n"); + } + } + dFree(rcline); + return (value); +} + +/** * Get a line from a FILE stream. * Return value: read line on success, NULL on EOF. */ diff -upr a/dlib/dlib.h b/dlib/dlib.h --- a/dlib/dlib.h Sun Aug 11 22:21:59 2024 +++ b/dlib/dlib.h Tue Sep 3 13:49:00 2024 @@ -175,6 +175,7 @@ void dLib_show_messages(bool_t show); */ char *dGetcwd(void); char *dGethomedir(void); +char *dGetuseragent(void); char *dGetline(FILE *stream); int dClose(int fd); int dUsleep(unsigned long us); diff -upr a/dpi/downloads.cc b/dpi/downloads.cc --- a/dpi/downloads.cc Sun Aug 11 22:21:59 2024 +++ b/dpi/downloads.cc Tue Sep 3 13:49:06 2024 @@ -45,6 +45,7 @@ #include "config.h" #include "dpiutil.h" #include "../dpip/dpip.h" +#include "../dlib/dlib.h" /* * Debugging macros @@ -324,7 +325,7 @@ DLItem::DLItem(const char *full_filename, const char * cookies_path = dStrconcat(dGethomedir(), "/.dillo/cookies.txt", NULL); dl_argv = new char*[10]; int i = 0; - const char *user_agent = "Dillo/" VERSION; + const char *user_agent = dGetuseragent(); dl_argv[i++] = (char*)"wget"; if (stat(fullname, &ss) == 0) init_bytesize = (int)ss.st_size;

Sept. 3, 2024

Re: [WIP PATCH] Use SIGUSR1 to reload the page

by Rodrigo Arias

Hi, On Mon, Sep 02, 2024 at 04:04:30PM +0200, a1ex(a)dismail.de wrote: >Hi Rodrigo, > >I find your recent idea[0] to refresh the page on local file changes >interesting, and it could be a quite useful feature. > >[0] https://github.com/dillo-browser/dillo/issues/255 > >I imagine it can be triggered with a tool like entr[1]: > >ls /tmp/test.html | entr 'pkill -SIGUSR1 dillo' > >[1] http://eradman.com/entrproject/ Yeah, that was the idea. >So far, I have a working signal handler which will act upon receipt of >a SIGUSR1 signal. It just prints the signal # to the console right now. >I am trying to make it perform an 'a_UIcmd_reload', but haven't been >successful yet: > >dillo.cc:389:19: error: use of undeclared identifier 'vbw' > a_UIcmd_reload(*vbw); > ^ >1 error generated You need to find a pointer to BrowserWindow and pass it to a_UIcmd_reload(). Check a_Bw_num() and a_Bw_get() in src/bw.c. We should reload the current page on all windows. I'm not sure if we can do this from the context of a signal, we probably need to register a timeout in FLTK and handle it from there. Probably not a very easy issue. I should start labeling the ones that are easy. Best, Rodrigo.

Sept. 2, 2024

[WIP PATCH] Use SIGUSR1 to reload the page

by a1ex＠dismail.de

Hi Rodrigo, I find your recent idea[0] to refresh the page on local file changes interesting, and it could be a quite useful feature. [0] https://github.com/dillo-browser/dillo/issues/255 I imagine it can be triggered with a tool like entr[1]: ls /tmp/test.html | entr 'pkill -SIGUSR1 dillo' [1] http://eradman.com/entrproject/ So far, I have a working signal handler which will act upon receipt of a SIGUSR1 signal. It just prints the signal # to the console right now. I am trying to make it perform an 'a_UIcmd_reload', but haven't been successful yet: dillo.cc:389:19: error: use of undeclared identifier 'vbw' a_UIcmd_reload(*vbw); ^ 1 error generated bw.h and uicmd.hh are defined in dillo.cc, so its probably a skill issue on my end :) Any hints would be appreciated! Regards, Alex --- a/src/dillo.cc Sun Aug 11 22:21:59 2024 +++ b/src/dillo.cc Mon Sep 2 14:21:08 2024 @@ -381,6 +381,14 @@ static DilloUrl *makeStartUrl(char *str, bool local) } /* + * Use SIGUSR1 to reload the page + */ +static void reload_handler(int signum) +{ + MSG("signum=%d\n\n", signum); +} + +/* * MAIN */ int main(int argc, char **argv) @@ -579,6 +587,10 @@ int main(int argc, char **argv) a_Url_free(start_url); } } + + // Reload page on SIGUSR1 + if (signal (SIGUSR1, reload_handler) == SIG_IGN) + signal (SIGUSR1, SIG_IGN); Fl::run();

Sept. 2, 2024

Re: Issues with HTTP multipart/form-data file upload

by Rodrigo Arias

Hi Xavier, On Sun, Sep 01, 2024 at 11:17:53PM +0200, Xavier Del Campo Romero wrote: >> This other method has a slightly not >> uniform distribution, but only runs rand() 70 times: >> >> Which is probably okay for this case. > >I also think it is good enough. > >> Then, I think using our own set may be the most readable solution here, >> which also allows the quick module method. > >I agree. Please, take a look at this PR: https://github.com/dillo-browser/dillo/pull/256 Here is the raw patch series for "git am": https://github.com/dillo-browser/dillo/pull/256.patch It would be nice if you can test it with slcl to know it works fine. Best, Rodrigo.

Sept. 1, 2024

Re: Issues with HTTP multipart/form-data file upload

by Xavier Del Campo Romero

Hi Rodrigo, > Not sure what you mean with "unnecessary use of the heap". I meant > something like this: You are right: I was too fixated with strcspn(3), when in fact strchr(3) would already do. Also, this use of dStr_append_c() looks good to me. > The variable c is stored in a register, as you can see in the > disassembly (built with -Og): Thanks for the detailed explanation. Anyway, I do not expect this function to become a performance bottleneck. > This other method has a slightly not > uniform distribution, but only runs rand() 70 times: > > Which is probably okay for this case. I also think it is good enough. > Then, I think using our own set may be the most readable solution here, > which also allows the quick module method. I agree. Best regards, Xavi On 1/9/24 16:12, Rodrigo Arias wrote: > Hi Xavier, > >> Thank you. I am still unfamiliar with that part of Dillo, so please let >> me know about any progress. > > For now I'm still writing the RFC and doing some proof of concepts. I > can see that it will take a while. > >> Limiting ourselves to a-z, A-Z and 0-9 would still account for 62 out of >> the 75 possible characters, so roughly 82% of the set. I think that >> removing the quoting in favour of the limited set reduce the risk for >> broken implementations, yet still provide a good amount of randomness. > > Yes, I think so too. > >> I am not sure whether this was an intentional modification from your >> side. My patch is adding a <space> as defined by POSIX.1-2017 [1], so >> that sizeof " " would always return 2. Was it your intention to flag >> this potential confusion? > > Yes, my point was that it is that is not easy to determine the length of > a UTF-8 string by just looking at it. > >> Also, there was not strict reason to use sizeof " ". Any other >> character would do e.g.: sizeof "x", sizeof "A", etc. > > Same problem: > > ᕁ 𝓍 𝙭 х 𝐱 𝗑 ⤫ 𝑥 𝘅 ⤬ ᙮ ⨯ 𝕩 𝖝 × 𝔁 𝚡 x ｘ ⅹ 𝔵 ᕽ 𝒙 𝘹 > > 𝙰 A 𝐴 ᗅ 𝑨 𝚨 𝕬 𝖠 А Α 𝛢 𝝖 𝒜 𝜜 𖽀 𝓐 𝞐 Ａ 𝗔 𝘈 Ꭺ ꓮ 𝐀 𝔄 𝔸 𐊠 𝘼 > > Check: https://util.unicode.org/UnicodeJsps/confusables.jsp > > It is generally safer to use the explicit length. > >>> You can also use dStr_append_c() to only append one character, so you >>> only need a single character. >> >> That would be an unnecessary use of the heap, because the size is static. > > Not sure what you mean with "unnecessary use of the heap". I meant > something like this: > > static void generate_boundary(Dstr *boundary) > { > for (int i = 0; i < 70; i++) { > /* Extracted from RFC 2046, section 5.1.1. */ > static const char set[] = "abcdefghijklmnopqrstuvwxyz" > "ABCDEFGHIJKLMNOPQRSTUVWXYZ" > "0123456789"; > int c; > > do { > c = rand() & 0xff; > } while (!strchr(set, c)); > > dStr_append_c(boundary, c); > } > } > > The variable c is stored in a register, as you can see in the > disassembly (built with -Og): > > hop% r2 build/src/dillo > WARN: Relocs has not been applied. Please use `-e bin.relocs.apply=true` > or `-e bin.cache=true` next time > [0x000e3860]> s sym.generate_boundary_Dstr_ > [0x0012cf99]> af > [0x0012cf99]> pdf > ┌ 85: sym.generate_boundary_Dstr_ (int64_t arg1); > │ ; arg int64_t arg1 @ rdi > │ 0x0012cf99 55 push rbp ; > Fl_Pixmap.H:1250 ; generate_boundary(Dstr*) > │ 0x0012cf9a 4889e5 mov rbp, rsp > │ 0x0012cf9d 4155 push r13 > │ 0x0012cf9f 4154 push r12 > │ 0x0012cfa1 53 push rbx > │ 0x0012cfa2 4883ec08 sub rsp, 8 > │ 0x0012cfa6 4989fd mov r13, rdi ; > arg1 > │ 0x0012cfa9 41bc00000000 mov r12d, 0 ; > Fl_Pixmap.H:1251 > │ ┌─< 0x0012cfaf eb2c jmp 0x12cfdd > │ ┌┌──> 0x0012cfb1 ff1509cd1e00 call qword [reloc.rand] ; > Fl_Pixmap.H:1253 ; [0x319cc0:8]=0 > │ ╎╎│ 0x0012cfb7 0fb6d8 movzx ebx, al ; > <--- Perform the "& 0xff" by zero-extending the lowest byte in eax > │ ╎╎│ 0x0012cfba 89de mov esi, ebx ; > Fl_Pixmap.H:1260 > │ ╎╎│ 0x0012cfbc 488d3dbd5b.. lea rdi, > obj.generate_boundary_Dstr_::set ; 0x2a2b80 ; > "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789" > │ ╎╎│ 0x0012cfc3 ff15afc51e00 call qword [reloc.strchr] ; > [0x319578:8]=0 > │ ╎╎│ 0x0012cfc9 4885c0 test rax, rax > │ └───< 0x0012cfcc 74e3 je 0x12cfb1 > │ ╎│ 0x0012cfce 89de mov esi, ebx ; > Fl_Pixmap.H:1262 > │ ╎│ 0x0012cfd0 4c89ef mov rdi, r13 > │ ╎│ 0x0012cfd3 67e830570800 call sym.dStr_append_c > │ ╎│ 0x0012cfd9 4183c401 add r12d, 1 ; > Fl_Pixmap.H:1251 > │ ╎│ ; CODE XREF from generate_boundary(Dstr*) @ 0x12cfaf(x) > │ ╎└─> 0x0012cfdd 4183fc45 cmp r12d, 0x45 ; > 'E' > │ └──< 0x0012cfe1 7ece jle 0x12cfb1 > │ 0x0012cfe3 4883c408 add rsp, 8 ; > Fl_Pixmap.H:1264 > │ 0x0012cfe7 5b pop rbx > │ 0x0012cfe8 415c pop r12 > │ 0x0012cfea 415d pop r13 > │ 0x0012cfec 5d pop rbp > └ 0x0012cfed c3 ret > > This method gives us a perfect uniform distribution when RAND_MAX is a > power of 2, at the cost of executing rand() more times than required > (70/(62/256) ≈ 289 on average). This other method has a slightly not > uniform distribution, but only runs rand() 70 times: > > static void generate_boundary(Dstr *boundary) > { > /* Extracted from RFC 2046, section 5.1.1. */ > static const char set[] = "abcdefghijklmnopqrstuvwxyz" > "ABCDEFGHIJKLMNOPQRSTUVWXYZ" > "0123456789"; > static const int n = strlen(set); > > for (int i = 0; i < 70; i++) { > int c = (unsigned char) set[rand() % n]; > dStr_append_c(boundary, c); > } > } > > Which is probably okay for this case. > > Notice I haven't tested any of these methods yet, I'll need to add a > form upload test case to be able to see them in action (or a unit test). > >>> If we only use alphanumeric characters, we can just use isalnum() right? >> >> According to POSIX.1-2017 [2], isalnum(3) depends on the current locale >> configured by the system. For example, characters such as Ä or ú could >> return non-zero. > > Oh right, for some reason I was thinking this was the other way around, > and isalnum only worked with ASCII. I think we should review the other > uses of isalnum and friends as I think there may be used under similar > assumptions. > >> To avoid this, there are two possible solutions: >> >> 1. Use isalnum_l(3) to specify a locale_t object corresponding to the >> "POSIX" locale (equivalent to "C" [3]), which must be previously >> allocated by the newlocale(3) function [3] and released by the >> freelocal(3) function [4]. A minimalist example is shown below: >> >> locale_t l = newlocale(LC_CTYPE, "POSIX", NULL); >> >> for (unsigned char i = 0; i < 255; i++) >> printf("hhu=%hhu, c=%c, isalnum=%d\n", i, i, >> isalnum_l(i, l)); >> >> freelocale(l); >> >> 2. Define a known subset from the portable character set defined by >> POSIX.1-2017 [5] and use strspn(3), as already suggested by the patch. >> IMHO this approach is better because: >> - It does not deal with locales, so developers not familiar with them >> would understand the code better. >> - It is also portable outside a POSIX environment (not sure if this a >> requirement, though). >> - It does not require dynamic allication via newlocale(3). >> - It is the only possible option if non-alnum characters, such as ':' >> or '/', are appended to the boundary string. > > Then, I think using our own set may be the most readable solution here, > which also allows the quick module method. > > Best, > Rodrigo > _______________________________________________ > Dillo-dev mailing list -- dillo-dev(a)mailman3.com > To unsubscribe send an email to dillo-dev-leave(a)mailman3.com

Sept. 1, 2024

Re: Issues with HTTP multipart/form-data file upload

by Rodrigo Arias

Hi Xavier, >Thank you. I am still unfamiliar with that part of Dillo, so please let >me know about any progress. For now I'm still writing the RFC and doing some proof of concepts. I can see that it will take a while. >Limiting ourselves to a-z, A-Z and 0-9 would still account for 62 out of >the 75 possible characters, so roughly 82% of the set. I think that >removing the quoting in favour of the limited set reduce the risk for >broken implementations, yet still provide a good amount of randomness. Yes, I think so too. >I am not sure whether this was an intentional modification from your >side. My patch is adding a <space> as defined by POSIX.1-2017 [1], so >that sizeof " " would always return 2. Was it your intention to flag >this potential confusion? Yes, my point was that it is that is not easy to determine the length of a UTF-8 string by just looking at it. >Also, there was not strict reason to use sizeof " ". Any other >character would do e.g.: sizeof "x", sizeof "A", etc. Same problem: ᕁ 𝓍 𝙭 х 𝐱 𝗑 ⤫ 𝑥 𝘅 ⤬ ᙮ ⨯ 𝕩 𝖝 × 𝔁 𝚡 x ｘ ⅹ 𝔵 ᕽ 𝒙 𝘹 𝙰 A 𝐴 ᗅ 𝑨 𝚨 𝕬 𝖠 А Α 𝛢 𝝖 𝒜 𝜜 𖽀 𝓐 𝞐 Ａ 𝗔 𝘈 Ꭺ ꓮ 𝐀 𝔄 𝔸 𐊠 𝘼 Check: https://util.unicode.org/UnicodeJsps/confusables.jsp It is generally safer to use the explicit length. >> You can also use dStr_append_c() to only append one character, so you >> only need a single character. > >That would be an unnecessary use of the heap, because the size is static. Not sure what you mean with "unnecessary use of the heap". I meant something like this: static void generate_boundary(Dstr *boundary) { for (int i = 0; i < 70; i++) { /* Extracted from RFC 2046, section 5.1.1. */ static const char set[] = "abcdefghijklmnopqrstuvwxyz" "ABCDEFGHIJKLMNOPQRSTUVWXYZ" "0123456789"; int c; do { c = rand() & 0xff; } while (!strchr(set, c)); dStr_append_c(boundary, c); } } The variable c is stored in a register, as you can see in the disassembly (built with -Og): hop% r2 build/src/dillo WARN: Relocs has not been applied. Please use `-e bin.relocs.apply=true` or `-e bin.cache=true` next time [0x000e3860]> s sym.generate_boundary_Dstr_ [0x0012cf99]> af [0x0012cf99]> pdf ┌ 85: sym.generate_boundary_Dstr_ (int64_t arg1); │ ; arg int64_t arg1 @ rdi │ 0x0012cf99 55 push rbp ; Fl_Pixmap.H:1250 ; generate_boundary(Dstr*) │ 0x0012cf9a 4889e5 mov rbp, rsp │ 0x0012cf9d 4155 push r13 │ 0x0012cf9f 4154 push r12 │ 0x0012cfa1 53 push rbx │ 0x0012cfa2 4883ec08 sub rsp, 8 │ 0x0012cfa6 4989fd mov r13, rdi ; arg1 │ 0x0012cfa9 41bc00000000 mov r12d, 0 ; Fl_Pixmap.H:1251 │ ┌─< 0x0012cfaf eb2c jmp 0x12cfdd │ ┌┌──> 0x0012cfb1 ff1509cd1e00 call qword [reloc.rand] ; Fl_Pixmap.H:1253 ; [0x319cc0:8]=0 │ ╎╎│ 0x0012cfb7 0fb6d8 movzx ebx, al ; <--- Perform the "& 0xff" by zero-extending the lowest byte in eax │ ╎╎│ 0x0012cfba 89de mov esi, ebx ; Fl_Pixmap.H:1260 │ ╎╎│ 0x0012cfbc 488d3dbd5b.. lea rdi, obj.generate_boundary_Dstr_::set ; 0x2a2b80 ; "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789" │ ╎╎│ 0x0012cfc3 ff15afc51e00 call qword [reloc.strchr] ; [0x319578:8]=0 │ ╎╎│ 0x0012cfc9 4885c0 test rax, rax │ └───< 0x0012cfcc 74e3 je 0x12cfb1 │ ╎│ 0x0012cfce 89de mov esi, ebx ; Fl_Pixmap.H:1262 │ ╎│ 0x0012cfd0 4c89ef mov rdi, r13 │ ╎│ 0x0012cfd3 67e830570800 call sym.dStr_append_c │ ╎│ 0x0012cfd9 4183c401 add r12d, 1 ; Fl_Pixmap.H:1251 │ ╎│ ; CODE XREF from generate_boundary(Dstr*) @ 0x12cfaf(x) │ ╎└─> 0x0012cfdd 4183fc45 cmp r12d, 0x45 ; 'E' │ └──< 0x0012cfe1 7ece jle 0x12cfb1 │ 0x0012cfe3 4883c408 add rsp, 8 ; Fl_Pixmap.H:1264 │ 0x0012cfe7 5b pop rbx │ 0x0012cfe8 415c pop r12 │ 0x0012cfea 415d pop r13 │ 0x0012cfec 5d pop rbp └ 0x0012cfed c3 ret This method gives us a perfect uniform distribution when RAND_MAX is a power of 2, at the cost of executing rand() more times than required (70/(62/256) ≈ 289 on average). This other method has a slightly not uniform distribution, but only runs rand() 70 times: static void generate_boundary(Dstr *boundary) { /* Extracted from RFC 2046, section 5.1.1. */ static const char set[] = "abcdefghijklmnopqrstuvwxyz" "ABCDEFGHIJKLMNOPQRSTUVWXYZ" "0123456789"; static const int n = strlen(set); for (int i = 0; i < 70; i++) { int c = (unsigned char) set[rand() % n]; dStr_append_c(boundary, c); } } Which is probably okay for this case. Notice I haven't tested any of these methods yet, I'll need to add a form upload test case to be able to see them in action (or a unit test). >> If we only use alphanumeric characters, we can just use isalnum() right? > >According to POSIX.1-2017 [2], isalnum(3) depends on the current locale >configured by the system. For example, characters such as Ä or ú could >return non-zero. Oh right, for some reason I was thinking this was the other way around, and isalnum only worked with ASCII. I think we should review the other uses of isalnum and friends as I think there may be used under similar assumptions. > To avoid this, there are two possible solutions: > >1. Use isalnum_l(3) to specify a locale_t object corresponding to the >"POSIX" locale (equivalent to "C" [3]), which must be previously >allocated by the newlocale(3) function [3] and released by the >freelocal(3) function [4]. A minimalist example is shown below: > > locale_t l = newlocale(LC_CTYPE, "POSIX", NULL); > > for (unsigned char i = 0; i < 255; i++) > printf("hhu=%hhu, c=%c, isalnum=%d\n", i, i, >isalnum_l(i, l)); > > freelocale(l); > >2. Define a known subset from the portable character set defined by >POSIX.1-2017 [5] and use strspn(3), as already suggested by the patch. >IMHO this approach is better because: > - It does not deal with locales, so developers not familiar with them >would understand the code better. > - It is also portable outside a POSIX environment (not sure if this a >requirement, though). > - It does not require dynamic allication via newlocale(3). > - It is the only possible option if non-alnum characters, such as ':' >or '/', are appended to the boundary string. Then, I think using our own set may be the most readable solution here, which also allows the quick module method. Best, Rodrigo

Sept. 1, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by Rodrigo Arias

Hi Alex, On Thu, Aug 29, 2024 at 12:04:12PM +0200, a1ex(a)dismail.de wrote: >Hi Rodrigo, > >On Wed, 28 Aug 2024 23:02:38 +0200 >Rodrigo Arias <rodarima(a)gmail.com> wrote: > >> I think your IP may be banned from narkive. > >Yeah, that does sound like it could be the case. Funny, because I've >never even used that site before. I have a static IP address, so that >might have something to do with it. I'm thinking if you ever ran an exit Tor node that could be a plausible explanation. >On the plus side, without this issue we wouldn't have found the crash >in Dillo, so, thank you narkive for inadvertently helping Dillo :) Yes, this is more a symptom that we don't have enough tests. I'm thinking how to improve the test infrastructure so we can include more complex tests like bad TLS responses from a https server, among others. Best, Rodrigo.

Sept. 1, 2024

Re: Firefox bookmarks plugin

by a1ex＠dismail.de

Hi, On Fri, 30 Aug 2024 10:58:57 +0200 Diego <darkspirit5000(a)gmail.com> wrote: > > It doesn't require describing the available dpis in the file. Not > > sure what would be the advantage of doing so. > > For reference. To explain to the user how to switch between a built-in > DPI and another DPI for the same service. I think this part in the docs[0] covers it: "Plugins are searched in the ~/.dillo/dpi/ directory first, and then in the system directory if not found. The dpi_dir option sets the system dpi directory and must be specified once." [0] https://dillo-browser.github.io/user_help.html#dpidrc It seems reasonable to me, but if you have a better way of wording it, that might be the best place to start. Regards, Alex

Aug. 30, 2024

Re: Issues with HTTP multipart/form-data file upload

by Xavier Del Campo Romero

Hi Rodrigo, > Dillo has a mechanism to read chunks of data from different sources as they are arriving and pass them to the next stage for processing. However, AFAIK it always reads a chunk and appends it to a large buffer. It doesn't free the processed part until is done with the whole thing. > > This would require a change in the way Dillo processes data, but I think it would be required for large files. There are more details in the > devdoc/CCCwork.txt file and in src/chain.c if you want to take a closer look. > > As I'm planning to change the design of the CCC, I think I can take this into account too so it would be doable. I'll add it to the list of shortcomings of the current design. Thank you. I am still unfamiliar with that part of Dillo, so please let me know about any progress. > Okay, I'll focus on the boundary patch first, which is the easiest to merge and then I'll take a closer look at the others. > > Yeah, I would assume a lot of implementations are broken, so we want to try to minimize the chances we run into problems. Limiting ourselves to a-z, A-Z and 0-9 would still account for 62 out of the 75 possible characters, so roughly 82% of the set. I think that removing the quoting in favour of the limited set reduce the risk for broken implementations, yet still provide a good amount of randomness. > Check sizeof " ": https://godbolt.org/z/7Tso8ooYz Interestingly, the " " character on your last email is not really a <space> (<U0020>): $ printf "%s" " " | hd 00000000 e2 80 88 |...| 00000003 Compared to an ASCII whitespace: $ printf "%s" " " | hd 00000000 20 | | 00000001 Both Godbolt and my editor also flag that multi-byte character with a yellow rectangle around it because it would be highly confusing otherwise. For example: printf("len=%zu\n", strlen(" ")); Confusingly returns "len=3". I am not sure whether this was an intentional modification from your side. My patch is adding a <space> as defined by POSIX.1-2017 [1], so that sizeof " " would always return 2. Was it your intention to flag this potential confusion? Also, there was not strict reason to use sizeof " ". Any other character would do e.g.: sizeof "x", sizeof "A", etc. > You can also use dStr_append_c() to only append one character, so you only need a single character. That would be an unnecessary use of the heap, because the size is static. > If we only use alphanumeric characters, we can just use isalnum() right? According to POSIX.1-2017 [2], isalnum(3) depends on the current locale configured by the system. For example, characters such as Ä or ú could return non-zero. To avoid this, there are two possible solutions: 1. Use isalnum_l(3) to specify a locale_t object corresponding to the "POSIX" locale (equivalent to "C" [3]), which must be previously allocated by the newlocale(3) function [3] and released by the freelocal(3) function [4]. A minimalist example is shown below: locale_t l = newlocale(LC_CTYPE, "POSIX", NULL); for (unsigned char i = 0; i < 255; i++) printf("hhu=%hhu, c=%c, isalnum=%d\n", i, i, isalnum_l(i, l)); freelocale(l); 2. Define a known subset from the portable character set defined by POSIX.1-2017 [5] and use strspn(3), as already suggested by the patch. IMHO this approach is better because: - It does not deal with locales, so developers not familiar with them would understand the code better. - It is also portable outside a POSIX environment (not sure if this a requirement, though). - It does not require dynamic allication via newlocale(3). - It is the only possible option if non-alnum characters, such as ':' or '/', are appended to the boundary string. [1]: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap06.html [2]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/isalnum.html [3]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/newlocale.html [4]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/freelocale.html [5]: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap06.html > I meant when is the next KoVoꓘ concert :-) No gigs ahead, but I will keep you informed. :) Best regards, Xavi On 28/8/24 22:47, Rodrigo Arias wrote: > Hi Xavier, > > On Wed, Aug 28, 2024 at 01:04:04AM +0200, Xavier Del Campo Romero wrote: >> Hi Rodrigo, >> >>> Glad to read that you also consider Dillo for slcl, and thanks for >>> preparing the patches :-) >> >> Thank you! I want slcl to be useful to anyone, including users who care >> about minimalist software like Dillo. The web is already too crowded >> with bloated "webapps" and other terrible things. :) > > Agreed! > >>> Sounds good, not sure how complicated it would be to do this. >> >> I still need to investigate this further, but I assume this would >> require Dillo to at least implement a sink callback. >> >> In other words, the component responsible for transmitting the data >> (probably src/IO/IO.c) should trigger a user-defined callback with an >> arbitrarily-sized buffer (typically, of BUFSIZ bytes, as defined by >> stdio.h) that must filled with file data. Then, the user-defined >> callback can fill from zero up to BUFSIZ bytes, which are eventually >> trasmitted to the server. > > Dillo has a mechanism to read chunks of data from different sources as > they are arriving and pass them to the next stage for processing. > However, AFAIK it always reads a chunk and appends it to a large buffer. > It doesn't free the processed part until is done with the whole thing. > > This would require a change in the way Dillo processes data, but I think > it would be required for large files. There are more details in the > devdoc/CCCwork.txt file and in src/chain.c if you want to take a closer > look. > > As I'm planning to change the design of the CCC, I think I can take this > into account too so it would be doable. I'll add it to the list of > shortcomings of the current design. > >> That said, I am still not sure how much actual effort this would take. >> But I am glad to receive positive feedback so far - I will then continue >> to find a solution. >> >>> However, being able to upload multiple files at the same time sounds >>> reasonable, so feel free to try on your own in the meanwhile. >> >> Uploading multiple files at once seems doable - the patches I sent on my >> previous email are probably already doing most of the required work. >> Again, the trickiest task is to send data on-the-fly for each selected >> file. > > Okay, I'll focus on the boundary patch first, which is the easiest to > merge and then I'll take a closer look at the others. > >> >>> Shouldn't it be 68 then? >> >> I understand the opposite: the boundary string with the two leading >> dashes ("--") included can be up to 72 bytes long, and 74 bytes long for >> the ending boundary (which includes two more dashes after the boundary >> string). This is confirmed by reading the BNF defined by RFC 2046 (some >> bits omitted for simplicity), section 5.1.1 [1]: >> >>> boundary := 0*69<bchars> bcharsnospace >>> bchars := bcharsnospace / " " >>> bcharsnospace := DIGIT / ALPHA / "'" / "(" / ")" / >>> "+" / "_" / "," / "-" / "." / >>> "/" / ":" / "=" / "?" >>> dash-boundary := "--" boundary >>> ; boundary taken from the value of >>> ; boundary parameter of the >>> ; Content-Type field> multipart-body := >>> [preamble CRLF] >>> dash-boundary transport-padding CRLF >>> body-part *encapsulation >>> close-delimiter transport-padding >>> [CRLF epilogue] >>> delimiter := CRLF dash-boundary >>> close-delimiter := delimiter "--" > > Oh right! I see that we are already using 70 characters anyway. > >> Note: even if the specification tells receivers to handle transport >> padding, for the time being I am assuming "transport-padding" as zero >> length since composers must not generate non-zero length transport >> padding. I am still not sure where transport padding would apply, >> anyway. Probably outside web browsers? >> >>> I would leave out all the symbols to avoid quoting and only use A-Z >>> a-z and 0-9. >> >> Interestingly, Dillo would always quote boundary strings [2], even if >> only using A-Z, a-z and 0-9. In fact, this is one of the wrong >> assumptions I spotted when testing slcl against Dillo. > > Yeah, I would assume a lot of implementations are broken, so we want to > try to minimize the chances we run into problems. > > Apart from slcl we should also test this with some sites and see if they > continue to work okay. > > This will also increase the fingerprinting information to distinguish > Dillo among other browsers, but I think it is not more information that > the already leaked by the user agent. > >>> Which, if I computed it correctly, is still too small to worry about. >> >> Not only it is too small of a chance: if we really wanted to do "the >> right thing" and make Dillo absolutely sure the boundary string is not >> contained within the selected files, this would imply a noticeable >> performance impact when dealing with large files, much likely for a >> near-zero benefit. >> >> I have not inspected their source code yet (and I do not want to), but I >> understand both Gecko and Chromium are also making that assumption, >> because otherwise it would take them a lot of CPU time to upload large >> files. > > But then they would be doing such assumption with a "much larger" > probability it hits the file. > > Skipping it with 70 characters is safe for one file, but also probably > safe for all files ever uploaded with Dillo. > > Maybe curl or other small codebases are easier to read, but not really > needed. > >> >>> Why sizeof " " instead of just 2? >> >> Because, to my eyes, sizeof " " has more meaningful semantics, compared >> to a magic integer constant such as 2. However, for this simple >> scenario, I would still consider both acceptable. > > Check sizeof " ": https://godbolt.org/z/7Tso8ooYz > > You can also use dStr_append_c() to only append one character, so you > only need a single character. > > If we only use alphanumeric characters, we can just use isalnum() right? > >> I can replace it with 2 if you find the other construct unacceptable. >> >>> PS: When are you playing? >> >> Sorry, I did not understand your last sentence. Could you please give a >> bit more context? :) > > I meant when is the next KoVoꓘ concert :-) > > Best, > Rodrigo. > _______________________________________________ > Dillo-dev mailing list -- dillo-dev(a)mailman3.com > To unsubscribe send an email to dillo-dev-leave(a)mailman3.com

Aug. 30, 2024