- Dillo-dev - lists.mailman3.com

Re: TLS connect error: "an EOF was observed that violates the protocol"

by Theo Buehler

> Nav_open_url: new url='https://narkive.com' > Dns_server [0]: narkive.com is 149.248.211.108 > Connecting to 149.248.211.108:443 > TLS connect error: "an EOF was observed that violates the protocol" > Tls_close_by_key: Avoiding SSL shutdown for: https://narkive.com > fd 9 is done and failed > dillo(84242) in free(): write to free mem 0xb5efb754c40[24..31]@32 ok, that's a use after free, perhaps triggered by the unexpected EOF... Can't repro even with vm.malloc_conf flags set. $ dillo https://narkive.com paths: Cannot open file '/home/tb/.dillo/dillorc': No such file or directory paths: Using /etc/dillo/dillorc paths: Cannot open file '/home/tb/.dillo/keysrc': No such file or directory paths: Using /etc/dillo/keysrc paths: Cannot open file '/home/tb/.dillo/domainrc': No such file or directory paths: Using /etc/dillo/domainrc Domain: Default accept. dillo_dns_init: Here we go! (threaded) TLS library: LibreSSL 3.9.0 Disabling cookies. paths: Cannot open file '/home/tb/.dillo/hsts_preload': No such file or directory paths: Cannot open file '/etc/dillo/hsts_preload': No such file or directory paths: Using internal defaults... Nav_open_url: new url='https://narkive.com' Dns_server [0]: narkive.com is 149.248.211.108 Connecting to 149.248.211.108:443 narkive.com: TLSv1.3, cipher TLS_AES_128_GCM_SHA256 sha256 2048-bit RSA: /CN=narkive.com sha256 2048-bit RSA: /C=US/O=Let's Encrypt/CN=R11 root: /C=US/O=Internet Security Research Group/CN=ISRG Root X1 NumPendingStyleSheets=1 Dns_server [0]: narkive.net is 188.114.96.12 188.114.97.12 2a06:98c1:3120::c 2a06:98c1:3121::c narkive.net: TLSv1.3, cipher TLS_AES_256_GCM_SHA384 ecdsa-with-SHA256 256-bit EC: /CN=narkive.net ecdsa-with-SHA384 256-bit EC: /C=US/O=Google Trust Services/CN=WE1 sha256 384-bit EC: /C=US/O=Google Trust Services LLC/CN=GTS Root R4 root: /C=BE/O=GlobalSign nv-sa/OU=Root CA/CN=GlobalSign Root CA >>>> a_Nav_repush <<<< Nav_open_url: new url='https://narkive.com' a_Nav_expect_done: repush! Dillo: normal exit! > gdb: > Program received signal SIGABRT, Aborted. > thrkill () at /tmp/-:2 > 2 /tmp/-: No such file or directory. > in /tmp/- Please pkg_add gdb, use 'egdb dillo dillo.core and show the output of 'bt full', that might give a clue. > dillo(84242) in free(): write to free mem 0xb5efb754c40[24..31]@32 If you manage figure out what was at 0xb5efb754c40 that would help a lot. > My nc output matches yours, but ftp still gets the handshake > failure. Maybe it's something network specific on my end? But like I > mentioned, an older OpenBSD system on my network with LibreSSL 3.6.0 > does not have an issue with that site. I disabled pf just in case, but > to no avail. No real idea at this point - sorry.

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by a1ex＠dismail.de

Hi, On Wed, 28 Aug 2024 13:45:04 +0200 Theo Buehler <tb(a)theobuehler.org> wrote: > > Certainly. Ideally this should not crash Dillo, no matter how > > obscure. > > What crashes dillo? Opening https://narkive.com crashes Dillo for me, It doesn't happen on any other sites: Nav_open_url: new url='https://narkive.com' Dns_server [0]: narkive.com is 149.248.211.108 Connecting to 149.248.211.108:443 TLS connect error: "an EOF was observed that violates the protocol" Tls_close_by_key: Avoiding SSL shutdown for: https://narkive.com fd 9 is done and failed dillo(84242) in free(): write to free mem 0xb5efb754c40[24..31]@32 gdb: Program received signal SIGABRT, Aborted. thrkill () at /tmp/-:2 2 /tmp/-: No such file or directory. in /tmp/- > > I don't know why it shows that version number instead of the latest > > one, this is a fresh install of a recent snapshot. > > It's a bit unfortunate, but the development release on current openbsd > snapshots still has 3.9.0 as the version number. It's complicated... > The short story is that we can't crank to 4.0.0 because it would break > the build of some important ports (e.g., rust). Thank you for clarifying this. > > $ ftp https://narkive.com/test > > Trying 149.248.211.108... > > TLS handshake failure: handshake failed: unexpected EOF > > I don't see this: > > $ ftp https://narkive.com/test > Trying 149.248.211.108... > Requesting https://narkive.com/test > ftp: Error retrieving https://narkive.com/test: 404 Not Found > $ nc -cvz narkive.com 443 > Connection to narkive.com (149.248.211.108) 443 port [tcp/https] > succeeded! TLS handshake negotiated TLSv1.3/TLS_AES_128_GCM_SHA256 > with host narkive.com Peer name: narkive.com > Subject: /CN=narkive.com > Issuer: /C=US/O=Let's Encrypt/CN=R11 > Valid From: Fri Jul 26 06:13:58 2024 > Valid Until: Thu Oct 24 06:13:57 2024 > Cert Hash: > SHA256:456275146f4b2e65c16d2432ff87917a4501ddb41dcb158195255c5995abbbf1 > OCSP URL: http://r11.o.lencr.org > > This looks all good from here. My nc output matches yours, but ftp still gets the handshake failure. Maybe it's something network specific on my end? But like I mentioned, an older OpenBSD system on my network with LibreSSL 3.6.0 does not have an issue with that site. I disabled pf just in case, but to no avail. I'm open to any clues on how to diagnose this! Regards, Alex

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by Theo Buehler

On Wed, Aug 28, 2024 at 01:25:21PM +0200, a1ex(a)dismail.de wrote: > Hi Rodrigo, > > On Tue, 27 Aug 2024 23:09:42 +0200 > Rodrigo Arias <rodarima(a)gmail.com> wrote: > > > >> >dillo-dev.auriga.wearlab.narkive.com:443 > > >> > > > >> >Is this site just broken/misconfigured? > > >> > > >> Cannot reproduce with LibreSSL 3.9.2 on Linux. > > >> ... > > >> Can you test with the latest LibreSSL 3.9.2? > > >> > > >> Is this it happening with the proxy enabled? Also, which user agent > > >> are you using in curl and Dillo? > > > > > >It happens with the stock user agent and no proxy, same with curl. > > >I'm running the latest snapshot of OpenBSD, which would have the > > >latest version of LibreSSL. > > > > > >I don't care about that site, my only worry is that the error can > > >crash Dillo. > > > > If it happens it is a bug on Dillo side as it is not handling all > > errors, regardless of the site. I would like to reproduce it to fix > > it. > > Certainly. Ideally this should not crash Dillo, no matter how obscure. What crashes dillo? > > >This could be an OpenBSD specific issue which wouldn't show up on > > >Linux. > > > > > It could be, but I would first reject that is not due to mismatch of > > versions. > > > > The last LibreSSL as per https://www.libressl.org/ is: > > > > > The latest stable release is 3.9.2 > > > > Which should be printed in the first lines when starting Dillo: > > > > > TLS library: LibreSSL 3.9.2 > > > > If it says 3.9.0, then Dillo is not using the last release. > > I don't know why it shows that version number instead of the latest > one, this is a fresh install of a recent snapshot. It's a bit unfortunate, but the development release on current openbsd snapshots still has 3.9.0 as the version number. It's complicated... The short story is that we can't crank to 4.0.0 because it would break the build of some important ports (e.g., rust). > Anyway, I installed 3.9.2 from source and built Dillo against that. > Now it reports the correct version, but the crash still happens the > same. > > I guess we would need to hear from some other OpenBSD users to confirm > if this a real issue, or if its something whacky on my end. > > I do have an older OpenBSD system which uses LibreSSL 3.6.0, and it does > not exhibit the problem. But on 3 newer systems here the problem occurs. > > Here is an easy way to confirm using only OpenBSD base tools: > > $ ftp https://narkive.com/test > Trying 149.248.211.108... > TLS handshake failure: handshake failed: unexpected EOF I don't see this: $ ftp https://narkive.com/test Trying 149.248.211.108... Requesting https://narkive.com/test ftp: Error retrieving https://narkive.com/test: 404 Not Found $ nc -cvz narkive.com 443 Connection to narkive.com (149.248.211.108) 443 port [tcp/https] succeeded! TLS handshake negotiated TLSv1.3/TLS_AES_128_GCM_SHA256 with host narkive.com Peer name: narkive.com Subject: /CN=narkive.com Issuer: /C=US/O=Let's Encrypt/CN=R11 Valid From: Fri Jul 26 06:13:58 2024 Valid Until: Thu Oct 24 06:13:57 2024 Cert Hash: SHA256:456275146f4b2e65c16d2432ff87917a4501ddb41dcb158195255c5995abbbf1 OCSP URL: http://r11.o.lencr.org This looks all good from here. > This doesn't happen on any other site that I have seen. > > Maybe I should report this to the OpenBSD/LibreSSL people as well, so > I'm CC'ing tb@ What is an example of a site that you can't connect to or even crashes?

Aug. 28, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by a1ex＠dismail.de

Hi Rodrigo, On Tue, 27 Aug 2024 23:09:42 +0200 Rodrigo Arias <rodarima(a)gmail.com> wrote: > >> >dillo-dev.auriga.wearlab.narkive.com:443 > >> > > >> >Is this site just broken/misconfigured? > >> > >> Cannot reproduce with LibreSSL 3.9.2 on Linux. > >> ... > >> Can you test with the latest LibreSSL 3.9.2? > >> > >> Is this it happening with the proxy enabled? Also, which user agent > >> are you using in curl and Dillo? > > > >It happens with the stock user agent and no proxy, same with curl. > >I'm running the latest snapshot of OpenBSD, which would have the > >latest version of LibreSSL. > > > >I don't care about that site, my only worry is that the error can > >crash Dillo. > > If it happens it is a bug on Dillo side as it is not handling all > errors, regardless of the site. I would like to reproduce it to fix > it. Certainly. Ideally this should not crash Dillo, no matter how obscure. > >This could be an OpenBSD specific issue which wouldn't show up on > >Linux. > > > It could be, but I would first reject that is not due to mismatch of > versions. > > The last LibreSSL as per https://www.libressl.org/ is: > > > The latest stable release is 3.9.2 > > Which should be printed in the first lines when starting Dillo: > > > TLS library: LibreSSL 3.9.2 > > If it says 3.9.0, then Dillo is not using the last release. I don't know why it shows that version number instead of the latest one, this is a fresh install of a recent snapshot. Anyway, I installed 3.9.2 from source and built Dillo against that. Now it reports the correct version, but the crash still happens the same. I guess we would need to hear from some other OpenBSD users to confirm if this a real issue, or if its something whacky on my end. I do have an older OpenBSD system which uses LibreSSL 3.6.0, and it does not exhibit the problem. But on 3 newer systems here the problem occurs. Here is an easy way to confirm using only OpenBSD base tools: $ ftp https://narkive.com/test Trying 149.248.211.108... TLS handshake failure: handshake failed: unexpected EOF This doesn't happen on any other site that I have seen. Maybe I should report this to the OpenBSD/LibreSSL people as well, so I'm CC'ing tb@ Regards, Alex

Aug. 28, 2024

Re: Issues with HTTP multipart/form-data file upload

by Xavier Del Campo Romero

Hi Rodrigo, > Glad to read that you also consider Dillo for slcl, and thanks for preparing the patches :-) Thank you! I want slcl to be useful to anyone, including users who care about minimalist software like Dillo. The web is already too crowded with bloated "webapps" and other terrible things. :) > Sounds good, not sure how complicated it would be to do this. I still need to investigate this further, but I assume this would require Dillo to at least implement a sink callback. In other words, the component responsible for transmitting the data (probably src/IO/IO.c) should trigger a user-defined callback with an arbitrarily-sized buffer (typically, of BUFSIZ bytes, as defined by stdio.h) that must filled with file data. Then, the user-defined callback can fill from zero up to BUFSIZ bytes, which are eventually trasmitted to the server. That said, I am still not sure how much actual effort this would take. But I am glad to receive positive feedback so far - I will then continue to find a solution. > However, being able to upload multiple files at the same time sounds reasonable, so feel free to try on your own in the meanwhile. Uploading multiple files at once seems doable - the patches I sent on my previous email are probably already doing most of the required work. Again, the trickiest task is to send data on-the-fly for each selected file. > Shouldn't it be 68 then? I understand the opposite: the boundary string with the two leading dashes ("--") included can be up to 72 bytes long, and 74 bytes long for the ending boundary (which includes two more dashes after the boundary string). This is confirmed by reading the BNF defined by RFC 2046 (some bits omitted for simplicity), section 5.1.1 [1]: > boundary := 0*69<bchars> bcharsnospace > bchars := bcharsnospace / " " > bcharsnospace := DIGIT / ALPHA / "'" / "(" / ")" / > "+" / "_" / "," / "-" / "." / > "/" / ":" / "=" / "?" > dash-boundary := "--" boundary > ; boundary taken from the value of > ; boundary parameter of the > ; Content-Type field> multipart-body := [preamble CRLF] > dash-boundary transport-padding CRLF > body-part *encapsulation > close-delimiter transport-padding > [CRLF epilogue] > delimiter := CRLF dash-boundary > close-delimiter := delimiter "--" Note: even if the specification tells receivers to handle transport padding, for the time being I am assuming "transport-padding" as zero length since composers must not generate non-zero length transport padding. I am still not sure where transport padding would apply, anyway. Probably outside web browsers? > I would leave out all the symbols to avoid quoting and only use A-Z a-z and 0-9. Interestingly, Dillo would always quote boundary strings [2], even if only using A-Z, a-z and 0-9. In fact, this is one of the wrong assumptions I spotted when testing slcl against Dillo. > Which, if I computed it correctly, is still too small to worry about. Not only it is too small of a chance: if we really wanted to do "the right thing" and make Dillo absolutely sure the boundary string is not contained within the selected files, this would imply a noticeable performance impact when dealing with large files, much likely for a near-zero benefit. I have not inspected their source code yet (and I do not want to), but I understand both Gecko and Chromium are also making that assumption, because otherwise it would take them a lot of CPU time to upload large files. > Why sizeof " " instead of just 2? Because, to my eyes, sizeof " " has more meaningful semantics, compared to a magic integer constant such as 2. However, for this simple scenario, I would still consider both acceptable. I can replace it with 2 if you find the other construct unacceptable. > PS: When are you playing? Sorry, I did not understand your last sentence. Could you please give a bit more context? :) Best regards, Xavi [1]: https://www.rfc-editor.org/rfc/rfc2046.html#section-5.1.1 [2]: https://github.com/dillo-browser/dillo/blob/8a360e32ac3136494a494379a6dbbac… On 27/8/24 22:59, Rodrigo Arias wrote: > Hi Xavier, > > On Mon, Aug 26, 2024 at 01:27:29AM +0200, Xavier Del Campo Romero wrote: >> Hello Dillo dev community, >> >> I am testing support among web browsers for slcl [1], a JS-less >> minimalist web storage solution. slcl relies on >> "multipart/form-data"-encoded HTTP requests to upload files to a server. >> Whereas Dillo helped me to uncover a few wrong assumptions on my code, I >> have realised a few issues on Dillo itself related to filue uploads that >> should be considered. > > Glad to read that you also consider Dillo for slcl, and thanks for > preparing the patches :-) > >> Issue #1: >> >> While Dillo is fine uploading small files (up to a few MiB), things go >> wrong with larger files, so much that memory usage increases up to >> multiple GiB and can even lock the system up. This is because Dillo is >> designed to send requests always from memory, which means file contents >> must be dumped into memory first, and this might be unfeasible for large >> files. >> >> Suggestion: >> >> Instead, Dillo should ideally send file contents on-the-fly, so that >> memory usage is kept to a minimum regardless the file size. > > Sounds good, not sure how complicated it would be to do this. > >> Issue #2: >> >> Even if Dillo generates a 70-byte, random boundary string (yet mostly >> filled with '-', similarly to Firefox [2]), it ensures it is not found >> anywhere inside the file contents. Again, this can be a serious >> bottleneck in the case of large files, as it requires to scan the whole >> file for a match. >> >> Suggestion: >> >> Define *all* of the 70 bytes in the boundary string as random, and >> assume they would never be found inside a file. The chance of accidental >> collision is so low that it is not worth the effort into checking them. >> >> Suggested patches: >> >> - 0001-dialog.cc-Generate-more-random-boundaries.patch) > > Yes, I think is a good idea. > > @@ -1246,6 +1246,24 @@ Dstr *DilloHtmlForm::buildQueryData(DilloHtmlInput > *active_submit) > return DataStr; > } > > +static void generate_boundary(Dstr *boundary) > +{ > + for (int i = 0; i < 70; i++) { > > I think this is too long: > >> Boundary delimiters must not appear within the encapsulated material, >> and must be no longer than 70 characters, not counting the two >> leading hyphens. > > Shouldn't it be 68 then? > > + /* Extracted from RFC 2046, section 5.1.1. */ > + static const char set[] = "abcdefghijklmnopqrstuvwxyz" > + "ABCDEFGHIJKLMNOPQRSTUVWXYZ" > + "0123456789" > + "'()+_,-./:=? "; > > I would leave out all the symbols to avoid quoting and only use A-Z a-z > and 0-9. > > Assuming you are left with 62 symbols (26*2 + 10), the probability of > finding a random 70 character string in a random sample of 70 characters > is (using ** as ^): > > p = 1/62**70 = 3.4086e-126 > > But as you will be dealing with large files, each line of the input > could match it. We are interested in the probability that a given > boundary will mach any of the N lines. So we compute the probability > that it will match none of the lines (1-p)**N and then the opposite of > that (matches at least one line): > > 1 - (1 - p) ** N > > For example, a 1 TB encoded file, with 2**40/70 lines will have: > > p = 1 - (1 - 1/62**70) ** (2**40 / 70) > > Of occurring, which is around 5.354e-116. > > Which, if I computed it correctly, is still too small to worry about. > > + char s[sizeof " "] = {0}; > > Why sizeof " " instead of just 2? > > + > + do { > + *s = rand(); > + } while (!strspn(s, set)); > + > + dStr_append(boundary, s); > + } > +} > + > /** > * Generate a boundary string for use in separating the parts of a > * multipart/form-data submission. > >> Issue #3: >> >> Dillo only supports uploading 1 file at a time. This is mostly because >> it relies (probably temporarily?) on the a_Dialog_save_file function >> [3]. However, this is not a limitation on the HTTP protocol, and other >> implementation such as Firefox or Chromium-based browsers support this. >> >> Suggested patches: >> >> - 0002-dialog-Add-a_Dialog_select_files.patch >> - 0003-WIP-multi-file-uploads.patch >> >> Conclusions: >> >> Dillo seems designed to always send requests from memory, so it is not >> straightforward to break this assumption in order to support large file >> uploads. 0003-WIP-multi-file-uploads.patch is an incomplete first step >> into fixing this, but it surely needs deeper design changes. >> >> I did not put more effort into these patches for the time being because, >> after seeing the potential complexity behind this task, I thought it was >> a better idea to ask the community for feedback and guidelines. > > I'm not very familiar with this part of Dillo, so I'll have to check a > bit more before providing more feedback. > > However, being able to upload multiple files at the same time sounds > reasonable, so feel free to try on your own in the meanwhile. > > PS: When are you playing? > > Best, > Rodrigo. > _______________________________________________ > Dillo-dev mailing list -- dillo-dev(a)mailman3.com > To unsubscribe send an email to dillo-dev-leave(a)mailman3.com

Aug. 27, 2024

Re: Firefox bookmarks plugin

by Rodrigo Arias

Hi, On Mon, Aug 26, 2024 at 01:31:13AM +0200, Diego wrote: >Good work and very interesting idea. Thanks! >If I can suggest a change I would add this lines to dpidrc at Makefile >the installation > ># Default dillo bookmarks dpi >#bookmarks=bookmarks/bookmarks.dpi ># ffbm: A Dillo plugin to sync bookmarks with Firefox >bookmarks=$(DPI_DIR)/$(BIN) > > >maybe it's time to patch dpidrc to make it more descriptive > >I sugest this >dpi_dir=/usr/local/lib/dillo/dpi > ># standard dillo distributed DPIs >#bookmarks=bookmarks/bookmarks.dpi >#cookies=cookies/cookies.dpi >#downloads=downloads/downloads.dpi >#vsource=vsource/vsource.filter.dpi The file dpidrc only registers the protocols (ftp: -> ftp dpi), the dpis are taken from the dpi_dir or ~/.dillo/dpi automatically. It doesn't require describing the available dpis in the file. Not sure what would be the advantage of doing so. Best, Rodrigo.

Aug. 27, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by Rodrigo Arias

Hi Alex, On Tue, Aug 27, 2024 at 09:56:46PM +0200, a1ex(a)dismail.de wrote: >Hi Rodrigo, > >On Tue, 27 Aug 2024 20:19:32 +0200 >Rodrigo Arias <rodarima(a)gmail.com> wrote: > >> >1) socks5 proxy on 127.0.0.1:8080 not working. >> >> There is currently no support for socks5 proxies, only HTTP proxies >> are supported. > >Thanks for confirming, the docs are not really clear on that. >Could be a nice feature for the future :) Yeah, unfortunately I only have two hands :-) >> >dillo-dev.auriga.wearlab.narkive.com:443 >> > >> >Is this site just broken/misconfigured? >> >> Cannot reproduce with LibreSSL 3.9.2 on Linux. >> ... >> Can you test with the latest LibreSSL 3.9.2? >> >> Is this it happening with the proxy enabled? Also, which user agent >> are you using in curl and Dillo? > >It happens with the stock user agent and no proxy, same with curl. I'm >running the latest snapshot of OpenBSD, which would have the latest >version of LibreSSL. > >I don't care about that site, my only worry is that the error can crash >Dillo. If it happens it is a bug on Dillo side as it is not handling all errors, regardless of the site. I would like to reproduce it to fix it. >This could be an OpenBSD specific issue which wouldn't show up on >Linux. Maybe this is a similar issue: >https://github.com/mitmproxy/mitmproxy/issues/7128 It could be, but I would first reject that is not due to mismatch of versions. The last LibreSSL as per https://www.libressl.org/ is: > The latest stable release is 3.9.2 Which should be printed in the first lines when starting Dillo: > TLS library: LibreSSL 3.9.2 If it says 3.9.0, then Dillo is not using the last release. I can try installing 3.9.0 and see if I can reproduce it with that one. Best, Rodrigo.

Aug. 27, 2024

Re: Issues with HTTP multipart/form-data file upload

by Rodrigo Arias

Hi Xavier, On Mon, Aug 26, 2024 at 01:27:29AM +0200, Xavier Del Campo Romero wrote: >Hello Dillo dev community, > >I am testing support among web browsers for slcl [1], a JS-less >minimalist web storage solution. slcl relies on >"multipart/form-data"-encoded HTTP requests to upload files to a server. >Whereas Dillo helped me to uncover a few wrong assumptions on my code, I >have realised a few issues on Dillo itself related to filue uploads that >should be considered. Glad to read that you also consider Dillo for slcl, and thanks for preparing the patches :-) >Issue #1: > >While Dillo is fine uploading small files (up to a few MiB), things go >wrong with larger files, so much that memory usage increases up to >multiple GiB and can even lock the system up. This is because Dillo is >designed to send requests always from memory, which means file contents >must be dumped into memory first, and this might be unfeasible for large >files. > >Suggestion: > >Instead, Dillo should ideally send file contents on-the-fly, so that >memory usage is kept to a minimum regardless the file size. Sounds good, not sure how complicated it would be to do this. >Issue #2: > >Even if Dillo generates a 70-byte, random boundary string (yet mostly >filled with '-', similarly to Firefox [2]), it ensures it is not found >anywhere inside the file contents. Again, this can be a serious >bottleneck in the case of large files, as it requires to scan the whole >file for a match. > >Suggestion: > >Define *all* of the 70 bytes in the boundary string as random, and >assume they would never be found inside a file. The chance of accidental >collision is so low that it is not worth the effort into checking them. > >Suggested patches: > >- 0001-dialog.cc-Generate-more-random-boundaries.patch) Yes, I think is a good idea. @@ -1246,6 +1246,24 @@ Dstr *DilloHtmlForm::buildQueryData(DilloHtmlInput *active_submit) return DataStr; } +static void generate_boundary(Dstr *boundary) +{ + for (int i = 0; i < 70; i++) { I think this is too long: > Boundary delimiters must not appear within the encapsulated material, > and must be no longer than 70 characters, not counting the two > leading hyphens. Shouldn't it be 68 then? + /* Extracted from RFC 2046, section 5.1.1. */ + static const char set[] = "abcdefghijklmnopqrstuvwxyz" + "ABCDEFGHIJKLMNOPQRSTUVWXYZ" + "0123456789" + "'()+_,-./:=? "; I would leave out all the symbols to avoid quoting and only use A-Z a-z and 0-9. Assuming you are left with 62 symbols (26*2 + 10), the probability of finding a random 70 character string in a random sample of 70 characters is (using ** as ^): p = 1/62**70 = 3.4086e-126 But as you will be dealing with large files, each line of the input could match it. We are interested in the probability that a given boundary will mach any of the N lines. So we compute the probability that it will match none of the lines (1-p)**N and then the opposite of that (matches at least one line): 1 - (1 - p) ** N For example, a 1 TB encoded file, with 2**40/70 lines will have: p = 1 - (1 - 1/62**70) ** (2**40 / 70) Of occurring, which is around 5.354e-116. Which, if I computed it correctly, is still too small to worry about. + char s[sizeof " "] = {0}; Why sizeof " " instead of just 2? + + do { + *s = rand(); + } while (!strspn(s, set)); + + dStr_append(boundary, s); + } +} + /** * Generate a boundary string for use in separating the parts of a * multipart/form-data submission. >Issue #3: > >Dillo only supports uploading 1 file at a time. This is mostly because >it relies (probably temporarily?) on the a_Dialog_save_file function >[3]. However, this is not a limitation on the HTTP protocol, and other >implementation such as Firefox or Chromium-based browsers support this. > >Suggested patches: > >- 0002-dialog-Add-a_Dialog_select_files.patch >- 0003-WIP-multi-file-uploads.patch > >Conclusions: > >Dillo seems designed to always send requests from memory, so it is not >straightforward to break this assumption in order to support large file >uploads. 0003-WIP-multi-file-uploads.patch is an incomplete first step >into fixing this, but it surely needs deeper design changes. > >I did not put more effort into these patches for the time being because, >after seeing the potential complexity behind this task, I thought it was >a better idea to ask the community for feedback and guidelines. I'm not very familiar with this part of Dillo, so I'll have to check a bit more before providing more feedback. However, being able to upload multiple files at the same time sounds reasonable, so feel free to try on your own in the meanwhile. PS: When are you playing? Best, Rodrigo.

Aug. 27, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by a1ex＠dismail.de

Hi Rodrigo, On Tue, 27 Aug 2024 20:19:32 +0200 Rodrigo Arias <rodarima(a)gmail.com> wrote: > >1) socks5 proxy on 127.0.0.1:8080 not working. > > There is currently no support for socks5 proxies, only HTTP proxies > are supported. Thanks for confirming, the docs are not really clear on that. Could be a nice feature for the future :) > >dillo-dev.auriga.wearlab.narkive.com:443 > > > >Is this site just broken/misconfigured? > > Cannot reproduce with LibreSSL 3.9.2 on Linux. > ... > Can you test with the latest LibreSSL 3.9.2? > > Is this it happening with the proxy enabled? Also, which user agent > are you using in curl and Dillo? It happens with the stock user agent and no proxy, same with curl. I'm running the latest snapshot of OpenBSD, which would have the latest version of LibreSSL. I don't care about that site, my only worry is that the error can crash Dillo. This could be an OpenBSD specific issue which wouldn't show up on Linux. Maybe this is a similar issue: https://github.com/mitmproxy/mitmproxy/issues/7128 -Alex

Aug. 27, 2024

Re: TLS connect error: "an EOF was observed that violates the protocol"

by Rodrigo Arias

Hi Alex, On Tue, Aug 27, 2024 at 11:55:04AM +0200, a1ex(a)dismail.de wrote: >Hi, > >Here are 2 separate issues: > >1) socks5 proxy on 127.0.0.1:8080 not working. There is currently no support for socks5 proxies, only HTTP proxies are supported. >This is a proxy over ssh, for example: >ssh -N -D 8080 user(a)example.com > >in dillorc: >http_proxy="http://localhost:8080/" > >console output when trying to connect to a site: >Connecting to 127.0.0.1:8080 >CONNECT through proxy failed. Full reply not received: >(nothing) >** WARNING **: CCC: call on already finished chain. Flags=CCC_Ended > >This setup works fine under Firefox. > >I tried doing a tcpdump while attempting the connection, and there is no >activity. > >So, while trying to research this, I ran into another issue: > >2) Any time I go to a page on this site, Dillo crashes with the >following: > >Nav_open_url: new >url='https://dillo-dev.auriga.wearlab.narkive.com/WT0JYUZq/dillo-won-t-resolve-u…' Dns_server [0]: dillo-dev.auriga.wearlab.narkive.com is 149.248.211.108 >Connecting to 149.248.211.108:443 >TLS connect error: "an EOF was observed that violates the protocol" >Tls_close_by_key: Avoiding SSL shutdown >for: https://dillo-dev.auriga.wearlab.narkive.com/WT0JYUZq/dillo-won-t-resolve-u… fd 6 is done and failed >dillo(13689) in malloc(): write to free mem 0x60383b59140[24..31]@32 >Abort trap > >gdb output: >Program received signal SIGABRT, Aborted. >thrkill () at /tmp/-:2 >2 /tmp/-: No such file or directory. > in /tmp/- > >This is on OpenBSD-current amd64 with LibreSSL 3.9.0, running an >unmodified fresh checkout of Dillo master. Also tested on OpenBSD 7.5 >with the same result. > >I tested the site with: >https://www.ssllabs.com/ssltest/analyze.html?d=dillo-dev.auriga.wearlab.nar… > >There seem to be some handshake failures during the simulation. > >This probably is not be the fault of Dillo, but maybe there is a more >graceful to handle this, rather than crashing. > >Even a test with curl has issues: > >curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to >dillo-dev.auriga.wearlab.narkive.com:443 > >Is this site just broken/misconfigured? Cannot reproduce with LibreSSL 3.9.2 on Linux. % LD_LIBRARY_PATH=/usr/lib/libressl src/dillo https://dillo-dev.auriga.wearlab.narkive.com/WT0JYUZq/dillo-won-t-resolve-u… dillo_dns_init: Here we go! (threaded) TLS library: LibreSSL 3.9.2 Enabling cookies as from cookiesrc... Nav_open_url: new url='https://dillo-dev.auriga.wearlab.narkive.com/WT0JYUZq/dillo-won-t-resolve-u…' Dns_server [0]: dillo-dev.auriga.wearlab.narkive.com is 149.248.211.108 Connecting to 149.248.211.108:443 dillo-dev.auriga.wearlab.narkive.com: TLSv1.3, cipher TLS_AES_128_GCM_SHA256 sha256 2048-bit RSA: /CN=narkive.com sha256 2048-bit RSA: /C=US/O=Let's Encrypt/CN=R11 root: /C=US/O=Internet Security Research Group/CN=ISRG Root X1 NumPendingStyleSheets=1 Dns_server [0]: narkive.net is 188.114.97.5 188.114.96.5 narkive.net: TLSv1.3, cipher TLS_AES_256_GCM_SHA384 ecdsa-with-SHA256 256-bit EC: /CN=narkive.net ecdsa-with-SHA384 256-bit EC: /C=US/O=Google Trust Services/CN=WE1 sha256 384-bit EC: /C=US/O=Google Trust Services LLC/CN=GTS Root R4 root: /C=BE/O=GlobalSign nv-sa/OU=Root CA/CN=GlobalSign Root CA Can you test with the latest LibreSSL 3.9.2? Is this it happening with the proxy enabled? Also, which user agent are you using in curl and Dillo? Best, Rodrigo.

Aug. 27, 2024