Hi Kevin, On Thu, Jun 20, 2024 at 04:07:27PM +1000, Kevin Koster wrote:
On Thu, 20 Jun 2024 10:03:59 +1000 "Kevin Koster" <dillo@ombertech.com> wrote:
Rodrigo Arias <rodarima-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
However, if you start Dillo from a shell where the http_proxy variable is set in the environment, I would imagine it would be already picked by Dillo and the internal wget command. The dpid daemon has to start from this shell, so you should do a "dpidc stop" command before to ensure it.
I'm not sure if we're on the same page, Wget supports http_proxy and https_proxy environment variables. The former may or may not tunnel HTTPS requests through a Web proxy, but that's not what I want since then the proxy can't manipulate the data and the client needs to handle the encryption. https_proxy with Wget uses the proxy to do all the encryption and therefore those issues don't apply. More browsers support the HTTPS tunneling proxy feature, but that doesn't achieve anything that I'm interested in personally.
I did some tests and yes the https_proxy environment variable is picked up by Wget for the downloads DPI. It doesn't affect Dillo itself. The http_proxy environment variable is picked up by Dillo, but as I say that doesn't do what I want with Dillo or Wget.
It appears that GNU Wget's behaviour has changed. Newer versions _do_ try to tunnel the HTTPS connection through the proxy set to https_proxy (not what I want), whereas older versions ask that proxy to establish the HTTPS connection and send data back unencrypted (which is what I want). So I picked a bad example with Wget. See that blog post for other examples like Lynx, but I haven't tried them myself.
I can't find this change in the Wget changelog. One version that works as I intend it to is GNU Wget 1.9, circa 2003. Looking through the Wget mailing list archives it seems like many people either considered the old behaviour a bug or a useless feature, so I may be swimming against the tide.
My content-modifying proxy application might be very niche, but I still think it would be more widely useful for people running Dillo on very old computers. Let me know whether it's something that might be accepted for Dillo even if I decide to try implementing it myself.
Sorry for the confusion.
Provided that those machines cannot deal with HTTPS traffic, I think is a reasonable use case. I haven't had time to dig a bit more into this issue to see if there is a way to make those old machines use TLS instead, which I think it would be preferred. I won't oppose to it, but we probably want to ensure that we don't send plain traffic to a proxy when browsing HTTPS pages by accident. We should add an option like "allow_insecure_proxy" (false by default) or similar, so it is harder to leak HTTPS traffic accidentally. In any case, support for https_proxy should be added to Dillo first. Best, Rodrigo.