- Dillo-dev - lists.mailman3.com

[Dillo-dev]Dillo doesn't work!! :(

by Andrea Spadaccini

Hi everyone, i compiled and installed Dillo 0.83, but it doesn't open anything! Here there are the error messages I get: [andrea@iridiumII andrea]$ dillo dillorc:98: unexpected string constant "force_visited_color", expected symbol dillo_dns_init: Here we go! Disabling cookies. Nav_open_url: Url=>about:splash< Nav_open_url: Url=>dpi:/bm/< dpi_socket_dir.c:61: tst_dir: access : No such file or directory - /tmp/andrea-0c137u/ debug_msg - init_sockdir: The socket directory /tmp/andrea-0c137u does not exist or is not a directory dpid.c:500: init_all_dpi_sockets: bind : Address already in use/tmp/andrea-3vbac6/bookmarks.dpi dpid started Capi_dpi_conn_timeout:: try 1 main.c:153: get_command: dpid tag is NULL : main.c:326: main: get_command failed : ** ERROR **: file main.c: line 351 (main): assertion failed: (numsocks >= 0) aborting... waitpid: No child processes IO_write_chunk: Pipe rotta It seems like it cannot access the dir /tmp/andrea-0c137u, but it should be able to do it: [andrea@iridiumII andrea]$ ll /tmp/ totale 4 drwx------ 2 andrea users 4096 2004-11-30 00:59 andrea-3vbac6/ srwxr-xr-x 1 andrea users 0 2004-11-30 00:59 sylpheed-1000= srwxr-xr-x 1 andrea users 0 2004-11-29 22:20 xmms_andrea.0= srwxr-xr-x 1 giorgia users 0 2004-11-03 16:57 xmms_giorgia.0= srwxr-xr-x 1 luther users 0 2004-11-27 11:50 xmms_luther.0= [andrea@iridiumII andrea]$ cd /tmp/andrea-3vbac6/ [andrea@iridiumII andrea-3vbac6]$ ls bookmarks.dpi= dpid.srs= ftp.filter.dpi= https.filter.dpi= downloads.dpi= file.dpi= hello.filter.dpi= I get the same problems if I'm root too, so it shouldn't be a permissions problem... I've tried installing dpis in my home directory, but it doesn't work.. Can you help me, please? Thanks! :) -- [ Andrea Spadaccini a.k.a. Lupino / Lupin85 - from Catania ] [ MAIL: lupin85(at)email.it - ICQ: 91528290 - Linux 2.4.22 ] [ Linux Registered User #313388 - Powered by Slackware 9.1 ]

Nov. 30, 2004

Re: [Dillo-dev]pthreads

by Jorge Arellano Cid

On Fri, Nov 26, 2004 at 02:14:53PM -0200, Livio Baldini Soares wrote: > Hi Jorge! > > Jorge Arellano Cid writes: > > Errata: > > > > > > > > but it usually ends with: > > > > > > jcid 19262 0.0 0.4 1652 400 pts/10 S+ 08:18 0:00 ./pth_mem2 cd > > > [...] > > > jcid 19027 0.0 0.4 2676 408 pts/10 S+ 08:14 0:00 ./pth_mem2 cd > > > > Oh, the the PID and time in the above example are quoted wrong, > > it should say: > > > > jcid 19262 0.0 0.4 1652 400 pts/10 S+ 08:18 0:00 ./pth_mem2 cd > > [...] > > jcid 19262 0.0 0.4 2676 408 pts/10 S+ 08:18 0:00 ./pth_mem2 cd > > I can't really tell if that's a leak or not. In my system, there is > an increase use of memory between the first iteration and the second, > but from then on, things become _very_ stable (box with Linux 2.4 and > libc 2.3.2 from Debian/unstable): I understand. In my box the "leak" usually occurs near the fifth if at all... > livio 20097 0.0 0.0 1664 336 pts/21 S+ 10:24 0:00 ./pth_mem cd > livio 20097 0.0 0.0 32556 468 pts/21 S+ 10:24 0:00 ./pth_mem cd > livio 20097 0.0 0.0 1796 396 pts/21 S+ 10:24 0:00 ./pth_mem cd > livio 20097 0.0 0.0 32556 476 pts/21 S+ 10:24 0:00 ./pth_mem cd > livio 20097 0.0 0.0 1796 396 pts/21 S+ 10:25 0:00 ./pth_mem cd > > So there is certainly an increase in memory use (in my case a lot > smaller than yours). But look, there is no leak of memory, because > with each interation, the memory is equal to the previous phase. > > Or do you get an increase of 0.5-1 MiB for _each_ phase? No but I get a "random" one in one of them. For instance: I just run this one and there was no leak: <q> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND jcid 26194 0.0 0.3 1640 336 pts/29 S+ 17:50 0:00 ./pth_mem2 cd jcid 26194 0.0 0.5 32412 476 pts/29 S+ 17:50 0:00 ./pth_mem2 cd jcid 26194 0.0 0.4 1652 404 pts/29 S+ 17:50 0:00 ./pth_mem2 cd jcid 26194 0.0 0.5 32412 484 pts/29 S+ 17:50 0:00 ./pth_mem2 cd jcid 26194 0.0 0.4 1652 404 pts/29 S+ 17:50 0:00 ./pth_mem2 cd jcid 26194 0.0 0.5 32412 484 pts/29 S+ 17:50 0:00 ./pth_mem2 cd jcid 26194 0.0 0.4 1652 404 pts/29 S+ 17:50 0:00 ./pth_mem2 cd jcid 26194 0.0 0.5 32412 484 pts/29 S+ 17:50 0:00 ./pth_mem2 cd jcid 26194 0.0 0.4 1652 404 pts/29 S+ 17:50 0:00 ./pth_mem2 cd jcid 26194 0.0 0.5 32412 484 pts/29 S+ 17:50 0:00 ./pth_mem2 cd jcid 26194 0.0 0.4 1652 404 pts/29 S+ 17:50 0:00 ./pth_mem2 cd jcid 26194 0.0 0.5 32412 484 pts/29 S+ 17:50 0:00 ./pth_mem2 cd jcid 26194 0.0 0.4 1652 404 pts/29 S+ 17:50 0:00 ./pth_mem2 cd jcid 26194 0.0 0.5 32412 484 pts/29 S+ 17:50 0:00 ./pth_mem2 cd jcid 26194 0.0 0.4 1652 404 pts/29 S+ 17:50 0:00 ./pth_mem2 cd jcid 26194 0.0 0.5 32412 484 pts/29 S+ 17:50 0:00 ./pth_mem2 cd jcid 26194 0.0 0.4 1652 404 pts/29 S+ 17:50 0:00 ./pth_mem2 cd </q> Then I run this other one (w/leak): <q> jcid 26878 0.0 0.3 1640 336 pts/29 S+ 18:27 0:00 ./pth_mem2 d jcid 26878 0.0 0.5 32412 472 pts/29 S+ 18:27 0:00 ./pth_mem2 d jcid 26878 0.0 0.4 1652 400 pts/29 S+ 18:27 0:00 ./pth_mem2 d jcid 26878 0.0 0.5 32412 480 pts/29 S+ 18:27 0:00 ./pth_mem2 d jcid 26878 0.0 0.4 1652 400 pts/29 S+ 18:27 0:00 ./pth_mem2 d jcid 26878 0.0 0.5 32412 480 pts/29 S+ 18:27 0:00 ./pth_mem2 d jcid 26878 0.0 0.4 1652 400 pts/29 S+ 18:27 0:00 ./pth_mem2 d jcid 26878 0.0 0.5 33436 484 pts/29 S+ 18:27 0:00 ./pth_mem2 d * jcid 26878 0.0 0.4 2676 404 pts/29 S+ 18:27 0:00 ./pth_mem2 d jcid 26878 0.0 0.5 33436 484 pts/29 S+ 18:27 0:00 ./pth_mem2 d jcid 26878 0.0 0.4 2676 404 pts/29 S+ 18:27 0:00 ./pth_mem2 d jcid 26878 0.0 0.5 33436 484 pts/29 S+ 18:27 0:00 ./pth_mem2 d jcid 26878 0.0 0.4 2676 404 pts/29 S+ 18:27 0:00 ./pth_mem2 d jcid 26878 0.0 0.5 33436 484 pts/29 S+ 18:27 0:00 ./pth_mem2 d jcid 26878 0.0 0.4 2676 404 pts/29 S+ 18:27 0:00 ./pth_mem2 d jcid 26878 0.0 0.5 33436 484 pts/29 S+ 18:27 0:00 ./pth_mem2 d jcid 26878 0.0 0.4 2676 404 pts/29 S+ 18:27 0:00 ./pth_mem2 d </q> I can easily get results with an without leaks for the three ways though. As from my tests, it seems like the "leak" can appear on any iteration > > To make my point clearer, look at how my home box handles the fixed > test program (I'm using Linux kernel 2.6 with NTPL, and libc 2.3.2 > from Debian/unstable): > > livio 4300 0.0 0.0 1440 316 pts/7 S+ 10:40 0:00 ./pth_mem d > livio 4300 0.0 0.0 93812 432 pts/7 Sl+ 10:40 0:00 ./pth_mem d > livio 4300 0.0 0.0 34356 372 pts/7 S+ 10:40 0:00 ./pth_mem d > livio 4300 0.0 0.0 93812 436 pts/7 Sl+ 10:40 0:00 ./pth_mem d > > etc. Even though I use humengeous use of memory, I'm pretty sure > it's not leaking (since the numbers are the same after the first > iteration). A thing that's not happening on my box. > > Looking at straces, I see that libc is doing mmap2() on anonymous > memory for allocation, and in the first phase, actually allocates 8 > MiB worth of stack for each thread! (I know it's stack because it's > setting PROT_EXEC on, whoch it doesn't for regular > mallocs()). Subsequently, each thread mmap2() _2_ MiB of anonymous > memory (instead of only one). > > Anyways, not wanting to get too much into the details, it's hard to > predict what your libc is doing for you. I can see that libc correctly > munmaps() all the mapped segments (effectively freeing the > memory). _And_ the VSZ stays the same throughout interations. > > REMINDER: In this case a big VSZ does not mean big memory > usage. It means a _potential_ to use a lot of memory (I've seen rare > cases where a thread actually makes use of more than 1MiB of stack, > for example, let alone 8MiB). If the mmap2() is is never touched, a > page never gets assigned to the process' page table. BTW, Is the VSZ reserved somehow, on swap at least? or is just a potential size that we can happily ignore? I'm adding six more test results, this time with Dillo and a small page with a hundred tiny images. It has three samples for each technique: Note: I'm using the file dpi server, so dillo uses a pthread for each IO. Using 'd' technique: <q> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 13820 11168 ./dillo 15492 12624 ./dillo 15600 12736 ./dillo </q> Using 'cd' technique: <q> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 15496 12640 ./dillo 9272 6888 ./dillo 10408 7940 ./dillo </q> Here the RSS is smaller for 'cd' so I opted this one for the interim patch just commited to CVS. -- Cheers Jorge.-

Nov. 28, 2004

Re: [Dillo-dev]pthreads

by Livio Baldini Soares

Hi Jorge! Jorge Arellano Cid writes: > Errata: > > > > > but it usually ends with: > > > > jcid 19262 0.0 0.4 1652 400 pts/10 S+ 08:18 0:00 ./pth_mem2 cd > > [...] > > jcid 19027 0.0 0.4 2676 408 pts/10 S+ 08:14 0:00 ./pth_mem2 cd > > Oh, the the PID and time in the above example are quoted wrong, > it should say: > > jcid 19262 0.0 0.4 1652 400 pts/10 S+ 08:18 0:00 ./pth_mem2 cd > [...] > jcid 19262 0.0 0.4 2676 408 pts/10 S+ 08:18 0:00 ./pth_mem2 cd I can't really tell if that's a leak or not. In my system, there is an increase use of memory between the first iteration and the second, but from then on, things become _very_ stable (box with Linux 2.4 and libc 2.3.2 from Debian/unstable): livio 20097 0.0 0.0 1664 336 pts/21 S+ 10:24 0:00 ./pth_mem cd livio 20097 0.0 0.0 32556 468 pts/21 S+ 10:24 0:00 ./pth_mem cd livio 20097 0.0 0.0 1796 396 pts/21 S+ 10:24 0:00 ./pth_mem cd livio 20097 0.0 0.0 32556 476 pts/21 S+ 10:24 0:00 ./pth_mem cd livio 20097 0.0 0.0 1796 396 pts/21 S+ 10:25 0:00 ./pth_mem cd So there is certainly an increase in memory use (in my case a lot smaller than yours). But look, there is no leak of memory, because with each interation, the memory is equal to the previous phase. Or do you get an increase of 0.5-1 MiB for _each_ phase? To make my point clearer, look at how my home box handles the fixed test program (I'm using Linux kernel 2.6 with NTPL, and libc 2.3.2 from Debian/unstable): livio 4300 0.0 0.0 1440 316 pts/7 S+ 10:40 0:00 ./pth_mem d livio 4300 0.0 0.0 93812 432 pts/7 Sl+ 10:40 0:00 ./pth_mem d livio 4300 0.0 0.0 34356 372 pts/7 S+ 10:40 0:00 ./pth_mem d livio 4300 0.0 0.0 93812 436 pts/7 Sl+ 10:40 0:00 ./pth_mem d etc. Even though I use humengeous use of memory, I'm pretty sure it's not leaking (since the numbers are the same after the first iteration). Looking at straces, I see that libc is doing mmap2() on anonymous memory for allocation, and in the first phase, actually allocates 8 MiB worth of stack for each thread! (I know it's stack because it's setting PROT_EXEC on, whoch it doesn't for regular mallocs()). Subsequently, each thread mmap2() _2_ MiB of anonymous memory (instead of only one). Anyways, not wanting to get too much into the details, it's hard to predict what your libc is doing for you. I can see that libc correctly munmaps() all the mapped segments (effectively freeing the memory). _And_ the VSZ stays the same throughout interations. REMINDER: In this case a big VSZ does not mean big memory usage. It means a _potential_ to use a lot of memory (I've seen rare cases where a thread actually makes use of more than 1MiB of stack, for example, let alone 8MiB). If the mmap2() is is never touched, a page never gets assigned to the process' page table. hope this helps, -- Livio B. Soares

Nov. 26, 2004

Re: [Dillo-dev]pthreads

by Jorge Arellano Cid

Errata: > > but it usually ends with: > > jcid 19262 0.0 0.4 1652 400 pts/10 S+ 08:18 0:00 ./pth_mem2 cd > [...] > jcid 19027 0.0 0.4 2676 408 pts/10 S+ 08:14 0:00 ./pth_mem2 cd Oh, the the PID and time in the above example are quoted wrong, it should say: jcid 19262 0.0 0.4 1652 400 pts/10 S+ 08:18 0:00 ./pth_mem2 cd [...] jcid 19262 0.0 0.4 2676 408 pts/10 S+ 08:18 0:00 ./pth_mem2 cd -- Cheers Jorge.-

Nov. 26, 2004

Re: [Dillo-dev]pthreads

by Jorge Arellano Cid

On Thu, Nov 25, 2004 at 04:48:07PM +0100, Lars Segerlund wrote: > > I REALLY understand chasing the leaks and then finding > something like this stings ... Yes it does. Note that even knowing that it was my fault (assuming wrongly about the API), the irony remains. ...and the soothing side of it, is that now we have a Dillo running on less than 1/5 of the memory it used to. -- Cheers Jorge.-

Nov. 26, 2004

Re: [Dillo-dev]pthreads

by Jorge Arellano Cid

On Thu, Nov 25, 2004 at 02:39:01PM -0200, Livio Baldini Soares wrote: > Hi Jorge, > > I just got the chance to look at your little test program. It has a > few bugs: > > 1) In pthread_create() you are passing a pointer to the unintialized > thrdID[i] variable. Pthreads does not guarantee that the threadID > argument is initialized before the start_routine is called. If you > try printing the value of p_thrID in the 'thr_function', you'll see > you'll get some correct and some random values. Gotcha! This explains most of it. So the quid is that Pthreads does not guarantee that the threadID argument is initialized before the start_routine is called. My fault! Even though some other pthread implementations seem to do it, the document from the opengroup you cite is clear: <q> APPLICATION USAGE There is no requirement on the implementation that the ID of the created thread be available before the newly created thread starts executing. The calling thread can obtain the ID of the created thread through the return value of the pthread_create() function, and the newly created thread can obtain its ID by a call to pthread_self(). </q> Now, even after patching the test program to use: > - pthread_detach(*p_thrID); > + pthread_detach(pthread_self()); it still leaks some memory in my machine! (near half a MB). It is interesting to note that as Brian reports: > [Brian wrote:] > > [Jorge wrote:] > > > > Has anyone had the opportunity to run the test program under > > Solaris? This is an interesting test because Solaris has its own > > threads implementation. I'd expect the detached pthreads test not > > to leak memory on Solaris. > > did it just now. all three ways leak zero memory. Zero leak on Solaris. On my machine, the three ways can still leak, but sometimes they don't. For instance, running the test should give: jcid 19262 0.0 0.4 1652 400 pts/10 S+ 08:18 0:00 ./pth_mem2 cd [...] jcid 19262 0.0 0.4 1652 400 pts/10 S+ 08:18 0:00 ./pth_mem2 cd but it usually ends with: jcid 19262 0.0 0.4 1652 400 pts/10 S+ 08:18 0:00 ./pth_mem2 cd [...] jcid 19027 0.0 0.4 2676 408 pts/10 S+ 08:14 0:00 ./pth_mem2 cd (the same amount leaked either with "d", "cd" or "j" mode). At some point in time "cd" seemed to be the "better" one, but surely more testing is required. Or better, finding the bug. ;) -- Cheers Jorge.-

Nov. 26, 2004

Re: [Dillo-dev]pthreads

by Livio Baldini Soares

Hi Brian, Brian Hechinger writes: > i just did a test on my mac, which runs NetBSD 2.0 RC4, and it also does > not leak memory in either of the three cases. > > i know they use the pth library, but i wonder if they do something > differently than linux does? Did you get a chance to read my bugreport messages? http://lists.auriga.wearlab.de/pipermail/dillo-dev/2004-November/002448.html and http://lists.auriga.wearlab.de/pipermail/dillo-dev/2004-November/002450.html It is possible that different pthread implementations initialize (or set) the threadID (first argument passed to pthread_create()) in different orders. In Linux, I have seen different systems behave randomly (probably because it issues a clone(), and from then on let's the OS take over the scheduling, which is usually non-deterministic). Even if you tried sucessfully on Solaris and *BSD, it does not mean that it is a safe construct to rely on. The OS might happen to schedule the threads in a certain order, or the specific version of the libraries synchronizes the thread creation in a specific manner, etc. Bottom line, the POSIX threads specificiation does *not* require the threadID to be set before the new thread begin execution:

Nov. 26, 2004

Re: [Dillo-dev]pthreads

by Brian Hechinger

i just did a test on my mac, which runs NetBSD 2.0 RC4, and it also does not leak memory in either of the three cases. i know they use the pth library, but i wonder if they do something differently than linux does? -brian -- "Now you know why I got the everliving hell OUT of Windows administration. Knowing it doesn't make it any easier. It's just broken-as-designed."

Nov. 26, 2004

Re: [Dillo-dev]Ugly fonts

by higuita

hi On Thu, 25 Nov 2004 04:41:19 -0200, Livio Baldini Soares <livio(a)ime.usp.br> wrote: > > try to do the xfontsel in that computer and compare then > You probably mean 'xlsfonts', which will list all the fonts the > current X server has access to (in a nicer format for comparison, than > xfontsel). agree, xfontsel is ugly but better to choose a font, but for comparing xlsfont is *alot* better... but i also suspect that the difference would be huge (maybe 10 fonts against 100) so enough to search for the cause thanks anyway for the correction 8) higuita -- Naturally the common people don't want war... but after all it is the leaders of a country who determine the policy, and it is always a simple matter to drag the people along, whether it is a democracy, or a fascist dictatorship, or a parliament, or a communist dictatorship. Voice or no voice, the people can always be brought to the bidding of the leaders. That is easy. All you have to do is tell them they are being attacked, and denounce the pacifists for lack of patriotism and exposing the country to danger. It works the same in every country. -- Hermann Goering, Nazi and war criminal, 1883-1946

Nov. 26, 2004

Re: [Dillo-dev]pthreads

by Brian Hechinger

On Wed, Nov 24, 2004 at 10:46:30AM -0300, Jorge Arellano Cid wrote: > > Has anyone had the opportunity to run the test program under > Solaris? This is an interesting test because Solaris has its own > threads implementation. I'd expect the detached pthreads test not > to leak memory on Solaris. did it just now. all three ways leak zero memory. -brian -- "Now you know why I got the everliving hell OUT of Windows administration. Knowing it doesn't make it any easier. It's just broken-as-designed."

Nov. 26, 2004