- Dillo-dev - lists.mailman3.com

Re: [Dillo-dev]pthreads

by Jorge Arellano Cid

On Thu, Nov 18, 2004 at 01:34:46PM -0300, Jorge Arellano Cid wrote: > [...] > I decided to cut glib out of the problem, then developed a test > program and did some research. *Very* interesting results came > out of it. > > [...] > > > [Lars wrote:] > > If the threads are allocating of a public heap it might not be > > possible to reclaim/remap the heap so it will look as if a lot of > > memory is allocated. There is a similar 'bug' in the C++ standard > > that means that memory allocated by objects might never be > > reclaimed under some circumstances. > > Yes it look like this kind of problem. > > > I DO NOT now if this is related to the problem, but it sounds a > > bit similar (I suppose you have tried running the whole lot under > > valgrind ?) > > I cut glib out and made a test program (attached), that > matches mallocs and frees trivially. > > The results were amazing: > > compile the program with: > > gcc pth_mem.c -o pth_mem -lpthread > > run it like this: > > ./pth_mem <j|d|cd> > > and observe the memory footprint from another terminal with: > > while [ 1 ]; do ps aux | grep [p]th_mem; sleep 4; done Has anyone had the opportunity to run the test program under Solaris? This is an interesting test because Solaris has its own threads implementation. I'd expect the detached pthreads test not to leak memory on Solaris. BTW, the new threads-based dpi server is in the CVS now. This fixes a critical race with large files that linked lots of instances of a minor image (a bullet for instance). It worked OK with my tests (up to a 2MB page), uses joinable threads, and didn't show any leaks while testing. Strange no one reported this bug before... Well I guess almost nobody writes large HTML pages these days, as the very big browsers will surely have a hard time with them! :-) -- Cheers Jorge.-

Nov. 24, 2004

[Dillo-dev]Parser orthogonalization

by Jorge Arellano Cid

Hi there, A few days ago I finished the: [1] Orthogonalize the parser (leaks, XML, CSS) task from the list. I didn't commit it inmediately because of the pthreads issue we're still workint in (it showed as a parser bug when in fact it was elsewhere). Now the large parser patch is commited (a 70KB beast!). From the ChangeLog: <q> * Orthogonalized the generic parser: - Fixes memory leaks and widget state when recovering from bad HTML. - Improves error detection and validation (needed by XHTML). - Makes DOC tree generation possible (needed by CSS). - Cleaner design of handling routines for bad HTML. - Orthodox treatment of double optional elements (HTML, HEAD, BODY). - Lots of minor code cleanups. </q> That's the summary. Now tags are always pushed into the stack when they're found and the respective closing functions are always called when finishing them, no matter if this is triggered by an explicit tag, optional close, or bad-HTML cleanup. This is very important because of the above explained reasons. The routines for handling HTML cleanup (optional, magic and bad HTML) were mostly isolated into three functions: Html_test_section() Html_stack_cleanup_at_open() Html_tag_cleanup_at_close() (fairly commented in the source). In brief: the good news of all of this patch is that it makes much easier to understand, improve and maintain the parser (eventually to chop, merge or reuse too), fixes memory handling, widget creation state, validation, and is also good for future CSS, XHTML. For instance, look at the new <P> and </P> tag treatment at: http://www-106.ibm.com/developerworks/eserver/articles/framework.html Now, I'm back to pthreads, ah, and I have more cleanups already done waiting for commit, some patches pending, the usual stuff! -- Cheers Jorge.-

Nov. 21, 2004

[Dillo-dev]Dillo and multimedia keys

by Andrea Spadaccini

Hi everyone, does Dillo support "strange" keys that are configured with Xmodmap? I'd like to hit the "back" button (XF86Back) and go back one page, and so on... If Dillo doesn't support this, you could give me suggestions on how to implement it. I know a little C, and maybe I could be able to help you. Thanks in advance -- [ Andrea Spadaccini a.k.a. Lupino / Lupin85 - from Catania ] [ MAIL: lupin85(at)email.it - ICQ: 91528290 - Linux 2.4.22 ] [ Linux Registered User #313388 - Powered by Slackware 9.1 ] -- Email.it, the professional e-mail, gratis per te: http://www.email.it/f Sponsor: Email.it Phone Card: chiami in tutto il mondo a tariffe imbattibili da tutti i telefoni fissi e cellulari! Clicca e scopri come Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=2686&d=20-11

Nov. 20, 2004

Re: [Dillo-dev]pthreads

by Lars Segerlund

Thanks, I'm so glad I actually was in the ballpark :-) ... Usually when I have a good idéa of what's happening, it's wrong :-) .. I'll also look at it, since I debugged some stuff like this a long time ago :-) ... ( C++ project big -> bugs -> I rewrote in C and put up some wrappers ... -> worked ( well, it looked good anyhow )). I'm really intrigued now ! / Lars ... On Thu, 18 Nov 2004 13:34:46 -0300 Jorge Arellano Cid <jcid(a)dillo.org> wrote: > Livio, Lars, > ...snip... > > > [Lars wrote:] > > If the threads are allocating of a public heap it might not be > > possible to reclaim/remap the heap so it will look as if a lot of > > memory is allocated. There is a similar 'bug' in the C++ standard > > that means that memory allocated by objects might never be > > reclaimed under some circumstances. > > Yes it look like this kind of problem. > ...snip...

Nov. 18, 2004

Re: [Dillo-dev]pthreads

by Jorge Arellano Cid

Livio, Lars, On Tue, Nov 16, 2004 at 06:05:17PM -0200, Livio Baldini Soares wrote: > Hi Jorge, Hi Livio! Good to know you're still there. > I don't have time to really look at this, but are you using > g_thread_init(), gdk_threads_init(), gdk_threads_enter(), > gdk_threads_leave() and company? Here is some info: > > http://www.gtk.org/faq/#AEN482 > > I wouldn't be surprised if GLib gets confused without calling > g_thread_init(). Specifically, if I remember correctly, the locking > of GLibc's internal variables is only actually done when > g_thread_init() is executed. > > This _might_ explain your problem... but it might be a long shot, > since a race condition is needed to activate the possible problem. If > you're sure, however, that g_malloc and g_free are the source for > your memory leak, that the problem is something else (in GLib 1.2, at > least, g_malloc and g_free map almost directly to malloc() and free(), > which are thread-safe). > > The only big caveat of malloc/free is that they are not signal-safe, > i.e., you shouldn't use them (along with a hoard of other things in > libc and systems calls, etc) in signal handlers. I decided to cut glib out of the problem, then developed a test program and did some research. *Very* interesting results came out of it. > [Lars wrote:] > If the threads are allocating of a public heap it might not be > possible to reclaim/remap the heap so it will look as if a lot of > memory is allocated. There is a similar 'bug' in the C++ standard > that means that memory allocated by objects might never be > reclaimed under some circumstances. Yes it look like this kind of problem. > I DO NOT now if this is related to the problem, but it sounds a > bit similar (I suppose you have tried running the whole lot under > valgrind ?) I cut glib out and made a test program (attached), that matches mallocs and frees trivially. The results were amazing: compile the program with: gcc pth_mem.c -o pth_mem -lpthread run it like this: ./pth_mem <j|d|cd> and observe the memory footprint from another terminal with: while [ 1 ]; do ps aux | grep [p]th_mem; sleep 4; done The program makes 8 iterations, launching 10 pthreads on each one. Each launched thread allocates memory, sleeps and frees it 10 times using malloc/free, then it exits. The interesting part is that: ./pth_mem d ('d' for detached with pthread_detach()) leaks tons of memory!!! ./pth_mem cd ('cd' for create detached with pthread_attr_setdetachstate) leaks much less memory! ./pth_mem j ('j' for joinable) is almost perfect but sometimes leaks. The first two should be the same, even more, the manual says: <q> `pthread_detach' puts the thread TH in the detached state. This guarantees that the memory resources consumed by TH will be freed immediately when TH terminates. However, this prevents other threads from synchronizing on the termination of TH using `pthread_join'. </q> Unless I'm missing a key point, this looks like a bug in pthreads. Would you mind check it out and point my mistake, or giving me a pointer to pthreads library maintainer? -- Cheers Jorge.-

Nov. 18, 2004

Re: [Dillo-dev]Ugly fonts

by higuita

On Tue, 16 Nov 2004 17:52:01 -0500 (EST), "Chris de Vidal" <Chris(a)deVidal.tv> wrote: > The fonts on my machine look far worse than those in the screenshots. For > example, Slashdot: check with xfontsel what font do you really have for normal X programs, as gtk2 programs should be using a new way of generating fonts (XFT) and probably most of your truetype fonts arent really in the old X11 font interface try chossing a font listed in the xfontsel that looks good and its scalable... if you have bitstream vera, use it good luck higuita -- Naturally the common people don't want war... but after all it is the leaders of a country who determine the policy, and it is always a simple matter to drag the people along, whether it is a democracy, or a fascist dictatorship, or a parliament, or a communist dictatorship. Voice or no voice, the people can always be brought to the bidding of the leaders. That is easy. All you have to do is tell them they are being attacked, and denounce the pacifists for lack of patriotism and exposing the country to danger. It works the same in every country. -- Hermann Goering, Nazi and war criminal, 1883-1946

Nov. 18, 2004

[Dillo-dev]Ugly fonts

by Chris de Vidal

Wonderful, fast program! Especially dig the fullscreen mode and keyboard shortcuts. HTML layout rendering of pages like Google and Amazon and eBay and Slashdot looks great! Unfortunately the fonts don't... Dillo 0.8.3 Debian Sid XFCE 4.0.5-1 XFS 4.3.0.dfsg.1-8 Kernel 2.6.8.1 The fonts on my machine look far worse than those in the screenshots. For example, Slashdot: http://devidal.tv/~chris/ugly_dillo_fonts.png Compared to your screenshot: http://www.dillo.org/screenshots/slashdot1.png And just for reference, Firefox 1.0 RC1 from a Debian Sid source (probably includes gtk2 and xfs): http://devidal.tv/~chris/nice_firefox_fonts.png These fonts are about the only thing preventing my more regular use of Dillo. I built Dillo with just ./configure make make install I tried vw_fontname=lucida which helped some, but it's still uglier than the screenshots. Ideas? Need to know anything else? CD "You will exist forever. You and God are both in the universe to stay either as friends on his terms, or enemies on yours. Which it will be is proven in this life. And this life is a vapor. Two seconds, and we will be gone. Surely God means for our time on earth to count for something significant." -- John Piper in "Life as a Vapor." Read it here: http://www.desiringgod.org/library/onlinebooks_index.html#blv

Nov. 16, 2004

Re: [Dillo-dev]pthreads

by Livio Baldini Soares

Hi Jorge, Jorge Arellano Cid writes: > Hi, > > Is anyone here savvy on using dynamic memory allocation in > pthreads? > > The problem: while hunting a bug that showed after the large > parser orthogonalization patch, it was finally found between > dillo and the file dpi server; it was a critical race. I recoded > the file server to use pthreads and it went away, but... > > ... the new file server started to increase dramatically its > virtual size. > > The threads are allocating memory and freeing it with the usual > g_malloc/g_free pair, but it seems like these blocks remain > allocated even after the thread is finished. I don't have time to really look at this, but are you using g_thread_init(), gdk_threads_init(), gdk_threads_enter(), gdk_threads_leave() and company? Here is some info: http://www.gtk.org/faq/#AEN482 I wouldn't be surprised if GLib gets confused without calling g_thread_init(). Specifically, if I remember correctly, the locking of GLibc's internal variables is only actually done when g_thread_init() is executed. This _might_ explain your problem... but it might be a long shot, since a race condition is needed to activate the possible problem. If you're sure, however, that g_malloc and g_free are the source for your memory leak, that the problem is something else (in GLib 1.2, at least, g_malloc and g_free map almost directly to malloc() and free(), which are thread-safe). The only big caveat of malloc/free is that they are not signal-safe, i.e., you shouldn't use them (along with a hoard of other things in libc and systems calls, etc) in signal handlers. > This may be the same root for the reload memory leak > pointed somtime ago, do you remember? The IO engine uses a > pthread to make the request. > > It seems that there's an API to use "thread specific data" or > TSD by using pthread_key_create() and akin functions. Does anyone > know if this is the only way, or an URL of a good tutorial on how > to do this? Thread specific data is a solution, but I'd recommend avoiding it as much as possible. There should be a way to figure out what is going on with your memory leaks... Hope this helps, -- Livio B. Soares

Nov. 16, 2004

Re: [Dillo-dev]dns error with static dillo

by Jorge Arellano Cid

On Tue, Nov 16, 2004 at 05:18:50PM +0100, Roberto A. Foglietta wrote: > Hi, > > please try to reproduce this bug: I've compiled dillo with -static in > linking but 0.8.2 --> 0.8.4rc1 show the same problem: dns error! It works here. i.e. relinking by adding "-static" to the default make process. > [roberto@wsraf src]$ ./dillo > > Gdk-WARNING **: locale not supported by C library > > GLib-WARNING **: getpwuid_r(): failed due to: No such user 1309. Dillo doesn't call getpwuid_r(). You're having problems with gdk (locale) and glib (getpwuid_r). > > Gtk-WARNING **: libc.so.6: cannot handle TLS data ... and TLS. > > Gtk-WARNING **: libc.so.6: cannot handle TLS data > > Gtk-WARNING **: libc.so.6: cannot handle TLS data > > Gtk-WARNING **: libc.so.6: cannot handle TLS data > > Gtk-WARNING **: libc.so.6: cannot handle TLS data > > Gtk-WARNING **: libc.so.6: cannot handle TLS data > > Gtk-WARNING **: libc.so.6: cannot handle TLS data > > Gtk-WARNING **: libc.so.6: cannot handle TLS data > dillo_dns_init: Here we go! > Disabling cookies. > Nav_open_url: Url=>about:splash< > Nav_open_url: Url=>http://www.lugge.net< > Dns_server [0]: www.lugge.net is (nil) > Dillo: normal exit! No problem here to solve http://www.lugge.net with a "static" dillo. You can try with the "C" locale. If the problem persists, it may be a library configuration problem. -- Cheers Jorge.-

Nov. 16, 2004

[Dillo-dev]refresh tag patch for 0.8.4 (and fix)

by Roberto A. Foglietta

Hi, previous patch was little buggy, I hope this is better and it's against the last version I know. I hope you will interested in apply it. :-) Cheers, -- Roberto A. Foglietta Analista Programmatore GNU/Linux SAD Trasporto Locale S.p.a. Corso Italia 13/N 39100 BOLZANO (I) Tel. +39/0471-450.261 Fax +39/0471-450.253

Nov. 16, 2004