- Dillo-dev - lists.mailman3.com

Re: (working) prototype of charset decoding in cache.c
by jcid＠dillo.org May 27, 2008

May 27, 2008

Hi, On Fri, May 02, 2008 at 08:02:14PM +0000, corvid wrote: > I've been playing around with this now and then over the past few months > when I haven't had anything else going on. I've been running on it for > a day or so without incident, but I haven't gone through it with a > fine-toothed comb because I know there will be changes. > > It's 35K, so I realize I'm in little danger of hearing back too quickly :) > Besides, I know you said you have some nbsp thing that needs attention... I've made a second review over the code, and it doesn't look that bad to me. This is, the decoding in cache, even the stuffed cache-entry; if it can be isolated/tidied in the future better. The part that looks inconvenient to me is the charset choosing dialog. I'd prefer Dillo do this automatically. There's no need to try to recognize every charset in the world, but just a few "most used" ones like koi8-r. BTW, UTF-8 has become the preferred solution. The idea is that we can leave space for a expandable function just in case. BTW, there's an improved version of the data guesser in the ftp dpi. Please send me a koi8-r encoded page to test (or just let me know how to find it with google :). IIRC you sent me an explanation message on what this patch was for and remaining issues. I can't find it :(. Please resend it to me. > > There are plenty of issues. Going in order through the diff: > > - *_get_content_type(). I searched for some Russian text files where a > charset is given in the HTTP header. None of them were regarded as > text by the old test for ascii chars. > Maybe the way to go is to have tests for more filetypes and treat > unknown as text. Checking for all possible charsets would be no fun, > anyway. (Tests for more filetypes might be needed for file uploading, > as well.) The problem with "unknown as text" is that some HTTP servers send downloadable files as text. It is quite practical in this case to get a download dialog from dillo, intead of a binary rendering (especially for newbies). That's why the current data guesser is conservative. The data guesser can be improved. e.g. taking ASCCI control characters as a strong indication of a binary, and their absence as a text hint. I can give it a look after you send me the files. > - CacheEntry_t now has a horrifying amount of junk in it. Maybe it > would be good to pull out about half of it into some sort of Data thing. It's stuffed but still understandable. If it can be tidied in a second pass, better. > o TypeSet to be set from Meta or a user dialog. Meta can set it once; > the dialog can set it repeatedly. I'd like to avoid that dialog if possible. > o UTF8Data (LocalData?) that's been translated. cache.c has a lot of > "entry->UTF8Data ? entry->UTF8Data : entry->Data" in it now that > perhaps should be hidden behind something. Yes. > o DataReaders reference count. UTF8Data is destroyed when not needed. > o CharsetDecoder > > - a_Cache_unref_buf, a_Capi_unref_buf, a_Nav_unref_buf all just to > provide notification that the view source window isn't reading it anymore. > Ugh. Not pretty, but tolerable. I'm thinking along the line of getting the core of the patch committed and then polish the rough edges. > - a_Cache_[gs]et_content_type and capi and nav. Ugh again. > I don't know whether there's a timing issue with changing decoders > due to a meta tag at the same time that a view source window is being > opened. Yes there is. Simple to solve by allowing view source after parsing the HEAD element only. > (Why content-type and not charset? I find switching to text/plain to > use findtext on the source to be useful.) Sorry, I don't understand the question. > - Decode_charset() no longer free()s its input. This inconsistency would > be an excellent source of bugs, so it would probably be good to make it so > that none of the decoders free input. Yes, this is a must. Consistency should be our goal/horizon/friend/helper. :-) -- Cheers Jorge.-

2 6

[patch]: html.cc: DilloHtmlForm::getCurrentInput()
by onepoint＠starurchin.org May 27, 2008

May 27, 2008

This refactors DilloHtmlForm code into a new "inline DilloHtmlInput *getCurrentInput()" method. It's just like the earlier patch that created DilloHtml::getCurrentForm() . Comments are welcome, although I hope it is uncontroversial. Regards, Jeremy Henty

2 4

an experiment in widget coloring
by corvid＠lavabit.com May 27, 2008

May 27, 2008

I saw a page that reminded me that the matter of when to have light text vs. dark text on widgets was not decided. earlier discussion: http://lists.auriga.wearlab.de/pipermail/dillo-dev/2008-February/003760.html trimmed-down bit of the page I saw: <form> <table bgcolor="#F8C473"> <tr><td> <input type="submit" value="I can hardly see the text"> </table> </form>

1 0

[patch]: html.cc cleanup: DilloHtml::currentForm()
by onepoint＠starurchin.org May 26, 2008

May 26, 2008

This patch adds a DilloHtml::currentForm() method and uses it to refactor wherever possible. This reduces the number of instances of the "foo->size() - 1" idiom in html.cc from 23 to 13! Regards, Jeremy Henty

3 8

[patch] dw2 memory reduction
by Johannes.Hofmann＠gmx.de May 26, 2008

May 26, 2008

Hi, attached is patch to reduce the memory usage of dw2: * Styles are now reused if they are equal. This brings down the number of Style objects on the mysql page from around 180000 to 23000 and reduces memory usage by about 25M at the cost of an additional HashTable. Maybe the hash table should be part of fltkplatform as the hash tables for Color and Font? There would be even more sharing of styles possible, if x_link would not be part of style. On the other hand about 3M for styles on the huge mysql page is pretty ok. * Size of struct Word is reduced by 4 integers by using shorts. I know that this is questionable, but I think in the case of struct Word that can be allocated a lot it is worth the hassle. Please check whether my assumption that space lengths and word lengths can be stored in shorts are valid in general. * SimpleVector no longer preallocates memory if it's not necessary. This actually saves some megabytes here! With these changes memory usage on the mysql page is 179M here. Cheers, Johannes

3 8

Fuzzing dillo
by 4winter＠informatik.uni-hamburg.de May 26, 2008

May 26, 2008

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi :) I was playing around with Michal Zalewskis html fuzzer 'mangle' trying to crash dillo and succeeded :). Furthermore I was able to reduce the ~100k page that was triggering a NULL pointer dereferencing to three tags (see the attached page). Some digging around (html.cc is a monster ^^) revealed that style_attrs.setBorderColor ( Color::createShaded(HT2LT(html), style_attrs.backgroundColor->getColor())); in Html_tag_open_table was failing due to the fact that style_attrs.backgroundColor was NULL. Html_tag_open_select sets the backgroundColor to NULL and this is inherited by the table tag since it's a child of the select tag. Greping through the code shows that there are at least two other functions that set backgroundColor (and color) to NULL (Html_tag_open_input and Html_tag_open_isindex) and twelve locations calling getColor() on either color or backgroundColor. I have no idea how to _properly_ fix this issue. Furthermore I'd suggest to break down html.cc into several files. It is about 6000 lines long (-> hard to dig into) and it takes gcc roughly five seconds to compile on my box. Justus - -- gpg key fingerprint: C82D 382A AB38 1A54 5290 19D6 A0F9 B035 686C 6996 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIOUmmoPmwNWhsaZYRAprIAJ9X9HSzc4Isx+NyFWxlfSFPrwj/VACdF8GD x67scPvD6FCrOezl69NbaHA= =z0iv -----END PGP SIGNATURE-----

4 10

patch: http_proxyuser preference
by corvid＠lavabit.com May 26, 2008

May 26, 2008

After I sent the preferences mail yesterday, I found that I overlooked one. The headers look good, but I have no way to test it for real.

2 1

patch: goodbye to use_oblique and transient_dialogs
by corvid＠lavabit.com May 26, 2008

May 26, 2008

It's nice to get to delete things, and especially preferences, since they have a natural tendency to grow and grow and grow.

2 1

remaining preferences
by corvid＠lavabit.com May 26, 2008

May 26, 2008

# Dicache is where the Decompressed Images are cached (not the original ones). # If you have a lot of memory and a slow CPU, use YES, otherwise use NO use_dicache The code always behaves like use_dicache=NO /* * Unrefs the counter of a dicache entry, and _if_ no DwImage is acessing * this buffer, then we call Dicache_free to do the dirty job. */ void a_Dicache_unref(const DilloUrl *Url, int version) { DICacheEntry *entry; if ((entry = Dicache_get_entry_version(Url, version))) { /*if (--entry->RefCount == 0 && (entry->next || !prefs.use_dicache)) {*/ if (--entry->RefCount == 0) { Dicache_remove(Url, version); } } } Perhaps this was done for a reason. # If you prefer oblique over italic fonts, uncoment next line use_oblique FLTK doesn't know about oblique. I don't know whether anybody cares enough to change that. I don't. # Show tooltip popup for images? # Note: We use the "title" attribute and not "alt". # More info at: http://bugzilla.mozilla.org/show_bug.cgi?id=25537 show_tooltip I did the easy part of tooltips long ago, but I haven't been able to care enough to want to do the fltk part. # Set this to YES, if you want to limit the word wrap width to the vieport # width (may be useful for iPAQ) limit_text_width It doesn't do anything. I don't think it's ever done anything. It sure would be nice if it did something. But I don't have the expertise to change that. # Here you can choose to hide some widgets of the dillo panel... show_menubar Not currently used. Meaningless with the current panel-cycling with right click on the search button (which I imagine is not going to be the permanent way to do that). # Enabling this will restrain OpenUrl and FindText, but may be required # for the ION window manager. transient_dialogs from fltk::Window::child_of(): Tell the system that this window will not have an icon, it will dissappear and reappear when \a parent is iconized or shown, and it is forced to always be above \a parent. On X this is called a "Transient window", and Windows calls this a "overlapping child". \a parent is different than the parent(), which must be zero. (and down in fltk::CreatedWindow::create, there's a if (window->child_of() && window->child_of()->shown()) XSetTransientForHint(xdisplay, x->xid, window->child_of()->i->xid);) If Justus gets rid of the dialogs, obviously the pref can be removed. Enabling this means making our own dialogs for message(), choice(), input(). # When filling forms, our default behaviour is to submit on enterpress, # but only when there's a single text entry (to avoid incomplete submits). # OTOH, if you have to fill the same form lots of times, you may find # useful to keep away from the mouse by forcing enter to submit. enterpress_forces_submit The code currently always submits on enter. I like it that way and didn't realize that it wasn't supposed to.

3 7

[BUG]: Redirect to previous page + Reload crashes
by onepoint＠starurchin.org May 25, 2008

May 25, 2008

Dillo crashes as follows: * Point Dillo at http://starurchin.org:8080/dillo/here * Click the "There" link. Dillo is redirected back to the "Here" page. * Click the "Reload" button. * Dillo crashes inside a_Nav_reload() (nav.c) at the line "if (URL_FLAGS(url) & URL_Post) {" because url is NULL. I noticed this a few weeks ago but only just found a simple way to trigger it. Any ideas why this is happening? Regards, Jeremy Henty

3 6