packed trie for hyphenation patterns in the main repository
Hi, I just pushed the packed trie implementation from http://flpsed.org/hgweb/dillo_hyphen_mem to the main repository. It considerably reduces the memory consumption of the hyphenator. The <lang>.pat files of the original implementation without any changes. However one can reduce memory consumption even further by precompressing the patterns using the new test/trie program: Usage: trie <pattern file> > <prefix>/lib/dillo/hyphenation/<lang>.pat.trie e.g: ./trie hyph-de-1996.pat > /usr/local/lib/dillo/hyphenation/de.pat.trie If the <lang>.pat.trie is found, dillo will use this precompressed trie instead of the <lang>.pat file. These changes should not have any effect on where the hyphenator breaks up words. If you find any regressions or changes, please report them. Cheers, Johannes
On Thu, Sep 20, 2012 at 10:21:14PM +0200, Johannes Hofmann wrote:
Hi,
I just pushed the packed trie implementation from http://flpsed.org/hgweb/dillo_hyphen_mem to the main repository. It considerably reduces the memory consumption of the hyphenator. The <lang>.pat files of the original implementation without any changes. However one can reduce memory consumption even further by precompressing the patterns using the new test/trie program:
Usage: trie <pattern file> > <prefix>/lib/dillo/hyphenation/<lang>.pat.trie
e.g: ./trie hyph-de-1996.pat > /usr/local/lib/dillo/hyphenation/de.pat.trie
If the <lang>.pat.trie is found, dillo will use this precompressed trie instead of the <lang>.pat file.
These changes should not have any effect on where the hyphenator breaks up words. If you find any regressions or changes, please report them.
OK, now I'm using the *trie files. BTW, they're bigger than the original *pat files; I assume this is OK. -- Cheers Jorge.-
participants (2)
-
jcid@dillo.org
-
Johannes.Hofmann@gmx.de