[erlang-questions] how: prefix matching on external term format
Wed Jun 4 19:10:26 CEST 2008
Oh hey since there's interest, I got this working by forking
erl_compare_ext and adding special tags for smallest and largest erlang
term. My original fear of forking was unfounded since 1) the canonical
erl_compare_ext has several bugs and 2) the canonical erl_compare_ext does
not support ERL_NEW_FLOAT_EXT. That last one was pretty funny since my
main fear of forking was missing out on new type tags. Well, the
directory *does* say "legacy" ...
I'm in the end game now, within a week I should have something on google
code that sports the same interface as dets but supports ordered_set and
ordered_bag (and 8EB [9.22e18 bytes] byte file size limit).
On Wed, 4 Jun 2008, Scott Lystig Fritchie wrote:
> Paul Mineiro <> wrote:
> pm> so i've envisioned the following choices:
> Hrm, I wonder if it's a viable option to avoid the official Erlang
> external format? If I recall correctly(*), any tuple stored in ETS is
> not serialized using the external term format. Instead, the tuple is
> copied into a separate heap. That heap is big enough to store only the
> tuple. But the data inside that heap uses the same tagged pointer
> scheme as any Erlang term inside a process's heap, so all the term
> comparison macros & functions work, regardless of type (process heap or
> ETS mini-heap).
> (*) If I'm wrong, someone please yell loudly at me and the list. I
> wouldn't want the falsehood to live unrefuted forever.
> If my memory is correct, then storing an ETS mini-heap onto disk will
> suffer from pointer offset problems: the VM's tagged pointers for use in
> RAM won't point to the necessary byte offset within a file on disk. The
> simplest (?) kludge would be to write an ETS mini-heap with all pointers
> rewritten to be relative to the start of the mini-heap, and then re-set
> those pointers to RAM-valid values after reading from disk but before
> any comparisons. (Simplest = avoid reimplementing term comparison and
> match spec functions.)
In an artificial world, only extremists live naturally.
-- Paul Graham
More information about the erlang-questions