|flex scanner too huge, suggestions? firstname.lastname@example.org (Troy Cauble) (2001-02-17)|
|Re: flex scanner too huge, suggestions? email@example.com (Tim Josling) (2001-02-23)|
|Re: flex scanner too huge, suggestions? firstname.lastname@example.org (Scott Nicol) (2001-02-23)|
|Re: flex scanner too huge, suggestions? email@example.com (2001-02-23)|
|Re: flex scanner too huge, suggestions? firstname.lastname@example.org (2001-02-25)|
|Re: flex scanner too huge, suggestions? Ron@Profit-Master.com (Ron Pinkas) (2001-02-25)|
|Re: flex scanner too huge, suggestions? email@example.com (Troy Cauble) (2001-03-01)|
|From:||Troy Cauble <firstname.lastname@example.org>|
|Date:||1 Mar 2001 02:33:52 -0500|
|Organization:||Bell Labs, Holmdel, NJ, USA|
|Posted-Date:||01 Mar 2001 02:33:52 EST|
Troy Cauble <email@example.com> wrote:
: I have a working flex/bison implementation to parse a text
: protocol. The problem is that the scanner is huge.
: Made with LFLAGS = -Cfe, size gives
: 42068(.text) + 24(.data) + 60(.bss) + 1687886(.rodata) = 1730038
: Made with LFLAGS = -Cem, size gives
: 42496(.text) + 24(.data) + 60(.bss) + 508326(.rodata) = 550906
: This is still pretty large for some of our target environments.
: I haven't tried to disable features to determine what makes
: it so huge, but the protocol is case-insensitive and context
: (position) sensitive. I addressed the context sensitivity
: with flex start states.
Thanks to all who replied here and via e-mail.
I thought I'd post what worked for future archive readers.
The big win for my flex scanner was rewriting INCLUSIVE start
states as EXCLUSIVE. I had about 50 start states with
1 to 5 rules each, to handle portions of the protocol.
Moving to EXTERNAL start states meant I had to add additional
rules for each start state (but a small subset of the INITIAL
rules). This resulted in more rules in the source file, but
much smaller tables. .rodata is less than half of what it was
Given the table reduction, I'm *guessing* that 50 inclusive
start states generates 50 times the INITIAL (non start state)
rules. (Standard rules plus those in start state 1; standard
rules plus those in start state 2; ...)
Oddly enough, the oft-suggested "single identifier rule with
keyword lookup" actually increased my table size slightly
before I did rewrote the start states. I can't guess why.
(It wasn't the lookup function, I didn't even bother to
Return to the
Search the comp.compilers archives again.