Looking for help with YYLMAX

[Rush Manbert <ru...@manbert.com>]

We have some thrift source files that have rather large comments in  
them. When we try to compile them, we get this error:
token too large, exceeds YYLMAX

If I remove the comments, the compiler runs successfully.

I found the GNU Flex documentation, and I read about changing the  
definition of YYLMAX, and I googled for examples. Eventually, I tried  
adding this to the definitions section of compiler/cpp/src/thriftl.ll  
(at line 8 to be exact):
%{
#define YYLMAX 1048576
%}

When I run the make, then examine compiler/cpp/thriftl.cc, I find the  
#define, but it gets inserted after the all-important lines:
#ifndef YYLMAX
#define YYLMAX 8192
#endif

char yytext[YYLMAX];
char *yytext_ptr;

which begin at line 1150. If I use the compiler built from this, I (of  
course) get a segmentation fault at run time.

I have never looked at Flex before in my life, so I'm hoping someone  
who has experience with it can tell me what I need to do to get my  
definition of YYLMAX to appear before the "#ifndef YYLMAX" line shown  
above.

Thanks,
Rush

Re: Looking for help with YYLMAX

[Rush Manbert <ru...@manbert.com>]

Thanks David. That worked.

- Rush

On May 14, 2008, at 12:42 PM, David Reiss wrote:

> This is just a shot in the dark, but try running ./configure as
> ./configure CPPFLAGS='-DYYLMAX=whatever'
> This will cause it to be set on the g++ command line, which will
> take effect before any #defines in the file.
>
> --David
>
> Rush Manbert wrote:
>> We have some thrift source files that have rather large comments in
>> them. When we try to compile them, we get this error:
>> token too large, exceeds YYLMAX
>>
>> If I remove the comments, the compiler runs successfully.
>>
>> I found the GNU Flex documentation, and I read about changing the
>> definition of YYLMAX, and I googled for examples. Eventually, I tried
>> adding this to the definitions section of compiler/cpp/src/thriftl.ll
>> (at line 8 to be exact):
>> %{
>> #define YYLMAX 1048576
>> %}
>>
>> When I run the make, then examine compiler/cpp/thriftl.cc, I find the
>> #define, but it gets inserted after the all-important lines:
>> #ifndef YYLMAX
>> #define YYLMAX 8192
>> #endif
>>
>> char yytext[YYLMAX];
>> char *yytext_ptr;
>>
>> which begin at line 1150. If I use the compiler built from this, I  
>> (of
>> course) get a segmentation fault at run time.
>>
>> I have never looked at Flex before in my life, so I'm hoping someone
>> who has experience with it can tell me what I need to do to get my
>> definition of YYLMAX to appear before the "#ifndef YYLMAX" line shown
>> above.
>>
>> Thanks,
>> Rush
>>

Re: Looking for help with YYLMAX

[David Reiss <dr...@facebook.com>]

This is just a shot in the dark, but try running ./configure as
./configure CPPFLAGS='-DYYLMAX=whatever'
This will cause it to be set on the g++ command line, which will
take effect before any #defines in the file.

--David

Rush Manbert wrote:
> We have some thrift source files that have rather large comments in 
> them. When we try to compile them, we get this error:
> token too large, exceeds YYLMAX
> 
> If I remove the comments, the compiler runs successfully.
> 
> I found the GNU Flex documentation, and I read about changing the 
> definition of YYLMAX, and I googled for examples. Eventually, I tried 
> adding this to the definitions section of compiler/cpp/src/thriftl.ll 
> (at line 8 to be exact):
> %{
> #define YYLMAX 1048576
> %}
> 
> When I run the make, then examine compiler/cpp/thriftl.cc, I find the 
> #define, but it gets inserted after the all-important lines:
> #ifndef YYLMAX
> #define YYLMAX 8192
> #endif
> 
> char yytext[YYLMAX];
> char *yytext_ptr;
> 
> which begin at line 1150. If I use the compiler built from this, I (of 
> course) get a segmentation fault at run time.
> 
> I have never looked at Flex before in my life, so I'm hoping someone 
> who has experience with it can tell me what I need to do to get my 
> definition of YYLMAX to appear before the "#ifndef YYLMAX" line shown 
> above.
> 
> Thanks,
> Rush
>