Thread: [BUGS] base_yylex undefined in src/interface/ecpg/preproc/parser.c

Hello,

with the newest flex (v2.6.2-19-g6bea32e, which is newer than 2.6.3) I 
need this patch to compile postgres:

diff --git a/src/interfaces/ecpg/preproc/pgc.l 
b/src/interfaces/ecpg/preproc/pgc.l
index 0453409d37..1c82b58e18 100644
--- a/src/interfaces/ecpg/preproc/pgc.l
+++ b/src/interfaces/ecpg/preproc/pgc.l
@@ -30,7 +30,7 @@ static int            xcdepth = 0;    /* depth of 
nesting in slash-star comments */
  static char       *dolqstart = NULL;   /* current $foo$ quote start 
string */
  static YY_BUFFER_STATE scanbufhandle;
  static char *scanbuf;
-
+#define yylex base_yylex
  /*
   * literalbuf is used to accumulate literal values when multiple rules
   * are needed to parse a single literal.  Call startlit to reset buffer


otherwise the linker fails with:

gcc -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -O0 -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -L../../../../src/port 
-L../../../../src/common -L/usr/local/lib -L/usr/local/lib 
-L/usr/local/lib  -Wl,--as-needed 
-Wl,-rpath,'/usr/local/lib',--enable-new-dtags  preproc.o type.o ecpg.o 
output.o parser.o keywords.o c_keywords.o ecpg_keywords.o kwlookup.o 
../ecpglib/typename.o descriptor.o variable.o -lpgcommon -lpgport -lxslt 
-lxml2 -lssl -lcrypto -lgssapi_krb5 -lz -lreadline -lrt -lcrypt -ldl -lm 
  -lpthread    -o ecpg
parser.o: In function `filtered_base_yylex':
/git/postgresql/src/interfaces/ecpg/preproc/parser.c:68: undefined 
reference to `base_yylex'
/git/postgresql/src/interfaces/ecpg/preproc/parser.c:105: undefined 
reference to `base_yylex'
collect2: error: ld returned 1 exit status
make: *** [Makefile:42: ecpg] Error 1

because of ecpg/preproc/parser.c:68 (in REL9_5_STABLE)

With previous versions of flex the compilation has worked.  But I do not 
find in the code where is supposed yylex to be renamed to base_yylex 
when parser.c is proccessed.  pgc.l also does not use %option 
prefix=base_yy, which would be the right way to rename yylex (apart from 
#define YY_DECL).

What is the purpose to rename yylex to base_yylex?

Greetings
   Дилян


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Дилян Палаузов <dpa-postgres@aegee.org> writes:
> with the newest flex (v2.6.2-19-g6bea32e, which is newer than 2.6.3) I
> need this patch to compile postgres:

Ugh.

> With previous versions of flex the compilation has worked.  But I do not
> find in the code where is supposed yylex to be renamed to base_yylex
> when parser.c is proccessed.  pgc.l also does not use %option
> prefix=base_yy, which would be the right way to rename yylex (apart from
> #define YY_DECL).
> What is the purpose to rename yylex to base_yylex?

It's mostly for consistency with the backend, where we have multiple bison
parsers (and multiple flex lexers) so there's a policy of renaming all of
them away from the default name.  Since ecpg is a standalone program,
we could in principle not bother to rename its parser; but I'd rather
keep it in as close sync with the corresponding backend code as possible.

The reason for the current hacky approach is explained in pgc.l:

/** Change symbol names as expected by preproc.y.  It'd be better to do this* with %option prefix="base_yy", but that
affectssome other names that* various files expect *not* to be prefixed with "base_".  Cleaning it up* is not worth the
troubleright now.*/ 
#define yylex           base_yylex
#define yylval          base_yylval

Looks like it's time to pay down that technical debt.
        regards, tom lane


--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

I wrote:
> Looks like it's time to pay down that technical debt.

Patch pushed --- could you confirm things work with that new flex version?

            regards, tom lane


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Hello Tom,

it compiles with flex v2.6.2-19-g6bea32e (which is the newest one, as 
there is no flex 2.6.3).

Greetings
   Дилян

On 12/11/2016 09:04 PM, Tom Lane wrote:
> I wrote:
>> Looks like it's time to pay down that technical debt.
>
> Patch pushed --- could you confirm things work with that new flex version?
>
>             regards, tom lane
>


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Дилян Палаузов <dilyan.palauzov@aegee.org> writes:
> in fact the problem appears not only with 9.6, but also with Pg 9.5.

I wondered about that --- but the code is rather different in 9.5,
and I'm hesitant to try to fix it blind.  Where can I find the version
of flex you're using?
        regards, tom lane


--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Hello Tom,

here: https://github.com/westes/flex

Greetings  Дилян

On 12/11/2016 09:50 PM, Tom Lane wrote:
> Дилян Палаузов <dilyan.palauzov@aegee.org> writes:
>> in fact the problem appears not only with 9.6, but also with Pg 9.5.
>
> I wondered about that --- but the code is rather different in 9.5,
> and I'm hesitant to try to fix it blind.  Where can I find the version
> of flex you're using?
>
>             regards, tom lane
>


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Дилян Палаузов <dilyan.palauzov@aegee.org> writes:
> On 12/11/2016 09:50 PM, Tom Lane wrote:
>> I wondered about that --- but the code is rather different in 9.5,
>> and I'm hesitant to try to fix it blind.  Where can I find the version
>> of flex you're using?

> here: https://github.com/westes/flex

Thanks.  After some experimentation, HEAD now seems fine with
development-tip flex, but in our back branches I see symbol-redefinition
warnings in the main parser and ecpg, along with the link failure you
showed in ecpg.  I believe the reason those warnings aren't there in
HEAD/9.6 is commit 72b1e3a21, which undid our ancient hack of compiling
the scanner files as part of the grammar files.

What I'm inclined to do to make this better is to back-patch 72b1e3a21
along with today's fix.  We've had enough buildfarm and field testing
now to be pretty confident that that change didn't break anything,
and these results show that our old hacks aren't going to survive
contact with newer versions of flex.
        regards, tom lane


--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs