Thread: pgsql: Convert psql's flex lexer to be re-entrant, and make it compile
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.) Branch ------ master Details ------- http://git.postgresql.org/pg/commitdiff/27199058d98ef7ff2f468af44654bc35bb70fe4a Modified Files -------------- src/bin/psql/Makefile | 10 +- src/bin/psql/mainloop.c | 10 -- src/bin/psql/psqlscan.l | 274 ++++++++++++++++++++++++++++++------------------ 3 files changed, 176 insertions(+), 118 deletions(-)
Re: pgsql: Convert psql's flex lexer to be re-entrant, and make it compile
From
Michael Paquier
Date:
On Sat, Mar 19, 2016 at 10:22 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Also, stop compiling psqlscan as part of mainloop.c, and make it a > standalone build target instead. This is a lot cleaner than before, though > it doesn't really change much in practice as of this commit. (I'm not sure > whether the MSVC build scripts will need some help with this part, but the > buildfarm will soon tell us.) This is fine, no tweaking is needed. psql is listing directly psqlscan.l as a file, and .c from generated from .l files with flex are generated unconditionally before the compilation. -- Michael