Thread: pg_scanner - patch no.1

pg_scanner - patch no.1

From
Vladimir Kokovic
Date:
Hi,

This patch is my try to add lexical scanner for PostgreSQL SQL
commands in Pgadmin source tree(prerequisite for patches no.2 and 3).

The main reason why that is done is to scan and parse a SQL command in
the same way
 like PostgreSQL server, but for limited number of parse functions.

All scanner special cases are covered: TYPECAST(::), COLON_EQUALS(:=),
DOT_DOT(..) and $foo$.

Currently, three parsers implemented:

1. parse SQL command string for commands separated by ';'
      returns single linked list of commands
      struct myScannerNode *get_all_commands_cpp(const struct
myScannerNode *head);

2. parse RULE SQL command and substitute source schema with target schema
      returns CREATE RULE SQL command with new schema name
      char *parse_rule_cpp(const struct myScannerNode *head, const
char *rulename, const char *targetschema, const char *tablename);

3. parse TRIGGER SQL command and substitute source schema with target
schema (pgadmin/utils/pasteTables.cpp)
      returns CREATE TRIGGER SQL command with new schema name
      char *parse_trigger_cpp(const struct myScannerNode *head, const
char *targetschema);


For all parsers 'scan_SQL_cpp' method is prerequisite which scan SQL
command for tokens:
    returns single linked list of scan results (tokens)
    struct myScannerNode *scan_SQL_cpp(const char *command);

If some error occurred during scan process, variable
'pgadmin_scanner_last_error' contains diagnostic message.


*** This code snippet is from my pgadmin/frm/frmQuery.cpp ***

/*
 * The scanner returns extra data about scanned tokens in this union type.
 * Note that this is a subset of the fields used in YYSTYPE of the bison
 * parsers built atop the scanner.
 */
typedef union core_YYSTYPE
{
    int            ival;            /* for integer literals */
    char       *str;            /* for identifiers and non-integer literals */
    const char *keyword;        /* canonical spelling of keywords */
} core_YYSTYPE;


/*
    my token data struct
*/
struct myScannerData {
    int loc;
    int token;
    core_YYSTYPE val;
};


/*
    my single linked list for scan results
*/
struct myScannerNode {
    struct myScannerData data;
    struct myScannerNode *next;
};


//command offest from the beginning of SQL command string for each SQL command
wxArrayInt locs;

//SQL commands (queries)
wxArrayString    queries;

//this is the first step which must be executed
//scan_SQL_cpp returns single linked list of scan results (tokens)

struct myScannerNode *head = scan_SQL_cpp(query.ToUTF8());
if (head)
{
//parse scan results
//get_all_commands_cpp returns single linked list of commands
    struct myScannerNode *result = get_all_commands_cpp(head);

    struct myScannerNode *current = result;
    if (current)
    {
        do
        {
            wxString x = wxString(current->data.val.str, wxConvUTF8);
            queries.Add(x);
            locs.Add(current->data.loc);
            current = current->next;
        } while (current != result);
        destroylist(result);
    }
    destroylist(head);
}



*** This code snippet is from my pgadmin/utils/pasteTables.cpp ***

    struct myScannerNode *head =
scan_SQL_cpp(definition.mb_str(*srcrule->GetConnection()->GetConv()));
    if (head)
    {
        char *rule = parse_rule_cpp(
            head, qtname.ToAscii(), targetschema->GetIdentifier().ToAscii(),
newtablename.ToAscii());
        if (rule)
        {
            newdefinition = wxString(rule, *targetschema->GetConnection()->GetConv());
            free(rule);
        }
        destroylist_cpp(head);
    }


Best regards
Vladimir Kokovic
Belgrade, Serbia, 21.October 2012

Attachment

Re: pg_scanner - patch no.1

From
Dave Page
Date:
Just an FYI - I'm not ignoring your patches; I was away at PGConf.EU
last week and am very busy at the moment. They are on my TODO list
though.

Thanks.

On Sun, Oct 21, 2012 at 9:08 AM, Vladimir Kokovic
<vladimir.kokovic@gmail.com> wrote:
> Hi,
>
> This patch is my try to add lexical scanner for PostgreSQL SQL
> commands in Pgadmin source tree(prerequisite for patches no.2 and 3).
>
> The main reason why that is done is to scan and parse a SQL command in
> the same way
>  like PostgreSQL server, but for limited number of parse functions.
>
> All scanner special cases are covered: TYPECAST(::), COLON_EQUALS(:=),
> DOT_DOT(..) and $foo$.
>
> Currently, three parsers implemented:
>
> 1. parse SQL command string for commands separated by ';'
>       returns single linked list of commands
>       struct myScannerNode *get_all_commands_cpp(const struct
> myScannerNode *head);
>
> 2. parse RULE SQL command and substitute source schema with target schema
>       returns CREATE RULE SQL command with new schema name
>       char *parse_rule_cpp(const struct myScannerNode *head, const
> char *rulename, const char *targetschema, const char *tablename);
>
> 3. parse TRIGGER SQL command and substitute source schema with target
> schema (pgadmin/utils/pasteTables.cpp)
>       returns CREATE TRIGGER SQL command with new schema name
>       char *parse_trigger_cpp(const struct myScannerNode *head, const
> char *targetschema);
>
>
> For all parsers 'scan_SQL_cpp' method is prerequisite which scan SQL
> command for tokens:
>         returns single linked list of scan results (tokens)
>         struct myScannerNode *scan_SQL_cpp(const char *command);
>
> If some error occurred during scan process, variable
> 'pgadmin_scanner_last_error' contains diagnostic message.
>
>
> *** This code snippet is from my pgadmin/frm/frmQuery.cpp ***
>
> /*
>  * The scanner returns extra data about scanned tokens in this union type.
>  * Note that this is a subset of the fields used in YYSTYPE of the bison
>  * parsers built atop the scanner.
>  */
> typedef union core_YYSTYPE
> {
>         int                     ival;                   /* for integer literals */
>         char       *str;                        /* for identifiers and non-integer literals */
>         const char *keyword;            /* canonical spelling of keywords */
> } core_YYSTYPE;
>
>
> /*
>         my token data struct
> */
> struct myScannerData {
>         int loc;
>         int token;
>         core_YYSTYPE val;
> };
>
>
> /*
>         my single linked list for scan results
> */
> struct myScannerNode {
>         struct myScannerData data;
>         struct myScannerNode *next;
> };
>
>
> //command offest from the beginning of SQL command string for each SQL command
> wxArrayInt locs;
>
> //SQL commands (queries)
> wxArrayString   queries;
>
> //this is the first step which must be executed
> //scan_SQL_cpp returns single linked list of scan results (tokens)
>
> struct myScannerNode *head = scan_SQL_cpp(query.ToUTF8());
> if (head)
> {
> //parse scan results
> //get_all_commands_cpp returns single linked list of commands
>         struct myScannerNode *result = get_all_commands_cpp(head);
>
>         struct myScannerNode *current = result;
>         if (current)
>         {
>                 do
>                 {
>                         wxString x = wxString(current->data.val.str, wxConvUTF8);
>                         queries.Add(x);
>                         locs.Add(current->data.loc);
>                         current = current->next;
>                 } while (current != result);
>                 destroylist(result);
>         }
>         destroylist(head);
> }
>
>
>
> *** This code snippet is from my pgadmin/utils/pasteTables.cpp ***
>
>         struct myScannerNode *head =
> scan_SQL_cpp(definition.mb_str(*srcrule->GetConnection()->GetConv()));
>         if (head)
>         {
>                 char *rule = parse_rule_cpp(
>                         head, qtname.ToAscii(), targetschema->GetIdentifier().ToAscii(),
> newtablename.ToAscii());
>                 if (rule)
>                 {
>                         newdefinition = wxString(rule, *targetschema->GetConnection()->GetConv());
>                         free(rule);
>                 }
>                 destroylist_cpp(head);
>         }
>
>
> Best regards
> Vladimir Kokovic
> Belgrade, Serbia, 21.October 2012
>
>
> --
> Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgadmin-hackers
>



--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: pg_scanner - patch no.1

From
Dave Page
Date:
Hi

On Sun, Oct 21, 2012 at 9:08 AM, Vladimir Kokovic
<vladimir.kokovic@gmail.com> wrote:
> Hi,
>
> This patch is my try to add lexical scanner for PostgreSQL SQL
> commands in Pgadmin source tree(prerequisite for patches no.2 and 3).
>
> The main reason why that is done is to scan and parse a SQL command in
> the same way
>  like PostgreSQL server, but for limited number of parse functions.
>
> All scanner special cases are covered: TYPECAST(::), COLON_EQUALS(:=),
> DOT_DOT(..) and $foo$.

OK, so I finally got a chance to look at this. Unfortunately, it
doesn't compile here :-(

ccache gcc -c -DSCANNER_VERSION=92 -DPGADMIN_SCANNER
-I./../../xtra/pg_scanners \
-I./../../pgadmin/include/utils \
-I./../../xtra/pg_scanners/src/backend/parser/92
-I./../../xtra/pg_scanners/src/include \
-Wall -I`/usr/local/pgsql/bin/pg_config --includedir`
-I`/usr/local/pgsql/bin/pg_config --includedir-server` -o scansup.o
../../xtra/pg_scanners/scansup.c
ccache gcc -c -DSCANNER_VERSION=92 -DPGADMIN_SCANNER
-I./../../xtra/pg_scanners \
-I./../../pgadmin/include/utils \
-I./../../xtra/pg_scanners/src/backend/parser/92
-I./../../xtra/pg_scanners/src/include \
-Wall -I`/usr/local/pgsql/bin/pg_config --includedir`
-I`/usr/local/pgsql/bin/pg_config --includedir-server` -DXVER=0x92 -o
PgadminScannerCommon.o ../../xtra/pg_scanners/PgadminScannerCommon.c
ccache gcc -c -DSCANNER_VERSION=92 -DPGADMIN_SCANNER
-I./../../xtra/pg_scanners \
-I./../../pgadmin/include/utils \
-I./../../xtra/pg_scanners/src/backend/parser/92
-I./../../xtra/pg_scanners/src/include \
-Wall -I`/usr/local/pgsql/bin/pg_config --includedir`
-I`/usr/local/pgsql/bin/pg_config --includedir-server` -DXVER=0x92 -o
PgadminScanner92.o ../../xtra/pg_scanners/PgadminScanner.c
In file included from ../../xtra/pg_scanners/scansup.c:21:
./../../pgadmin/include/utils/PgadminScanner.h:1088: error:
conflicting types for ‘ExceptionalCondition’
/usr/local/pgsql/include/server/postgres.h:688: error: previous
declaration of ‘ExceptionalCondition’ was here
make[3]: *** [scansup.o] Error 1
make[3]: *** Waiting for unfinished jobs....
In file included from ../../xtra/pg_scanners/PgadminScannerCommon.c:4:
./../../pgadmin/include/utils/PgadminScanner.h:1088: error:
conflicting types for ‘ExceptionalCondition’
/usr/local/pgsql/include/server/postgres.h:688: error: previous
declaration of ‘ExceptionalCondition’ was here
../../xtra/pg_scanners/PgadminScannerCommon.c:155: error: conflicting
types for ‘ExceptionalCondition’
/usr/local/pgsql/include/server/postgres.h:688: error: previous
declaration of ‘ExceptionalCondition’ was here
make[3]: *** [PgadminScannerCommon.o] Error 1
In file included from ../../xtra/pg_scanners/PgadminScanner.c:4:
./../../pgadmin/include/utils/PgadminScanner.h:1088: error:
conflicting types for ‘ExceptionalCondition’
/usr/local/pgsql/include/server/postgres.h:688: error: previous
declaration of ‘ExceptionalCondition’ was here
make[3]: *** [PgadminScanner92.o] Error 1
make[2]: *** [../xtra/pg_scanners/libpgscanner92.a] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

That's on Mac OS X 10.8.2 with i686-apple-darwin11-llvm-gcc-4.2 (GCC)
4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00),
building against PG 9.1.

Can you look into that please?

(one minor nit-pick about the patch, please don't capitalise the first
letter of pgAdmin, e.g. use pgadminScannerCommon.c not
PgadminScannerCommon.c)

Thanks.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: pg_scanner - patch no.1

From
Vladimir Kokovic
Date:
Hi,

1. pg_scanner is PostgreSQL 9.2 version, but I've made some changes to
build with earlier versions(I hope).
2. PgadminScannerCommon.c renamed to pgadminScannerCommon.c
    PgadminScanner.c renamed to pgadminScanner.c
    PgadminScanner.h renamed to pgadminScanner.h

Best regards
Vladimir Kokovic
Belgrade, Serbia, 9.November 2012

Attachment

Re: pg_scanner - patch no.1

From
Dave Page
Date:
Hi

On Fri, Nov 9, 2012 at 5:23 AM, Vladimir Kokovic
<vladimir.kokovic@gmail.com> wrote:
> Hi,
>
> 1. pg_scanner is PostgreSQL 9.2 version, but I've made some changes to
> build with earlier versions(I hope).
> 2. PgadminScannerCommon.c renamed to pgadminScannerCommon.c
>     PgadminScanner.c renamed to pgadminScanner.c
>     PgadminScanner.h renamed to pgadminScanner.h

OK, we're getting there. I'm seeing an issue now that you probably
won't see unless you have a Mac to hand. Essentially, the scanner is
being built in the wrong architecture, so linking fails:

...
...
-lxml2 -lz -lpthread -liconv -lm ../xtra/pg_scanners/keywords.o
../xtra/pg_scanners/kwlookup.o ../xtra/pg_scanners/pgstrcasecmp.o
../xtra/pg_scanners/chklocale.o ../xtra/pg_scanners/wchar.o
../xtra/pg_scanners/mbutils.o ../xtra/pg_scanners/encnames.o
../xtra/pg_scanners/scansup.o
../xtra/pg_scanners/pgadminScannerCommon.o
../xtra/pg_scanners/libpgscanner92.a -L/usr/local/pgsql/lib -lpq
ld: warning: ignoring file ../xtra/pg_scanners/keywords.o, file was
built for unsupported file format ( 0xcf 0xfa 0xed 0xfe 0x 7 0x 0 0x 0
0x 1 0x 3 0x 0 0x 0 0x 0 0x 1 0x 0 0x 0 0x 0 ) which is not the
architecture being linked (i386): ../xtra/pg_scanners/keywords.o
ld: warning: ignoring file ../xtra/pg_scanners/kwlookup.o, file was
built for unsupported file format ( 0xcf 0xfa 0xed 0xfe 0x 7 0x 0 0x 0
0x 1 0x 3 0x 0 0x 0 0x 0 0x 1 0x 0 0x 0 0x 0 ) which is not the
architecture being linked (i386): ../xtra/pg_scanners/kwlookup.o
ld: warning: ignoring file ../xtra/pg_scanners/pgstrcasecmp.o, file
was built for unsupported file format ( 0xcf 0xfa 0xed 0xfe 0x 7 0x 0
0x 0 0x 1 0x 3 0x 0 0x 0 0x 0 0x 1 0x 0 0x 0 0x 0 ) which is not the
architecture being linked (i386): ../xtra/pg_scanners/pgstrcasecmp.o
ld: warning: ignoring file ../xtra/pg_scanners/chklocale.o, file was
built for unsupported file format ( 0xcf 0xfa 0xed 0xfe 0x 7 0x 0 0x 0
0x 1 0x 3 0x 0 0x 0 0x 0 0x 1 0x 0 0x 0 0x 0 ) which is not the
architecture being linked (i386): ../xtra/pg_scanners/chklocale.o
ld: warning: ignoring file ../xtra/pg_scanners/wchar.o, file was built
for unsupported file format ( 0xcf 0xfa 0xed 0xfe 0x 7 0x 0 0x 0 0x 1
0x 3 0x 0 0x 0 0x 0 0x 1 0x 0 0x 0 0x 0 ) which is not the
architecture being linked (i386): ../xtra/pg_scanners/wchar.o
ld: warning: ignoring file ../xtra/pg_scanners/mbutils.o, file was
built for unsupported file format ( 0xcf 0xfa 0xed 0xfe 0x 7 0x 0 0x 0
0x 1 0x 3 0x 0 0x 0 0x 0 0x 1 0x 0 0x 0 0x 0 ) which is not the
architecture being linked (i386): ../xtra/pg_scanners/mbutils.o
ld: warning: ignoring file ../xtra/pg_scanners/encnames.o, file was
built for unsupported file format ( 0xcf 0xfa 0xed 0xfe 0x 7 0x 0 0x 0
0x 1 0x 3 0x 0 0x 0 0x 0 0x 1 0x 0 0x 0 0x 0 ) which is not the
architecture being linked (i386): ../xtra/pg_scanners/encnames.o
ld: warning: ignoring file ../xtra/pg_scanners/scansup.o, file was
built for unsupported file format ( 0xcf 0xfa 0xed 0xfe 0x 7 0x 0 0x 0
0x 1 0x 3 0x 0 0x 0 0x 0 0x 1 0x 0 0x 0 0x 0 ) which is not the
architecture being linked (i386): ../xtra/pg_scanners/scansup.o
ld: warning: ignoring file ../xtra/pg_scanners/pgadminScannerCommon.o,
file was built for unsupported file format ( 0xcf 0xfa 0xed 0xfe 0x 7
0x 0 0x 0 0x 1 0x 3 0x 0 0x 0 0x 0 0x 1 0x 0 0x 0 0x 0 ) which is not
the architecture being linked (i386):
../xtra/pg_scanners/pgadminScannerCommon.o
ld: warning: ignoring file ../xtra/pg_scanners/libpgscanner92.a, file
was built for archive which is not the architecture being linked
(i386): ../xtra/pg_scanners/libpgscanner92.a

I've spotted what I think are a few issues with the build system
changes, which will contribute to this, and possibly other problems we
may see in the future:

- I believe the above issue is caused by not adding $OSX_ARCH to the
LDADD variable for pg_scanners. See /acinclude.m4, which sets this
sort of thing up and ensures we get the right flags passed to every
individual build in the code.

- You shouldn't use ../xtra/pg_scanners in Makefiles, but
$(top_srcdir)/xtra/pg_scanners. You can use the png2c as a roughly
equivalent guide.

- Do you need to define the rules for each source file explicitly in
the Makefile? At minimum this should be a generic rule automatically
picking up all source files in the SOURCES list - ideally there should
be no explicit rules there at all.

Some other questions:

- At some point we're going to need to go through all this on Windows
as well :-/. Do you have a Windows system to work on that when we get
to it?

- Currently the code builds scanners with "92" in the name. Is this
intended to allow us to have multiple versions of the same scanner in
the future? If so, then it will also need to allow for Postgres Plus
Advanced Server and Greenplum DB scanners, which may have the same
versions but do support different syntax. If not, we should probably
just remove the "92" and make it as generic as possible.

- Can you please add a README file to xtra/pg_scanners that describes
the various files in there, and explains what needs to be updated to
move us to a new scanner from a different version of PostgreSQL?

Thanks!

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: pg_scanner - patch no.1

From
Vladimir Kokovic
Date:
Hi,

On 11/9/12, Dave Page <dpage@pgadmin.org> wrote:

> I've spotted what I think are a few issues with the build system
> changes, which will contribute to this, and possibly other problems we
> may see in the future:
>
> - I believe the above issue is caused by not adding $OSX_ARCH to the
> LDADD variable for pg_scanners. See /acinclude.m4, which sets this
> sort of thing up and ensures we get the right flags passed to every
> individual build in the code.


Missing CFLAGS="$CFLAGS $OSX_ARCH"  added.


> - You shouldn't use ../xtra/pg_scanners in Makefiles, but
> $(top_srcdir)/xtra/pg_scanners. You can use the png2c as a roughly
> equivalent guide.


pgAdmin make system is VPATH enabled.
My build script 'build-debug.sh' works as expected:

#!/bin/sh
set -v
set -e

rm -rf debug
mkdir debug

cd pgadmin3
bash bootstrap

cd ./pgadmin
./ver_svn.sh
cd ../../debug

export CXXFLAGS="-gdwarf-2 -g3"
../pgadmin3/configure --prefix=/usr/local/pgadmin3-debug
--enable-debug --enable-databasedesigner --srcdir=../pgadmin3
--with-pgsql=/home/src/postgresql-devel/20120502 >
configure-out-debug.log 2>&1
make > make-out-debug.log 2>&1
make install > make-install-out-debug.log 2>&1

exit 0


> - Do you need to define the rules for each source file explicitly in
> the Makefile? At minimum this should be a generic rule automatically
> picking up all source files in the SOURCES list - ideally there should
> be no explicit rules there at all.


xtra/pg_scanners/Makefile.am is now simplified.


> Some other questions:
>
> - At some point we're going to need to go through all this on Windows
> as well :-/. Do you have a Windows system to work on that when we get
> to it?


I am linux only man !


> - Currently the code builds scanners with "92" in the name. Is this
> intended to allow us to have multiple versions of the same scanner in
> the future? If so, then it will also need to allow for Postgres Plus
> Advanced Server and Greenplum DB scanners, which may have the same
> versions but do support different syntax. If not, we should probably
> just remove the "92" and make it as generic as possible.


Name is optional, but folder name is xtra/pg_scanners/ which means the
place for more than one.
I like "92".


> - Can you please add a README file to xtra/pg_scanners that describes
> the various files in there, and explains what needs to be updated to
> move us to a new scanner from a different version of PostgreSQL?


xtra/pg_scanners/README added.
Please Dave,  take look for some English error or bad terms. Thanks.


Best regards
Vladimir Kokovic
Belgrade, Serbia, 10.November 2012

Attachment

Re: pg_scanner - patch no.1

From
Dave Page
Date:
Hi

On Sat, Nov 10, 2012 at 2:51 PM, Vladimir Kokovic
<vladimir.kokovic@gmail.com> wrote:
> Hi,
>
> On 11/9/12, Dave Page <dpage@pgadmin.org> wrote:
>
>> I've spotted what I think are a few issues with the build system
>> changes, which will contribute to this, and possibly other problems we
>> may see in the future:
>>
>> - I believe the above issue is caused by not adding $OSX_ARCH to the
>> LDADD variable for pg_scanners. See /acinclude.m4, which sets this
>> sort of thing up and ensures we get the right flags passed to every
>> individual build in the code.
>
>
> Missing CFLAGS="$CFLAGS $OSX_ARCH"  added.

OK.

>> - You shouldn't use ../xtra/pg_scanners in Makefiles, but
>> $(top_srcdir)/xtra/pg_scanners. You can use the png2c as a roughly
>> equivalent guide.
>
>
> pgAdmin make system is VPATH enabled.
> My build script 'build-debug.sh' works as expected:

Maybe, but that doesn't make needless inconsistency acceptable.

>> - Do you need to define the rules for each source file explicitly in
>> the Makefile? At minimum this should be a generic rule automatically
>> picking up all source files in the SOURCES list - ideally there should
>> be no explicit rules there at all.
>
>
> xtra/pg_scanners/Makefile.am is now simplified.

Hmm, that's definitely wrong - any parts of the pgadmin3 build target
should be in $SRC/pgadmin. Having it in xtra/ is acceptable if it's
being built as a library that then gets linked into other projects,
but if we're adding to pgadmin3_LDADD, then the code should definitely
not be in xtra/ but in pgadmin/. I'd prefer it not be a library
personally, so can you move it please?

>> Some other questions:
>>
>> - At some point we're going to need to go through all this on Windows
>> as well :-/. Do you have a Windows system to work on that when we get
>> to it?
>
>
> I am linux only man !

:-)

>> - Currently the code builds scanners with "92" in the name. Is this
>> intended to allow us to have multiple versions of the same scanner in
>> the future? If so, then it will also need to allow for Postgres Plus
>> Advanced Server and Greenplum DB scanners, which may have the same
>> versions but do support different syntax. If not, we should probably
>> just remove the "92" and make it as generic as possible.
>
>
> Name is optional, but folder name is xtra/pg_scanners/ which means the
> place for more than one.
> I like "92".

92 is fine - my point is that we support 3 different types of
PostgreSQL server, that each may have a 9.2 version with different
syntax, so if we're including the version number to allow multiple
scanners in the future, then we should also include something to
differentiate the forks - eg. pg92 for PostgreSQL, as92 for Postgres
Plus Advanced Server and gp92 for Greenplum Database.

>> - Can you please add a README file to xtra/pg_scanners that describes
>> the various files in there, and explains what needs to be updated to
>> move us to a new scanner from a different version of PostgreSQL?
>
>
> xtra/pg_scanners/README added.
> Please Dave,  take look for some English error or bad terms. Thanks.

Sure, I'll do that during commit.

Thanks.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: pg_scanner - patch no.1

From
Vladimir Kokovic
Date:
Hi,

On 11/13/12, Dave Page <dpage@pgadmin.org> wrote:

> Hmm, that's definitely wrong - any parts of the pgadmin3 build target
> should be in $SRC/pgadmin. Having it in xtra/ is acceptable if it's
> being built as a library that then gets linked into other projects,
> but if we're adding to pgadmin3_LDADD, then the code should definitely
> not be in xtra/ but in pgadmin/. I'd prefer it not be a library
> personally, so can you move it please?
>

Moved from xtra/pg_scanners to pgadmin/pg_scanners.

>
> 92 is fine - my point is that we support 3 different types of
> PostgreSQL server, that each may have a 9.2 version with different
> syntax, so if we're including the version number to allow multiple
> scanners in the future, then we should also include something to
> differentiate the forks - eg. pg92 for PostgreSQL, as92 for Postgres
> Plus Advanced Server and gp92 for Greenplum Database.
>

For now, pgadmin/pg_scanners contains only pg92 scanner !

Best regards
Vladimir Kokovic
Belgrade, Serbia, 14.November 2012

Attachment

Re: pg_scanner - patch no.1

From
Ashesh Vashi
Date:
On Wed, Nov 14, 2012 at 11:37 PM, Vladimir Kokovic <vladimir.kokovic@gmail.com> wrote:
Hi,

On 11/13/12, Dave Page <dpage@pgadmin.org> wrote:

> Hmm, that's definitely wrong - any parts of the pgadmin3 build target
> should be in $SRC/pgadmin. Having it in xtra/ is acceptable if it's
> being built as a library that then gets linked into other projects,
> but if we're adding to pgadmin3_LDADD, then the code should definitely
> not be in xtra/ but in pgadmin/. I'd prefer it not be a library
> personally, so can you move it please?
>

Moved from xtra/pg_scanners to pgadmin/pg_scanners.

>
> 92 is fine - my point is that we support 3 different types of
> PostgreSQL server, that each may have a 9.2 version with different
> syntax, so if we're including the version number to allow multiple
> scanners in the future, then we should also include something to
> differentiate the forks - eg. pg92 for PostgreSQL, as92 for Postgres
> Plus Advanced Server and gp92 for Greenplum Database.
>

For now, pgadmin/pg_scanners contains only pg92 scanner !
Not able to apply your patch cleanly.
$ git apply ~/Downloads/pgadmin3-1.diff
/Users/ashesh/Downloads/pgadmin3-1.diff:9: trailing whitespace.
CFLAGS="$CFLAGS $OSX_ARCH" 
fatal: corrupt patch at line 402
$ git reset HEAD --hard
HEAD is now at ff69692 Fix bad wording in create role dialog
$ patch -p1 < ~/Downloads/pgadmin3-1.diff
patching file acinclude.m4
patching file pgadmin/Makefile.am
patching file pgadmin/db/keywords.c
patching file pgadmin/include/parser/keywords.h
patching file pgadmin/include/parser/keywords_pgadmin.h
patching file pgadmin/include/parser/module.mk
patching file pgadmin/include/utils/module.mk
patch: **** malformed patch at line 402: diff --git a/pgadmin/include/utils/pgadminScanner.h b/pgadmin/include/utils/pgadminScanner.h

Can you please take a look at it?

--

Thanks & Regards,
Ashesh Vashi

EnterpriseDB INDIA: Enterprise PostgreSQL Company

http://www.linkedin.com/in/asheshvashi 


Best regards
Vladimir Kokovic
Belgrade, Serbia, 14.November 2012


--
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers


Re: pg_scanner - patch no.1

From
Vladimir Kokovic
Date:
Hi,

Sorry.
New diff is now tested with git HEAD repository !

root@vlD-kuci:/tmp#
root@vlD-kuci:/tmp# cd pgadmin-git/pgadmin3; git apply
/root/files/pgadmin/unified-diff-move1/pgadmin3-1.diff; cd /tmp
root@vlD-kuci:/tmp#

Best regards
Vladimir Kokovic
Belgrade, Serbia, 19.November 2012

Attachment

Re: pg_scanner - patch no.1

From
Ashesh Vashi
Date:
On Mon, Nov 19, 2012 at 10:57 PM, Vladimir Kokovic <vladimir.kokovic@gmail.com> wrote:
Hi,

Sorry.
No need to say sorry.
We're all trying create a good product and thanks for putting the efforts. 
New diff is now tested with git HEAD repository !
- There is no reason, we need two different files of keywords with similar content.
  Please reuse the header - pgadmin/include/parser/keywords.h

When compiled with PostgreSQL 9.0.3, I got this error.
gcc -c -arch i386 -arch i386 -DSCANNER_VERSION=92 -DPGADMIN_SCANNER -I./pg_scanners/pg92 -I./include/utils -I./pg_scanners/pg92/src/backend/parser/92 -I./pg_scanners/pg92/src/include -Wall -I`/Users/ashesh/Developments/pg//bin/pg_config --includedir` -I`/Users/ashesh/Developments/pg//bin/pg_config --includedir-server` -o pg_scanners/pg92/keywords.o pg_scanners/pg92/src/backend/parser/keywords.c
In file included from pg_scanners/pg92/src/backend/parser/keywords.c:29:
/Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:97: error: ‘CREATEDB’ undeclared here (not in a function)
/Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:98: error: ‘CREATEROLE’ undeclared here (not in a function)
/Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:99: error: ‘CREATEUSER’ undeclared here (not in a function)
/Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:229: error: ‘LOGIN_P’ undeclared here (not in a function)
/Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:245: error: ‘NOCREATEDB’ undeclared here (not in a function)
/Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:246: error: ‘NOCREATEROLE’ undeclared here (not in a function)
/Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:247: error: ‘NOCREATEUSER’ undeclared here (not in a function)
/Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:248: error: ‘NOINHERIT’ undeclared here (not in a function)
/Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:249: error: ‘NOLOGIN_P’ undeclared here (not in a function)
/Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:251: error: ‘NOSUPERUSER’ undeclared here (not in a function)
/Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:356: error: ‘SUPERUSER_P’ undeclared here (not in a function)
make[3]: *** [pg_scanners/pg92/keywords.o] Error 1
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

--

Thanks & Regards,


Ashesh Vashi

EnterpriseDB INDIA: Enterprise PostgreSQL Company

http://www.linkedin.com/in/asheshvashi 


root@vlD-kuci:/tmp#
root@vlD-kuci:/tmp# cd pgadmin-git/pgadmin3; git apply
/root/files/pgadmin/unified-diff-move1/pgadmin3-1.diff; cd /tmp
root@vlD-kuci:/tmp#

Best regards
Vladimir Kokovic
Belgrade, Serbia, 19.November 2012

Re: pg_scanner - patch no.1

From
Vladimir Kokovic
Date:
Hi,

On 11/20/12, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:
> When compiled with PostgreSQL 9.0.3, I got this error.
> *gcc -c -arch i386 -arch i386 -DSCANNER_VERSION=92 -DPGADMIN_SCANNER
> -I./pg_scanners/pg92 -I./include/utils
> -I./pg_scanners/pg92/src/backend/parser/92 -I./pg_scanners/pg92/src/include
> -Wall -I`/Users/ashesh/Developments/pg//bin/pg_config --includedir`
> -I`/Users/ashesh/Developments/pg//bin/pg_config --includedir-server` -o
> pg_scanners/pg92/keywords.o pg_scanners/pg92/src/backend/parser/keywords.c*
> *In file included from pg_scanners/pg92/src/backend/parser/keywords.c:29:*
> */Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:97:
> error: ‘CREATEDB’ undeclared here (not in a function)*
> */Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:98:
> error: ‘CREATEROLE’ undeclared here (not in a function)*
> */Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:99:
> error: ‘CREATEUSER’ undeclared here (not in a function)*
> */Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:229:
> error: ‘LOGIN_P’ undeclared here (not in a function)*
> */Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:245:
> error: ‘NOCREATEDB’ undeclared here (not in a function)*
> */Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:246:
> error: ‘NOCREATEROLE’ undeclared here (not in a function)*
> */Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:247:
> error: ‘NOCREATEUSER’ undeclared here (not in a function)*
> */Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:248:
> error: ‘NOINHERIT’ undeclared here (not in a function)*
> */Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:249:
> error: ‘NOLOGIN_P’ undeclared here (not in a function)*
> */Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:251:
> error: ‘NOSUPERUSER’ undeclared here (not in a function)*
> */Users/ashesh/Developments/pg/include/postgresql/server/parser/kwlist.h:356:
> error: ‘SUPERUSER_P’ undeclared here (not in a function)*
> *make[3]: *** [pg_scanners/pg92/keywords.o] Error 1*
> *make[2]: *** [all] Error 2*
> *make[1]: *** [all-recursive] Error 1*
> *make: *** [all] Error 2*
>

This version is tested with PostgreSQL 9.0.3,
Even scanner is from 9.2, now I hope that build and run on any 9.x will succeed.

Best regards
Vladimir Kokovic
Belgrade, Serbia, 20.November 2012

Attachment

Re: pg_scanner - patch no.1

From
Vladimir Kokovic
Date:
Hi,

This version is with 9.3devel support added.

Best regards
Vladimir Kokovic
Belgrade, Serbia, 31.December 2012

Attachment

Re: pg_scanner - patch no.1

From
Dave Page
Date:
On Mon, Dec 31, 2012 at 5:18 AM, Vladimir Kokovic
<vladimir.kokovic@gmail.com> wrote:
> Hi,
>
> This version is with 9.3devel support added.

Thanks Vladimir. Ashesh, I assume you'll look at this when you get a spare hour?

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: pg_scanner - patch no.1

From
Ashesh Vashi
Date:
On Wed, Jan 2, 2013 at 4:17 PM, Dave Page <dpage@pgadmin.org> wrote:
On Mon, Dec 31, 2012 at 5:18 AM, Vladimir Kokovic
<vladimir.kokovic@gmail.com> wrote:
> Hi,
>
> This version is with 9.3devel support added.

Thanks Vladimir. Ashesh, I assume you'll look at this when you get a spare hour?
Yeah - I will spend time by end of this week or next week.
-- Ashesh  

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company