Thread: Google SoC idea: FTS support in GUI tools

Hello,

There are many users (especially novice) asking about better and
easier configuration of tsearch2, which is quite complicated
currently. Even new FTS features in 8.3 (e.g. much clearer
configuration with SQL commands) won't help much for users that want
simple "several clicks" setup of a small system with full text
indexing. Adding support of nice user-friendly FTS configuration in
popular PostgreSQL GUI administration tools would help a lot to spread
PostgreSQL among wide group of potential users. Also certain GUI
features can be helpful even for experienced users.

That is why I would like to propose idea to implement FTS
configuration functionality in GUI tools in a frame of Google SoC
2007.

Basic plan of the work is to code all interfaces and debug ideas
firstly in phpPgAdmin and then re-implement ready solutions in
pgadmin3. I consider below features as most useful and would like to
hear any comments what to change/add to this list.

- interfaces to add/modify FTS configurations, dictionaries and tokens
with their processing rules (with drop down menus to exclude typos and
other useful features)
- user-frienly visualisation of ts_debug() to simplify understanding
of what processing happened with the input text and swtiches of
configuration against which we tested the input
- FTS operators in usual phpPgAdmin table select interface
- online query rewriting (special table to store rewrite rules and
test if given rule works as expected); this is especially helpful
because does not require reindexing
- possible interface to index statistics and heuristics to understand
if its healthy (to be discussed with Oleg)
- user-friendly index creation (tsvector column + FTS index + trigger
on parent text field)
- ability to check if FTS configured properly. This includes:
  * some knowledge contained in the interface to perform tests
  * possibly several FTS core features like check if dictionary files
exist and are accessible (will be proposed before 8.3 feature freeze,
to be discussed with Oleg)
  * ability to reset configuration and re-parse all dictionary files
- all user actions should generate on demand valid SQL suitable for
copy-paste into other clients
- possibly implement simple PHP interface for text searches with
themes support to simplify building of FTS-enabled web-interfaces
("just copypaste this PHP code and one line of HTML form code to your
site and switch on FTS on given table in phpPgAdmin")

Open questions:
- what to change/add to this list?
- should we support old tsearch2 configuration for backward
compatibilty with versions prior to 8.3? This seriously increases
amount of work to be done.
- should we analyze FTS configuration experience and interfaces
existing in $$$ databases?

One of the advantage of this work would be independance from
PostgreSQL 8.3 release that does not fit into SoC schedule. It means
that one does not need to wait till next PostgreSQL release since SoC
results can be released just after it finishes with nearest phpPgAdmin
release.

Any feedback is welcome.


Best regards,
Ivan Zolotukhin

[Forgot to CC the list]

Ivan Zolotukhin wrote:
> Hello,
>
> There are many users (especially novice) asking about better and
> easier configuration of tsearch2, which is quite complicated
> currently. Even new FTS features in 8.3 (e.g. much clearer
> configuration with SQL commands) won't help much for users that want
> simple "several clicks" setup of a small system with full text
> indexing. Adding support of nice user-friendly FTS configuration in
> popular PostgreSQL GUI administration tools would help a lot to spread
> PostgreSQL among wide group of potential users. Also certain GUI
> features can be helpful even for experienced users.
>
> That is why I would like to propose idea to implement FTS
> configuration functionality in GUI tools in a frame of Google SoC
> 2007.
>
> Basic plan of the work is to code all interfaces and debug ideas
> firstly in phpPgAdmin and then re-implement ready solutions in
> pgadmin3. I consider below features as most useful and would like to
> hear any comments what to change/add to this list.

Hi,

This would certainly be a useful addition to pgAdmin. Some quick
comments though:

- This is the pgAdmin hackers list, not phpPgAdmin which seems to be
your primary focus :-(. You might want to post to their list as well.

- You propose to implement support in both phpPgAdmin and then pgAdmin.
They are *very* different architectures, and in particular pgAdmin can
have quite a steep learning curve. Would you be able to successfully
work on both projects in the required timeframe?

- phpPgAdmin is a web app whereas pgAdmin is a native desktop app. This
results in applications that are very different to use for the user -
can you successfully prototype desktop functionality in a web app?


> Open questions:
> - what to change/add to this list?

It seems quite an ambitious list. I would expect it to take me a couple
of weeks to implement basic configuration support in pgAdmin, nevermind
the diagnostics functionality you describe. Even working on just one of
the two interfaces this seems like an ambitious project.

> - should we support old tsearch2 configuration for backward
> compatibilty with versions prior to 8.3? This seriously increases
> amount of work to be done.

No, I don't think so (and wouldn't want it in pgadmin for sure).

> - should we analyze FTS configuration experience and interfaces
> existing in $$$ databases?

It's always worth researching the approaches used in other products
where possible. Of course, you must be careful not to use patented
algorithms but that is rarely an issue in UI design.

Regards, Dave.

Dave Page wrote:
> It's always worth researching the approaches used in other products
> where possible. Of course, you must be careful not to use patented
> algorithms but that is rarely an issue in UI design.

You might want to check if amazon hasn't acquired a patent on
One-Click-FTS though ;-)

greetings, Florian Pflug

Hi,

> - This is the pgAdmin hackers list, not phpPgAdmin which seems to be
> your primary focus :-(. You might want to post to their list as well.

I posted a copy of my initial message to their sourceforge trackers.

> - You propose to implement support in both phpPgAdmin and then pgAdmin.
> They are *very* different architectures, and in particular pgAdmin can
> have quite a steep learning curve. Would you be able to successfully
> work on both projects in the required timeframe?

I'm essentially a web programmer with a very nice experience (several
big internet applications in Russia), but my C skills are modest in
this sense. It means that there's a chance that I finish phpPgAdmin
functionality before the end of SoC time and will have some time to
study pgAdmin and implement at least something. Anyway I will try to
continue this work after SoC also. I would say that phpPgAdmin
functionality is a must and pgAdmin functionality is an option I will
be glad to develop after SoC. Also, wxWidgets is not so difficult to
study, AFAIK.

> - phpPgAdmin is a web app whereas pgAdmin is a native desktop app. This
> results in applications that are very different to use for the user -
> can you successfully prototype desktop functionality in a web app?

Did not try yet, but there are many similar things from
user/interfaces point of view. So with no doubt interfaces, ideas and
approaches implemented in phpPgAdmin will help in pgadmin3
developement.

> It seems quite an ambitious list. I would expect it to take me a couple
> of weeks to implement basic configuration support in pgAdmin, nevermind
> the diagnostics functionality you describe. Even working on just one of
> the two interfaces this seems like an ambitious project.

Thanks but I really think that there will be some time left from phpPgAdmin.

> > - should we support old tsearch2 configuration for backward
> > compatibilty with versions prior to 8.3? This seriously increases
> > amount of work to be done.
>
> No, I don't think so (and wouldn't want it in pgadmin for sure).

Agree, thanks for your opinion.

Regards,
Ivan Zolotukhin

Florian G. Pflug wrote:
> Dave Page wrote:
>> It's always worth researching the approaches used in other products
>> where possible. Of course, you must be careful not to use patented
>> algorithms but that is rarely an issue in UI design.
>
> You might want to check if amazon hasn't acquired a patent on
> One-Click-FTS though ;-)

I did say rarely, Amazon being the only example I could actually think of!

/D