Thread: pgloader hangs with an invalid filename

Has anyone experienced the behavior of pgloader hanging when the .conf
file has an incorrect filename?
When I execute with the correct filename, everything works just fine.
However, when I use an incorrect filename, I get the below:

PG_DB> pgloader -c hangs.pgload.conf -v
pgloader     INFO     Logger initialized
pgloader     INFO     Reformat path is
['/usr/share/python-support/pgloader/reformat']
pgloader     INFO     Will consider following sections:
pgloader     INFO       emaillistuploaddata
emaillistuploaddata INFO     emaillistuploaddata processing
pgloader     INFO     All threads are started, wait for them to
terminate
emaillistuploaddata ERROR    [Errno 2] No such file or directory:
'mb2.up.txt.complete'
Exception in thread emaillistuploaddata:
Traceback (most recent call last):
  File "/usr/lib64/python2.4/threading.py", line 442, in __bootstrap
    self.run()
  File "/usr/share/python-support/pgloader/pgloader/pgloader.py", line
840, in run
    self.process()
  File "/usr/share/python-support/pgloader/pgloader/pgloader.py", line
1201, in process
    self.data_import()
  File "/usr/share/python-support/pgloader/pgloader/pgloader.py", line
1220, in data_import
    for offsets, line, columns in self.readlines():
  File "/usr/share/python-support/pgloader/pgloader/pgloader.py", line
1138, in readlines
    for offsets, line, columns in self.reader.readlines():
  File "/usr/share/python-support/pgloader/pgloader/textreader.py", line
86, in readlines
    check_count     = False)
  File "/usr/share/python-support/pgloader/pgloader/reader.py", line
192, in __init__
    raise PGLoader_Error, error
PGLoader_Error: [Errno 2] No such file or directory:
'mb2.up.txt.complete'

Here is my hangs.pgload.conf file (I named it 'hangs' because I
intentionally inserted an invalid filename):

[pgsql]
host = localhost
port = 5432
base = PG_DB
user = pg_user
pass = None

log_file            = /tmp/0000.pgload.log
log_min_messages    = DEBUG
client_min_messages = WARNING
lc_messages         = C

;client_encoding = 'utf-8'
client_encoding = 'latin1'
copy_every      = 100000
commit_every    = 100000

null         = ""
empty_string = "\ "


max_parallel_sections = 10

[emaillistuploaddata]
table           = emaillistuploaddata
format          = text
filename        = mb2.up.txt.complete
field_sep       = |
skip_head_lines = 0
columns         = listid,data

We would expect pgloader to kick back an error and exit if the source
file could not be found.

Thanks.

Tony Capobianco <tcapobianco@prospectiv.com> writes:
> Has anyone experienced the behavior of pgloader hanging when the .conf
> file has an incorrect filename?
> When I execute with the correct filename, everything works just fine.
> However, when I use an incorrect filename, I get the below:

That's certainly due to current threading implementation in pgloader, I
guess the control thread is still waiting for the load to terminate,
which will never happen.

It seems to be fixed in current version though.  See if you can upgrade.

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr     PostgreSQL : Expertise, Formation et Support

I'm running pgloader 2.3.2 and Postgres 9.  Is there an upgraded version
of pgloader on the PgFoundry site?


On Mon, 2011-08-15 at 20:34 +0200, Dimitri Fontaine wrote:
> Tony Capobianco <tcapobianco@prospectiv.com> writes:
> > Has anyone experienced the behavior of pgloader hanging when the .conf
> > file has an incorrect filename?
> > When I execute with the correct filename, everything works just fine.
> > However, when I use an incorrect filename, I get the below:
>
> That's certainly due to current threading implementation in pgloader, I
> guess the control thread is still waiting for the load to terminate,
> which will never happen.
>
> It seems to be fixed in current version though.  See if you can upgrade.
>
> Regards,

Tony Capobianco <tcapobianco@prospectiv.com> writes:

> I'm running pgloader 2.3.2 and Postgres 9.  Is there an upgraded version
> of pgloader on the PgFoundry site?

Yeah, it's now pgloader 2.3.3, 3 pre-releases have been done and the
later one, 2.3.3~dev3 will soon'ish get tagged final 2.3.3.  Also the
development now happens at github.

  http://packages.debian.org/search?keywords=pgloader
  http://github.com/dimitri/pgloader

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr     PostgreSQL : Expertise, Formation et Support