Thread: insert a text file into a variable in order to insert into a bytea column

insert a text file into a variable in order to insert into a bytea column

From

Arthur Lewis

Date:

28 January 2016, 21:10:58

Hello,
I have a file called file.txt that is 53604 bytes.  When I do the following

\set stext `cat file.txt`

select length(:'stext'); length
-------- 53603

it seems that one byte (the last newline character) is chomped off.  How can I import a text file into a variable and
insertit into a bytea column without losing the last newline byte.
 

What I want to do is something like this:

\set stext `cat file.txt`
insert into stored_files (file_name,file_text) values ('file',:'stext');

without losing a character within the variable stext.

In table stored_files, file_name is text and file_text is bytea.

I'm using psql 9.3.10 on Ubuntu 14 Linux (32 bit.)
I cannot use pg_read_file because that is for the server side, and I'm on the client side.

There should be an easy way to get a text file into a database column.

Thank you,
Arthur Lewis

Re: insert a text file into a variable in order to insert into a bytea column

From

Adrian Klaver

Date:

28 January 2016, 21:19:52

On 01/28/2016 10:10 AM, Arthur Lewis wrote:
> Hello,
> I have a file called file.txt that is 53604 bytes.  When I do the following
>
> \set stext `cat file.txt`
>
> select length(:'stext'); length
> --------
>    53603
>
> it seems that one byte (the last newline character) is chomped off.  How can I import a text file into a variable and
insertit into a bytea column without losing the last newline byte.
 


Are sure it is not cat that is chomping the byte?:

aklaver@killi:~> cat test.txt
Test
second line

aklaver@killi:~> cat -A test.txt
Test$
second line$

>
> What I want to do is something like this:
>
> \set stext `cat file.txt`
> insert into stored_files (file_name,file_text) values ('file',:'stext');
>
> without losing a character within the variable stext.
>
> In table stored_files, file_name is text and file_text is bytea.
>
> I'm using psql 9.3.10 on Ubuntu 14 Linux (32 bit.)
> I cannot use pg_read_file because that is for the server side, and I'm on the client side.
>
> There should be an easy way to get a text file into a database column.
>
> Thank you,
> Arthur Lewis
>
>


-- 
Adrian Klaver
adrian.klaver@aklaver.com

Re: insert a text file into a variable in order to insert into a bytea column

From

Arthur Lewis

Date:

28 January 2016, 21:46:24

Hi,
In my shell I did the following

$ cat file > file.temp
$ ls -l file file.temp 
-rw-rw-r-- 1 alewis alewis 53604 Jan 27 08:30 file
-rw-rw-r-- 1 alewis alewis 53604 Jan 28 10:28 file.temp
$ diff file file.temp 
$

So at least in the shell the cat command isn't losing a byte at the end of the file like in psql \set x `cat file`.

Arthur Lewis

----- Original Message -----
From: "Adrian Klaver" <adrian.klaver@aklaver.com>
To: "Arthur Lewis" <arthur.lewis@hypermediasystems.com>, pgsql-sql@postgresql.org
Sent: Thursday, January 28, 2016 10:20:43 AM
Subject: Re: [SQL] insert a text file into a variable in order to insert into a bytea column

On 01/28/2016 10:10 AM, Arthur Lewis wrote:
> Hello,
> I have a file called file.txt that is 53604 bytes.  When I do the following
>
> \set stext `cat file.txt`
>
> select length(:'stext'); length
> --------
>    53603
>
> it seems that one byte (the last newline character) is chomped off.  How can I import a text file into a variable and
insertit into a bytea column without losing the last newline byte.
 


Are sure it is not cat that is chomping the byte?:

aklaver@killi:~> cat test.txt
Test
second line

aklaver@killi:~> cat -A test.txt
Test$
second line$

>
> What I want to do is something like this:
>
> \set stext `cat file.txt`
> insert into stored_files (file_name,file_text) values ('file',:'stext');
>
> without losing a character within the variable stext.
>
> In table stored_files, file_name is text and file_text is bytea.
>
> I'm using psql 9.3.10 on Ubuntu 14 Linux (32 bit.)
> I cannot use pg_read_file because that is for the server side, and I'm on the client side.
>
> There should be an easy way to get a text file into a database column.
>
> Thank you,
> Arthur Lewis
>
>


-- 
Adrian Klaver
adrian.klaver@aklaver.com

Re: insert a text file into a variable in order to insert into a bytea column

From

Adrian Klaver

Date:

28 January 2016, 23:09:11

On 01/28/2016 10:46 AM, Arthur Lewis wrote:
> Hi,
> In my shell I did the following
>
> $ cat file > file.temp
> $ ls -l file file.temp
> -rw-rw-r-- 1 alewis alewis 53604 Jan 27 08:30 file
> -rw-rw-r-- 1 alewis alewis 53604 Jan 28 10:28 file.temp
> $ diff file file.temp
> $
>
> So at least in the shell the cat command isn't losing a byte at the end of the file like in psql \set x `cat file`.

Hmm, well there went that theory;)

What happens if you do?:

select octet_length(:'stext');

>
> Arthur Lewis
>


-- 
Adrian Klaver
adrian.klaver@aklaver.com

Re: insert a text file into a variable in order to insert into a bytea column

From

Arthur Lewis

Date:

29 January 2016, 00:08:22

postgres=> \set stext `cat file.txt`
postgres=> select octet_length(:'stext');octet_length 
--------------       53603

in my shell I did the following:
$ ls -l file.txt
-rw-rw-r-- 1 alewis alewis 53604 Jan 27 08:30 file.txt

I'm still one byte short.  So the problem has to be either the \set command chomping it or cat.  I just tested cat and
thatseems to be working.
 


----- Original Message -----
From: "Adrian Klaver" <adrian.klaver@aklaver.com>
To: "Arthur Lewis" <arthur.lewis@hypermediasystems.com>
Cc: pgsql-sql@postgresql.org
Sent: Thursday, January 28, 2016 12:08:03 PM
Subject: Re: [SQL] insert a text file into a variable in order to insert into a bytea column

On 01/28/2016 10:46 AM, Arthur Lewis wrote:
> Hi,
> In my shell I did the following
>
> $ cat file > file.temp
> $ ls -l file file.temp
> -rw-rw-r-- 1 alewis alewis 53604 Jan 27 08:30 file
> -rw-rw-r-- 1 alewis alewis 53604 Jan 28 10:28 file.temp
> $ diff file file.temp
> $
>
> So at least in the shell the cat command isn't losing a byte at the end of the file like in psql \set x `cat file`.

Hmm, well there went that theory;)

What happens if you do?:

select octet_length(:'stext');

>
> Arthur Lewis
>


-- 
Adrian Klaver
adrian.klaver@aklaver.com

Re: insert a text file into a variable in order to insert into a bytea column

From

"David G. Johnston"

Date:

29 January 2016, 00:57:44

On Thu, Jan 28, 2016 at 2:08 PM, Arthur Lewis <arthur.lewis@hypermediasystems.com> wrote:

postgres=> \set stext `cat file.txt`
postgres=> select octet_length(:'stext');
octet_length
--------------
53603

in my shell I did the following:
$ ls -l file.txt
-rw-rw-r-- 1 alewis alewis 53604 Jan 27 08:30 file.txt

I'm still one byte short. So the problem has to be either the \set command chomping it or cat. I just tested cat and that seems to be working.

Or the "back-ticks" implementation...

David J.

Re: insert a text file into a variable in order to insert into a bytea column

From

Arthur Lewis

Date:

29 January 2016, 01:36:11

Hi David,
Yes you're correct.  The problem is caused by the back-ticks (`)  Here is what I found in the docs about back-ticks

"Within an argument, text that is enclosed in backquotes (`) is taken as a command line that is passed to the shell.
Theoutput of the command (with any trailing newline removed) replaces the backquoted text." 

Arthur Lewis



----- Original Message -----
From: "David G. Johnston" <david.g.johnston@gmail.com>
To: "Arthur Lewis" <arthur.lewis@hypermediasystems.com>
Cc: "Adrian Klaver" <adrian.klaver@aklaver.com>, pgsql-sql@postgresql.org
Sent: Thursday, January 28, 2016 1:57:38 PM
Subject: Re: [SQL] insert a text file into a variable in order to insert into a bytea column



On Thu, Jan 28, 2016 at 2:08 PM, Arthur Lewis < arthur.lewis@hypermediasystems.com > wrote:




postgres=> \set stext `cat file.txt`
postgres=> select octet_length(:'stext');
octet_length
--------------
53603

in my shell I did the following:
$ ls -l file.txt
-rw-rw-r-- 1 alewis alewis 53604 Jan 27 08:30 file.txt

I'm still one byte short. So the problem has to be either the \set command chomping it or cat. I just tested cat and
thatseems to be working.  







Or the "back-ticks" implementation...




David J.

Re: insert a text file into a variable in order to insert into a bytea column

From

Adrian Klaver

Date:

29 January 2016, 02:02:51

On 01/28/2016 02:36 PM, Arthur Lewis wrote:
> Hi David,
> Yes you're correct.  The problem is caused by the back-ticks (`)  Here is what I found in the docs about back-ticks
>
> "Within an argument, text that is enclosed in backquotes (`) is taken as a command line that is passed to the shell.
Theoutput of the command (with any trailing newline removed) replaces the backquoted text."
 

Huh, I missed that in reading the docs.

So:

test-# \set stext `cat test.txt`
test-# \echo :stext 

Test 

second line 

test-# \set stext `cat test.txt` '\n' 

test-# \echo :stext 

Test 

second line 


test-#

New lesson learned.

>
> Arthur Lewis
>



-- 
Adrian Klaver
adrian.klaver@aklaver.com