Thread: no way in LargeObject API to detect short read?
Paul M. Aoki (aoki@acm.org) reports a bug with a severity of 3 The lower the number the more severe it is. Short Description no way in LargeObject API to detect short read? Long Description org.postgresql.largeobject.LargeObject.read(byte[],int,int) returns void instead of int. there's therefore no reliable,non-exceptional way to detect short reads. the natural way to write blocked-read loops would be to assume that read(byte[],int,int) returned the number of bytes reador (e.g.) -1 on EOF. Sample Code No file was uploaded with this report
Anyone able to fix this? > Paul M. Aoki (aoki@acm.org) reports a bug with a severity of 3 > The lower the number the more severe it is. > > Short Description no way in LargeObject API to detect short > read? > > Long Description > org.postgresql.largeobject.LargeObject.read(byte[],int,int) > returns void instead of int. there's therefore no reliable, > non-exceptional way to detect short reads. > > the natural way to write blocked-read loops would be to assume > that read(byte[],int,int) returned the number of bytes read or > (e.g.) -1 on EOF. > > Sample Code > > > No file was uploaded with this report > > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
Paul, I need a context diff. Thanks.
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > Anyone able to fix this?
>
> here's a hack we've been using in-house (written by Jun Gabayan,
> <jgabayan@pahv.xerox.com>).
>
> you may not like the style but it's a stab at a solution.
> --
> Paul M. Aoki / Xerox Palo Alto Research Center / 3333 Coyote Hill Road
> aoki@acm.org / Computer Science Laboratory / Palo Alto, CA 94304-1314
>
> Index: LargeObject.java
> ===================================================================
> RCS file: /project/placeless/cvsroot/placeless2/src/org/postgresql/largeobject/LargeObject.java,v
> retrieving revision 1.1
> retrieving revision 1.3
> diff -r1.1 -r1.3
> 64c64,67
> <
> ---
> >
> > private int pos = 0; //current position
> > private int size = 0;
> >
> 85a89,90
> > pos = tell();
> > size = size();
> 102a108
> > if(fd == 0) return;
> 105a112
> > fd = 0;
> 118a126,132
> > // calculate available data to read to avoid reading pass the end
> > // to avoid an exception
> > pos = tell();
> > int avail = size - pos;
> > if(avail == 0) return null;
> > if(avail < len) len = avail;
> > try {
> 123c137,141
> <
> ---
> > }catch(SQLException se) {
> > System.out.println("***LargeObject.read: Caught SQLException: " + se.getMessage());
> > return null;
> > }
> >
> 157c175
> < public void read(byte buf[],int off,int len) throws SQLException
> ---
> > public int read(byte buf[],int off,int len) throws SQLException
> 159c177,180
> < System.arraycopy(read(len),0,buf,off,len);
> ---
> > byte mybuf[] = read(len);
> > int sz = (mybuf != null) ? mybuf.length : -1; //must return -1 for end of data
> > if(sz > 0) System.arraycopy(mybuf,0,buf,off,sz);
> > return sz;
>
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026
Quoting pgsql-bugs@postgresql.org: > Paul M. Aoki (aoki@acm.org) reports a bug with a severity of 3 > The lower the number the more severe it is. > > Short Description > no way in LargeObject API to detect short read? Will check to see how libpq (which this API is based on) handles this. I've got on my list of Saturday items the stream support, which will need this. Peter > > Long Description > org.postgresql.largeobject.LargeObject.read(byte[],int,int) returns void > instead of int. there's therefore no reliable, non-exceptional way to > detect short reads. > > the natural way to write blocked-read loops would be to assume that > read(byte[],int,int) returned the number of bytes read or (e.g.) -1 on > EOF. > > Sample Code > > > No file was uploaded with this report > > > -- Peter Mount peter@retep.org.uk PostgreSQL JDBC Driver: http://www.retep.org.uk/postgres/ RetepPDF PDF library for Java: http://www.retep.org.uk/pdf/
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Anyone able to fix this?
here's a hack we've been using in-house (written by Jun Gabayan,
<jgabayan@pahv.xerox.com>).
you may not like the style but it's a stab at a solution.
--
Paul M. Aoki / Xerox Palo Alto Research Center / 3333 Coyote Hill Road
aoki@acm.org / Computer Science Laboratory / Palo Alto, CA 94304-1314
Index: LargeObject.java
===================================================================
RCS file: /project/placeless/cvsroot/placeless2/src/org/postgresql/largeobject/LargeObject.java,v
retrieving revision 1.1
retrieving revision 1.3
diff -r1.1 -r1.3
64c64,67
<
---
>
> private int pos = 0; //current position
> private int size = 0;
>
85a89,90
> pos = tell();
> size = size();
102a108
> if(fd == 0) return;
105a112
> fd = 0;
118a126,132
> // calculate available data to read to avoid reading pass the end
> // to avoid an exception
> pos = tell();
> int avail = size - pos;
> if(avail == 0) return null;
> if(avail < len) len = avail;
> try {
123c137,141
<
---
> }catch(SQLException se) {
> System.out.println("***LargeObject.read: Caught SQLException: " + se.getMessage());
> return null;
> }
>
157c175
< public void read(byte buf[],int off,int len) throws SQLException
---
> public int read(byte buf[],int off,int len) throws SQLException
159c177,180
< System.arraycopy(read(len),0,buf,off,len);
---
> byte mybuf[] = read(len);
> int sz = (mybuf != null) ? mybuf.length : -1; //must return -1 for end of data
> if(sz > 0) System.arraycopy(mybuf,0,buf,off,sz);
> return sz;
Hmmm, what's the performance issues with this? Is there going to be a problem
with very large LargeObject's?
Quoting "Paul M. Aoki" <aoki@acm.org>:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > Anyone able to fix this?
>
> here's a hack we've been using in-house (written by Jun Gabayan,
> <jgabayan@pahv.xerox.com>).
>
> you may not like the style but it's a stab at a solution.
> --
> Paul M. Aoki / Xerox Palo Alto Research Center / 3333 Coyote Hill Road
> aoki@acm.org / Computer Science Laboratory / Palo Alto, CA
> 94304-1314
>
> Index: LargeObject.java
> ===================================================================
> RCS file:
> /project/placeless/cvsroot/placeless2/src/org/postgresql/largeobject/LargeObje
ct.java,v
> retrieving revision 1.1
> retrieving revision 1.3
> diff -r1.1 -r1.3
> 64c64,67
> <
> ---
> >
> > private int pos = 0; //current position
> > private int size = 0;
> >
> 85a89,90
> > pos = tell();
> > size = size();
> 102a108
> > if(fd == 0) return;
> 105a112
> > fd = 0;
> 118a126,132
> > // calculate available data to read to avoid reading pass the
> end
> > // to avoid an exception
> > pos = tell();
> > int avail = size - pos;
> > if(avail == 0) return null;
> > if(avail < len) len = avail;
> > try {
> 123c137,141
> <
> ---
> > }catch(SQLException se) {
> > System.out.println("***LargeObject.read: Caught
> SQLException: " + se.getMessage());
> > return null;
> > }
> >
> 157c175
> < public void read(byte buf[],int off,int len) throws SQLException
> ---
> > public int read(byte buf[],int off,int len) throws SQLException
> 159c177,180
> < System.arraycopy(read(len),0,buf,off,len);
> ---
> > byte mybuf[] = read(len);
> > int sz = (mybuf != null) ? mybuf.length : -1; //must return -1 for
> end of data
> > if(sz > 0) System.arraycopy(mybuf,0,buf,off,sz);
> > return sz;
>
--
Peter Mount peter@retep.org.uk
PostgreSQL JDBC Driver: http://www.retep.org.uk/postgres/
RetepPDF PDF library for Java: http://www.retep.org.uk/pdf/
Re: [JDBC] Re: no way in LargeObject API to detect short read?
From
aoki@acm.org (Paul M. Aoki)
Date:
Peter T Mount <peter@retep.org.uk> writes: > Hmmm, what's the performance issues with this? Is there going to be a problem > with very large LargeObject's? you could probably be smarter about caching previous tell() state, if that's what you mean. jun's hack doesn't actually add any extra buffers or copies. -- Paul M. Aoki / Xerox Palo Alto Research Center / 3333 Coyote Hill Road aoki@acm.org / Computer Science Laboratory / Palo Alto, CA 94304-1314
At 01:58 14/11/00 -0500, pgsql-bugs@postgresql.org wrote: >Paul M. Aoki (aoki@acm.org) reports a bug with a severity of 3 >The lower the number the more severe it is. > >Short Description >no way in LargeObject API to detect short read? > >Long Description >org.postgresql.largeobject.LargeObject.read(byte[],int,int) returns void >instead of int. there's therefore no reliable, non-exceptional way to >detect short reads. > >the natural way to write blocked-read loops would be to assume that >read(byte[],int,int) returned the number of bytes read or (e.g.) -1 on EOF. > >Sample Code > > >No file was uploaded with this report This method now returns the number of bytes read. It will be in CVS by 0900GMT tomorrow. Peter
This is fixed in the current 7.1 snapshot. > Paul M. Aoki (aoki@acm.org) reports a bug with a severity of 3 > The lower the number the more severe it is. > > Short Description > no way in LargeObject API to detect short read? > > Long Description > org.postgresql.largeobject.LargeObject.read(byte[],int,int) returns void instead of int. there's therefore no reliable,non-exceptional way to detect short reads. > > the natural way to write blocked-read loops would be to assume that read(byte[],int,int) returned the number of bytes reador (e.g.) -1 on EOF. > > Sample Code > > > No file was uploaded with this report > > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026