Thread: problem with LargeObject and commit

problem with LargeObject and commit

From
Marc Cousin
Date:
Hi,

I'm having the following problem: I need to stream a LOB to a web client, without consuming
a lot of memory. So I am returning getInputStream() to the caller.

This works perfectly, except that I have to keep a transaction open to return the large object
stream, and I have (or found) no way of closing the transaction when the filestream is closed.

So of course, if I did that, I would get a lot of IDLE in transaction sessions...

So the logical way seemed to me to extend LargeObjectManager and LargeObject to have a
close method on my LargeObject that closes the transaction, meaning that my LargeObject
should have a connection attribute. So I wanted to do something like this (not working,
it's just for the sake of explanation):


public class StreamLargeObjectManager extends LargeObjectManager{

           public StreamLargeObject openStreamLargeObject(long oid, int mode)
             throws SQLException
           {
             LargeObject lo = super.open(oid, mode);
             StreamLargeObject so = new StreamLargeObject(lo, super.getConn()); //cannot be done because no access to
connection
             return so;
           }
}


and

public class StreamLargeObject extends org.postgresql.largeobject.LargeObject{


        Connection conn;

        LargeObject lobj;


        public StreamLargeObject(LargeObject lg, BaseConnection conn) throws SQLException {
                this.lobj = lg;
                this.conn = conn;
        }


        @Override
        public void close() throws SQLException {

                lobj.close();
                this.conn.commit();

        }


       public getInputStream() throws SQLException
    {
        return lobj.getInputStream();
    }
}


As the comment says it all, I cannot do this: conn is private in LargeObjectManager. For now, the problem has been
worked around by duplicating all the LargeObjectManager code, but this is obviously ugly and not
maintainable. So my question is: have I missed some obvious solution ? Could a getter be added to
the LargeObjectManager's connection ?

Regards,

Marc


Re: problem with LargeObject and commit

From
Dave Cramer
Date:
Marc,

You required the connection to create the LargeObjectManager, so why not just extend the InputStream and add the connection to that object.
When you close the input stream, close the connection ????

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca


On Mon, Sep 30, 2013 at 11:18 AM, Marc Cousin <cousinmarc@gmail.com> wrote:
Hi,

I'm having the following problem: I need to stream a LOB to a web client, without consuming
a lot of memory. So I am returning getInputStream() to the caller.

This works perfectly, except that I have to keep a transaction open to return the large object
stream, and I have (or found) no way of closing the transaction when the filestream is closed.

So of course, if I did that, I would get a lot of IDLE in transaction sessions...

So the logical way seemed to me to extend LargeObjectManager and LargeObject to have a
close method on my LargeObject that closes the transaction, meaning that my LargeObject
should have a connection attribute. So I wanted to do something like this (not working,
it's just for the sake of explanation):


public class StreamLargeObjectManager extends LargeObjectManager{

           public StreamLargeObject openStreamLargeObject(long oid, int mode)
             throws SQLException
           {
             LargeObject lo = super.open(oid, mode);
             StreamLargeObject so = new StreamLargeObject(lo, super.getConn()); //cannot be done because no access to connection
             return so;
           }
}


and

public class StreamLargeObject extends org.postgresql.largeobject.LargeObject{


        Connection conn;

        LargeObject lobj;


        public StreamLargeObject(LargeObject lg, BaseConnection conn) throws SQLException {
                this.lobj = lg;
                this.conn = conn;
        }


        @Override
        public void close() throws SQLException {

                lobj.close();
                this.conn.commit();

        }


       public getInputStream() throws SQLException
    {
        return lobj.getInputStream();
    }
}


As the comment says it all, I cannot do this: conn is private in LargeObjectManager. For now, the problem has been
worked around by duplicating all the LargeObjectManager code, but this is obviously ugly and not
maintainable. So my question is: have I missed some obvious solution ? Could a getter be added to
the LargeObjectManager's connection ?

Regards,

Marc


--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

Re: problem with LargeObject and commit

From
Marc Cousin
Date:
Sorry, I don't think I get it, so please correct me below if I didn't.

I currently use getInputStream() on my large object. How could I provide a
connection to the extended InputStream ? I am calling a LargeObject method.

Or maybe extend LargeObject to have a new getInputStream method with a
connection provided, and then provide this connection to the extended
InputStream ?

My initial idea was to not have to provide the connection as a parameter, so I
wanted to use the Lob Manager.

Anyway, shouldn't streaming a large object in this special case be easier to
do ? The original code (from SQL Server) didn't have to do all these
workarounds, and I'll get this objection from people that will have to use
this code :)

Just having conn as protected instead of private in the LargeObjectManager
would have made it possible to me to do it simply: I could have provided the
connection to my extended LargeObject, and from there end the transaction on
close().

Regards.

On Monday 30 September 2013 17:08:18 Dave Cramer wrote:
> Marc,
>
> You required the connection to create the LargeObjectManager, so why not
> just extend the InputStream and add the connection to that object.
> When you close the input stream, close the connection ????
>
> Dave Cramer
>
> dave.cramer(at)credativ(dot)ca
> http://www.credativ.ca
>
> On Mon, Sep 30, 2013 at 11:18 AM, Marc Cousin <cousinmarc@gmail.com> wrote:
> > Hi,
> >
> > I'm having the following problem: I need to stream a LOB to a web client,
> > without consuming
> > a lot of memory. So I am returning getInputStream() to the caller.
> >
> > This works perfectly, except that I have to keep a transaction open to
> > return the large object
> > stream, and I have (or found) no way of closing the transaction when the
> > filestream is closed.
> >
> > So of course, if I did that, I would get a lot of IDLE in transaction
> > sessions...
> >
> > So the logical way seemed to me to extend LargeObjectManager and
> > LargeObject to have a
> > close method on my LargeObject that closes the transaction, meaning that
> > my LargeObject
> > should have a connection attribute. So I wanted to do something like this
> > (not working,
> > it's just for the sake of explanation):
> >
> >
> > public class StreamLargeObjectManager extends LargeObjectManager{
> >
> >            public StreamLargeObject openStreamLargeObject(long oid, int
> >
> > mode)
> >
> >              throws SQLException
> >
> >            {
> >
> >              LargeObject lo = super.open(oid, mode);
> >              StreamLargeObject so = new StreamLargeObject(lo,
> >
> > super.getConn()); //cannot be done because no access to connection
> >
> >              return so;
> >
> >            }
> >
> > }
> >
> >
> > and
> >
> > public class StreamLargeObject extends
> > org.postgresql.largeobject.LargeObject{
> >
> >         Connection conn;
> >
> >         LargeObject lobj;
> >
> >
> >         public StreamLargeObject(LargeObject lg, BaseConnection conn)
> >
> > throws SQLException {
> >
> >                 this.lobj = lg;
> >                 this.conn = conn;
> >
> >         }
> >
> >
> >         @Override
> >         public void close() throws SQLException {
> >
> >                 lobj.close();
> >                 this.conn.commit();
> >
> >         }
> >
> >        public getInputStream() throws SQLException
> >
> >     {
> >
> >         return lobj.getInputStream();
> >
> >     }
> >
> > }
> >
> >
> > As the comment says it all, I cannot do this: conn is private in
> > LargeObjectManager. For now, the problem has been
> > worked around by duplicating all the LargeObjectManager code, but this is
> > obviously ugly and not
> > maintainable. So my question is: have I missed some obvious solution ?
> > Could a getter be added to
> > the LargeObjectManager's connection ?
> >
> > Regards,
> >
> > Marc
> >
> >
> > --
> > Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
> > To make changes to your subscription:
> > http://www.postgresql.org/mailpref/pgsql-jdbc



Re: problem with LargeObject and commit

From
Dave Cramer
Date:
Well I don't see any reason not to open up the LOM class to allow you to extend it.

Can you send a git pull?

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca


On Tue, Oct 1, 2013 at 3:52 AM, Marc Cousin <cousinmarc@gmail.com> wrote:
Sorry, I don't think I get it, so please correct me below if I didn't.

I currently use getInputStream() on my large object. How could I provide a
connection to the extended InputStream ? I am calling a LargeObject method.

Or maybe extend LargeObject to have a new getInputStream method with a
connection provided, and then provide this connection to the extended
InputStream ?

My initial idea was to not have to provide the connection as a parameter, so I
wanted to use the Lob Manager.

Anyway, shouldn't streaming a large object in this special case be easier to
do ? The original code (from SQL Server) didn't have to do all these
workarounds, and I'll get this objection from people that will have to use
this code :)

Just having conn as protected instead of private in the LargeObjectManager
would have made it possible to me to do it simply: I could have provided the
connection to my extended LargeObject, and from there end the transaction on
close().

Regards.

On Monday 30 September 2013 17:08:18 Dave Cramer wrote:
> Marc,
>
> You required the connection to create the LargeObjectManager, so why not
> just extend the InputStream and add the connection to that object.
> When you close the input stream, close the connection ????
>
> Dave Cramer
>
> dave.cramer(at)credativ(dot)ca
> http://www.credativ.ca
>
> On Mon, Sep 30, 2013 at 11:18 AM, Marc Cousin <cousinmarc@gmail.com> wrote:
> > Hi,
> >
> > I'm having the following problem: I need to stream a LOB to a web client,
> > without consuming
> > a lot of memory. So I am returning getInputStream() to the caller.
> >
> > This works perfectly, except that I have to keep a transaction open to
> > return the large object
> > stream, and I have (or found) no way of closing the transaction when the
> > filestream is closed.
> >
> > So of course, if I did that, I would get a lot of IDLE in transaction
> > sessions...
> >
> > So the logical way seemed to me to extend LargeObjectManager and
> > LargeObject to have a
> > close method on my LargeObject that closes the transaction, meaning that
> > my LargeObject
> > should have a connection attribute. So I wanted to do something like this
> > (not working,
> > it's just for the sake of explanation):
> >
> >
> > public class StreamLargeObjectManager extends LargeObjectManager{
> >
> >            public StreamLargeObject openStreamLargeObject(long oid, int
> >
> > mode)
> >
> >              throws SQLException
> >
> >            {
> >
> >              LargeObject lo = super.open(oid, mode);
> >              StreamLargeObject so = new StreamLargeObject(lo,
> >
> > super.getConn()); //cannot be done because no access to connection
> >
> >              return so;
> >
> >            }
> >
> > }
> >
> >
> > and
> >
> > public class StreamLargeObject extends
> > org.postgresql.largeobject.LargeObject{
> >
> >         Connection conn;
> >
> >         LargeObject lobj;
> >
> >
> >         public StreamLargeObject(LargeObject lg, BaseConnection conn)
> >
> > throws SQLException {
> >
> >                 this.lobj = lg;
> >                 this.conn = conn;
> >
> >         }
> >
> >
> >         @Override
> >         public void close() throws SQLException {
> >
> >                 lobj.close();
> >                 this.conn.commit();
> >
> >         }
> >
> >        public getInputStream() throws SQLException
> >
> >     {
> >
> >         return lobj.getInputStream();
> >
> >     }
> >
> > }
> >
> >
> > As the comment says it all, I cannot do this: conn is private in
> > LargeObjectManager. For now, the problem has been
> > worked around by duplicating all the LargeObjectManager code, but this is
> > obviously ugly and not
> > maintainable. So my question is: have I missed some obvious solution ?
> > Could a getter be added to
> > the LargeObjectManager's connection ?
> >
> > Regards,
> >
> > Marc
> >
> >
> > --
> > Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
> > To make changes to your subscription:
> > http://www.postgresql.org/mailpref/pgsql-jdbc


Re: problem with LargeObject and commit

From
Marc Cousin
Date:
Hi,

Attached is an attempt to do this (git diff on head).

The code seems to work. It has been tested on the program we are migrating
from SQL Server, we can now stream LOBs directly to web browsers...

Please note that I am not usually a Java developper. I did my best to do this
correctly, and the Java way (as far as I know :) ). If some things are not
done the right way, or there is a smarter solution, tell me, I'll improve
this.

Regards,

Marc

On Tuesday 01 October 2013 08:11:21 Dave Cramer wrote:
> Well I don't see any reason not to open up the LOM class to allow you to
> extend it.
>
> Can you send a git pull?
>
> Dave Cramer
>
> dave.cramer(at)credativ(dot)ca
> http://www.credativ.ca
>
> On Tue, Oct 1, 2013 at 3:52 AM, Marc Cousin <cousinmarc@gmail.com> wrote:
> > Sorry, I don't think I get it, so please correct me below if I didn't.
> >
> > I currently use getInputStream() on my large object. How could I provide a
> > connection to the extended InputStream ? I am calling a LargeObject
> > method.
> >
> > Or maybe extend LargeObject to have a new getInputStream method with a
> > connection provided, and then provide this connection to the extended
> > InputStream ?
> >
> > My initial idea was to not have to provide the connection as a parameter,
> > so I
> > wanted to use the Lob Manager.
> >
> > Anyway, shouldn't streaming a large object in this special case be easier
> > to
> > do ? The original code (from SQL Server) didn't have to do all these
> > workarounds, and I'll get this objection from people that will have to use
> > this code :)
> >
> > Just having conn as protected instead of private in the LargeObjectManager
> > would have made it possible to me to do it simply: I could have provided
> > the
> > connection to my extended LargeObject, and from there end the transaction
> > on
> > close().
> >
> > Regards.
> >
> > On Monday 30 September 2013 17:08:18 Dave Cramer wrote:
> > > Marc,
> > >
> > > You required the connection to create the LargeObjectManager, so why not
> > > just extend the InputStream and add the connection to that object.
> > > When you close the input stream, close the connection ????
> > >
> > > Dave Cramer
> > >
> > > dave.cramer(at)credativ(dot)ca
> > > http://www.credativ.ca
> > >
> > > On Mon, Sep 30, 2013 at 11:18 AM, Marc Cousin <cousinmarc@gmail.com>
> >
> > wrote:
> > > > Hi,
> > > >
> > > > I'm having the following problem: I need to stream a LOB to a web
> >
> > client,
> >
> > > > without consuming
> > > > a lot of memory. So I am returning getInputStream() to the caller.
> > > >
> > > > This works perfectly, except that I have to keep a transaction open to
> > > > return the large object
> > > > stream, and I have (or found) no way of closing the transaction when
> >
> > the
> >
> > > > filestream is closed.
> > > >
> > > > So of course, if I did that, I would get a lot of IDLE in transaction
> > > > sessions...
> > > >
> > > > So the logical way seemed to me to extend LargeObjectManager and
> > > > LargeObject to have a
> > > > close method on my LargeObject that closes the transaction, meaning
> >
> > that
> >
> > > > my LargeObject
> > > > should have a connection attribute. So I wanted to do something like
> >
> > this
> >
> > > > (not working,
> > > > it's just for the sake of explanation):
> > > >
> > > >
> > > > public class StreamLargeObjectManager extends LargeObjectManager{
> > > >
> > > >            public StreamLargeObject openStreamLargeObject(long oid,
> > > >            int
> > > >
> > > > mode)
> > > >
> > > >              throws SQLException
> > > >
> > > >            {
> > > >
> > > >              LargeObject lo = super.open(oid, mode);
> > > >              StreamLargeObject so = new StreamLargeObject(lo,
> > > >
> > > > super.getConn()); //cannot be done because no access to connection
> > > >
> > > >              return so;
> > > >
> > > >            }
> > > >
> > > > }
> > > >
> > > >
> > > > and
> > > >
> > > > public class StreamLargeObject extends
> > > > org.postgresql.largeobject.LargeObject{
> > > >
> > > >         Connection conn;
> > > >
> > > >         LargeObject lobj;
> > > >
> > > >
> > > >         public StreamLargeObject(LargeObject lg, BaseConnection conn)
> > > >
> > > > throws SQLException {
> > > >
> > > >                 this.lobj = lg;
> > > >                 this.conn = conn;
> > > >
> > > >         }
> > > >
> > > >
> > > >         @Override
> > > >         public void close() throws SQLException {
> > > >
> > > >                 lobj.close();
> > > >                 this.conn.commit();
> > > >
> > > >         }
> > > >
> > > >        public getInputStream() throws SQLException
> > > >
> > > >     {
> > > >
> > > >         return lobj.getInputStream();
> > > >
> > > >     }
> > > >
> > > > }
> > > >
> > > >
> > > > As the comment says it all, I cannot do this: conn is private in
> > > > LargeObjectManager. For now, the problem has been
> > > > worked around by duplicating all the LargeObjectManager code, but this
> >
> > is
> >
> > > > obviously ugly and not
> > > > maintainable. So my question is: have I missed some obvious solution ?
> > > > Could a getter be added to
> > > > the LargeObjectManager's connection ?
> > > >
> > > > Regards,
> > > >
> > > > Marc
> > > >
> > > >
> > > > --
> > > > Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
> > > > To make changes to your subscription:
> > > > http://www.postgresql.org/mailpref/pgsql-jdbc

Attachment

Re: problem with LargeObject and commit

From
Marc Cousin
Date:
Hi,

Did you give it a look ?

Regards

Marc

On Sunday 06 October 2013 19:14:41 Marc Cousin wrote:
> Hi,
>
> Attached is an attempt to do this (git diff on head).
>
> The code seems to work. It has been tested on the program we are migrating
> from SQL Server, we can now stream LOBs directly to web browsers...
>
> Please note that I am not usually a Java developper. I did my best to do
> this correctly, and the Java way (as far as I know :) ). If some things are
> not done the right way, or there is a smarter solution, tell me, I'll
> improve this.
>
> Regards,
>
> Marc
>
> On Tuesday 01 October 2013 08:11:21 Dave Cramer wrote:
> > Well I don't see any reason not to open up the LOM class to allow you to
> > extend it.
> >
> > Can you send a git pull?
> >
> > Dave Cramer
> >
> > dave.cramer(at)credativ(dot)ca
> > http://www.credativ.ca
> >
> > On Tue, Oct 1, 2013 at 3:52 AM, Marc Cousin <cousinmarc@gmail.com> wrote:
> > > Sorry, I don't think I get it, so please correct me below if I didn't.
> > >
> > > I currently use getInputStream() on my large object. How could I provide
> > > a
> > > connection to the extended InputStream ? I am calling a LargeObject
> > > method.
> > >
> > > Or maybe extend LargeObject to have a new getInputStream method with a
> > > connection provided, and then provide this connection to the extended
> > > InputStream ?
> > >
> > > My initial idea was to not have to provide the connection as a
> > > parameter,
> > > so I
> > > wanted to use the Lob Manager.
> > >
> > > Anyway, shouldn't streaming a large object in this special case be
> > > easier
> > > to
> > > do ? The original code (from SQL Server) didn't have to do all these
> > > workarounds, and I'll get this objection from people that will have to
> > > use
> > > this code :)
> > >
> > > Just having conn as protected instead of private in the
> > > LargeObjectManager
> > > would have made it possible to me to do it simply: I could have provided
> > > the
> > > connection to my extended LargeObject, and from there end the
> > > transaction
> > > on
> > > close().
> > >
> > > Regards.
> > >
> > > On Monday 30 September 2013 17:08:18 Dave Cramer wrote:
> > > > Marc,
> > > >
> > > > You required the connection to create the LargeObjectManager, so why
> > > > not
> > > > just extend the InputStream and add the connection to that object.
> > > > When you close the input stream, close the connection ????
> > > >
> > > > Dave Cramer
> > > >
> > > > dave.cramer(at)credativ(dot)ca
> > > > http://www.credativ.ca
> > > >
> > > > On Mon, Sep 30, 2013 at 11:18 AM, Marc Cousin <cousinmarc@gmail.com>
> > >
> > > wrote:
> > > > > Hi,
> > > > >
> > > > > I'm having the following problem: I need to stream a LOB to a web
> > >
> > > client,
> > >
> > > > > without consuming
> > > > > a lot of memory. So I am returning getInputStream() to the caller.
> > > > >
> > > > > This works perfectly, except that I have to keep a transaction open
> > > > > to
> > > > > return the large object
> > > > > stream, and I have (or found) no way of closing the transaction when
> > >
> > > the
> > >
> > > > > filestream is closed.
> > > > >
> > > > > So of course, if I did that, I would get a lot of IDLE in
> > > > > transaction
> > > > > sessions...
> > > > >
> > > > > So the logical way seemed to me to extend LargeObjectManager and
> > > > > LargeObject to have a
> > > > > close method on my LargeObject that closes the transaction, meaning
> > >
> > > that
> > >
> > > > > my LargeObject
> > > > > should have a connection attribute. So I wanted to do something like
> > >
> > > this
> > >
> > > > > (not working,
> > > > > it's just for the sake of explanation):
> > > > >
> > > > >
> > > > > public class StreamLargeObjectManager extends LargeObjectManager{
> > > > >
> > > > >            public StreamLargeObject openStreamLargeObject(long oid,
> > > > >            int
> > > > >
> > > > > mode)
> > > > >
> > > > >              throws SQLException
> > > > >
> > > > >            {
> > > > >
> > > > >              LargeObject lo = super.open(oid, mode);
> > > > >              StreamLargeObject so = new StreamLargeObject(lo,
> > > > >
> > > > > super.getConn()); //cannot be done because no access to connection
> > > > >
> > > > >              return so;
> > > > >
> > > > >            }
> > > > >
> > > > > }
> > > > >
> > > > >
> > > > > and
> > > > >
> > > > > public class StreamLargeObject extends
> > > > > org.postgresql.largeobject.LargeObject{
> > > > >
> > > > >         Connection conn;
> > > > >
> > > > >         LargeObject lobj;
> > > > >
> > > > >
> > > > >         public StreamLargeObject(LargeObject lg, BaseConnection
> > > > >         conn)
> > > > >
> > > > > throws SQLException {
> > > > >
> > > > >                 this.lobj = lg;
> > > > >                 this.conn = conn;
> > > > >
> > > > >         }
> > > > >
> > > > >
> > > > >         @Override
> > > > >         public void close() throws SQLException {
> > > > >
> > > > >                 lobj.close();
> > > > >                 this.conn.commit();
> > > > >
> > > > >         }
> > > > >
> > > > >        public getInputStream() throws SQLException
> > > > >
> > > > >     {
> > > > >
> > > > >         return lobj.getInputStream();
> > > > >
> > > > >     }
> > > > >
> > > > > }
> > > > >
> > > > >
> > > > > As the comment says it all, I cannot do this: conn is private in
> > > > > LargeObjectManager. For now, the problem has been
> > > > > worked around by duplicating all the LargeObjectManager code, but
> > > > > this
> > >
> > > is
> > >
> > > > > obviously ugly and not
> > > > > maintainable. So my question is: have I missed some obvious solution
> > > > > ?
> > > > > Could a getter be added to
> > > > > the LargeObjectManager's connection ?
> > > > >
> > > > > Regards,
> > > > >
> > > > > Marc
> > > > >
> > > > >
> > > > > --
> > > > > Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
> > > > > To make changes to your subscription:
> > > > > http://www.postgresql.org/mailpref/pgsql-jdbc



Re: problem with LargeObject and commit

From
Dave Cramer
Date:
Marc,

Sorry I have been travelling. Just looked at it, It's fine albeit more than just changing the visibility of the function.

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca


On Mon, Oct 14, 2013 at 10:38 AM, Marc Cousin <cousinmarc@gmail.com> wrote:
Hi,

Did you give it a look ?

Regards

Marc

On Sunday 06 October 2013 19:14:41 Marc Cousin wrote:
> Hi,
>
> Attached is an attempt to do this (git diff on head).
>
> The code seems to work. It has been tested on the program we are migrating
> from SQL Server, we can now stream LOBs directly to web browsers...
>
> Please note that I am not usually a Java developper. I did my best to do
> this correctly, and the Java way (as far as I know :) ). If some things are
> not done the right way, or there is a smarter solution, tell me, I'll
> improve this.
>
> Regards,
>
> Marc
>
> On Tuesday 01 October 2013 08:11:21 Dave Cramer wrote:
> > Well I don't see any reason not to open up the LOM class to allow you to
> > extend it.
> >
> > Can you send a git pull?
> >
> > Dave Cramer
> >
> > dave.cramer(at)credativ(dot)ca
> > http://www.credativ.ca
> >
> > On Tue, Oct 1, 2013 at 3:52 AM, Marc Cousin <cousinmarc@gmail.com> wrote:
> > > Sorry, I don't think I get it, so please correct me below if I didn't.
> > >
> > > I currently use getInputStream() on my large object. How could I provide
> > > a
> > > connection to the extended InputStream ? I am calling a LargeObject
> > > method.
> > >
> > > Or maybe extend LargeObject to have a new getInputStream method with a
> > > connection provided, and then provide this connection to the extended
> > > InputStream ?
> > >
> > > My initial idea was to not have to provide the connection as a
> > > parameter,
> > > so I
> > > wanted to use the Lob Manager.
> > >
> > > Anyway, shouldn't streaming a large object in this special case be
> > > easier
> > > to
> > > do ? The original code (from SQL Server) didn't have to do all these
> > > workarounds, and I'll get this objection from people that will have to
> > > use
> > > this code :)
> > >
> > > Just having conn as protected instead of private in the
> > > LargeObjectManager
> > > would have made it possible to me to do it simply: I could have provided
> > > the
> > > connection to my extended LargeObject, and from there end the
> > > transaction
> > > on
> > > close().
> > >
> > > Regards.
> > >
> > > On Monday 30 September 2013 17:08:18 Dave Cramer wrote:
> > > > Marc,
> > > >
> > > > You required the connection to create the LargeObjectManager, so why
> > > > not
> > > > just extend the InputStream and add the connection to that object.
> > > > When you close the input stream, close the connection ????
> > > >
> > > > Dave Cramer
> > > >
> > > > dave.cramer(at)credativ(dot)ca
> > > > http://www.credativ.ca
> > > >
> > > > On Mon, Sep 30, 2013 at 11:18 AM, Marc Cousin <cousinmarc@gmail.com>
> > >
> > > wrote:
> > > > > Hi,
> > > > >
> > > > > I'm having the following problem: I need to stream a LOB to a web
> > >
> > > client,
> > >
> > > > > without consuming
> > > > > a lot of memory. So I am returning getInputStream() to the caller.
> > > > >
> > > > > This works perfectly, except that I have to keep a transaction open
> > > > > to
> > > > > return the large object
> > > > > stream, and I have (or found) no way of closing the transaction when
> > >
> > > the
> > >
> > > > > filestream is closed.
> > > > >
> > > > > So of course, if I did that, I would get a lot of IDLE in
> > > > > transaction
> > > > > sessions...
> > > > >
> > > > > So the logical way seemed to me to extend LargeObjectManager and
> > > > > LargeObject to have a
> > > > > close method on my LargeObject that closes the transaction, meaning
> > >
> > > that
> > >
> > > > > my LargeObject
> > > > > should have a connection attribute. So I wanted to do something like
> > >
> > > this
> > >
> > > > > (not working,
> > > > > it's just for the sake of explanation):
> > > > >
> > > > >
> > > > > public class StreamLargeObjectManager extends LargeObjectManager{
> > > > >
> > > > >            public StreamLargeObject openStreamLargeObject(long oid,
> > > > >            int
> > > > >
> > > > > mode)
> > > > >
> > > > >              throws SQLException
> > > > >
> > > > >            {
> > > > >
> > > > >              LargeObject lo = super.open(oid, mode);
> > > > >              StreamLargeObject so = new StreamLargeObject(lo,
> > > > >
> > > > > super.getConn()); //cannot be done because no access to connection
> > > > >
> > > > >              return so;
> > > > >
> > > > >            }
> > > > >
> > > > > }
> > > > >
> > > > >
> > > > > and
> > > > >
> > > > > public class StreamLargeObject extends
> > > > > org.postgresql.largeobject.LargeObject{
> > > > >
> > > > >         Connection conn;
> > > > >
> > > > >         LargeObject lobj;
> > > > >
> > > > >
> > > > >         public StreamLargeObject(LargeObject lg, BaseConnection
> > > > >         conn)
> > > > >
> > > > > throws SQLException {
> > > > >
> > > > >                 this.lobj = lg;
> > > > >                 this.conn = conn;
> > > > >
> > > > >         }
> > > > >
> > > > >
> > > > >         @Override
> > > > >         public void close() throws SQLException {
> > > > >
> > > > >                 lobj.close();
> > > > >                 this.conn.commit();
> > > > >
> > > > >         }
> > > > >
> > > > >        public getInputStream() throws SQLException
> > > > >
> > > > >     {
> > > > >
> > > > >         return lobj.getInputStream();
> > > > >
> > > > >     }
> > > > >
> > > > > }
> > > > >
> > > > >
> > > > > As the comment says it all, I cannot do this: conn is private in
> > > > > LargeObjectManager. For now, the problem has been
> > > > > worked around by duplicating all the LargeObjectManager code, but
> > > > > this
> > >
> > > is
> > >
> > > > > obviously ugly and not
> > > > > maintainable. So my question is: have I missed some obvious solution
> > > > > ?
> > > > > Could a getter be added to
> > > > > the LargeObjectManager's connection ?
> > > > >
> > > > > Regards,
> > > > >
> > > > > Marc
> > > > >
> > > > >
> > > > > --
> > > > > Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
> > > > > To make changes to your subscription:
> > > > > http://www.postgresql.org/mailpref/pgsql-jdbc


Re: problem with LargeObject and commit

From
Dave Cramer
Date:
Marc,

I just pushed it to github, thanks!

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca


On Tue, Oct 15, 2013 at 6:48 AM, Dave Cramer <pg@fastcrypt.com> wrote:
Marc,

Sorry I have been travelling. Just looked at it, It's fine albeit more than just changing the visibility of the function.

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca


On Mon, Oct 14, 2013 at 10:38 AM, Marc Cousin <cousinmarc@gmail.com> wrote:
Hi,

Did you give it a look ?

Regards

Marc

On Sunday 06 October 2013 19:14:41 Marc Cousin wrote:
> Hi,
>
> Attached is an attempt to do this (git diff on head).
>
> The code seems to work. It has been tested on the program we are migrating
> from SQL Server, we can now stream LOBs directly to web browsers...
>
> Please note that I am not usually a Java developper. I did my best to do
> this correctly, and the Java way (as far as I know :) ). If some things are
> not done the right way, or there is a smarter solution, tell me, I'll
> improve this.
>
> Regards,
>
> Marc
>
> On Tuesday 01 October 2013 08:11:21 Dave Cramer wrote:
> > Well I don't see any reason not to open up the LOM class to allow you to
> > extend it.
> >
> > Can you send a git pull?
> >
> > Dave Cramer
> >
> > dave.cramer(at)credativ(dot)ca
> > http://www.credativ.ca
> >
> > On Tue, Oct 1, 2013 at 3:52 AM, Marc Cousin <cousinmarc@gmail.com> wrote:
> > > Sorry, I don't think I get it, so please correct me below if I didn't.
> > >
> > > I currently use getInputStream() on my large object. How could I provide
> > > a
> > > connection to the extended InputStream ? I am calling a LargeObject
> > > method.
> > >
> > > Or maybe extend LargeObject to have a new getInputStream method with a
> > > connection provided, and then provide this connection to the extended
> > > InputStream ?
> > >
> > > My initial idea was to not have to provide the connection as a
> > > parameter,
> > > so I
> > > wanted to use the Lob Manager.
> > >
> > > Anyway, shouldn't streaming a large object in this special case be
> > > easier
> > > to
> > > do ? The original code (from SQL Server) didn't have to do all these
> > > workarounds, and I'll get this objection from people that will have to
> > > use
> > > this code :)
> > >
> > > Just having conn as protected instead of private in the
> > > LargeObjectManager
> > > would have made it possible to me to do it simply: I could have provided
> > > the
> > > connection to my extended LargeObject, and from there end the
> > > transaction
> > > on
> > > close().
> > >
> > > Regards.
> > >
> > > On Monday 30 September 2013 17:08:18 Dave Cramer wrote:
> > > > Marc,
> > > >
> > > > You required the connection to create the LargeObjectManager, so why
> > > > not
> > > > just extend the InputStream and add the connection to that object.
> > > > When you close the input stream, close the connection ????
> > > >
> > > > Dave Cramer
> > > >
> > > > dave.cramer(at)credativ(dot)ca
> > > > http://www.credativ.ca
> > > >
> > > > On Mon, Sep 30, 2013 at 11:18 AM, Marc Cousin <cousinmarc@gmail.com>
> > >
> > > wrote:
> > > > > Hi,
> > > > >
> > > > > I'm having the following problem: I need to stream a LOB to a web
> > >
> > > client,
> > >
> > > > > without consuming
> > > > > a lot of memory. So I am returning getInputStream() to the caller.
> > > > >
> > > > > This works perfectly, except that I have to keep a transaction open
> > > > > to
> > > > > return the large object
> > > > > stream, and I have (or found) no way of closing the transaction when
> > >
> > > the
> > >
> > > > > filestream is closed.
> > > > >
> > > > > So of course, if I did that, I would get a lot of IDLE in
> > > > > transaction
> > > > > sessions...
> > > > >
> > > > > So the logical way seemed to me to extend LargeObjectManager and
> > > > > LargeObject to have a
> > > > > close method on my LargeObject that closes the transaction, meaning
> > >
> > > that
> > >
> > > > > my LargeObject
> > > > > should have a connection attribute. So I wanted to do something like
> > >
> > > this
> > >
> > > > > (not working,
> > > > > it's just for the sake of explanation):
> > > > >
> > > > >
> > > > > public class StreamLargeObjectManager extends LargeObjectManager{
> > > > >
> > > > >            public StreamLargeObject openStreamLargeObject(long oid,
> > > > >            int
> > > > >
> > > > > mode)
> > > > >
> > > > >              throws SQLException
> > > > >
> > > > >            {
> > > > >
> > > > >              LargeObject lo = super.open(oid, mode);
> > > > >              StreamLargeObject so = new StreamLargeObject(lo,
> > > > >
> > > > > super.getConn()); //cannot be done because no access to connection
> > > > >
> > > > >              return so;
> > > > >
> > > > >            }
> > > > >
> > > > > }
> > > > >
> > > > >
> > > > > and
> > > > >
> > > > > public class StreamLargeObject extends
> > > > > org.postgresql.largeobject.LargeObject{
> > > > >
> > > > >         Connection conn;
> > > > >
> > > > >         LargeObject lobj;
> > > > >
> > > > >
> > > > >         public StreamLargeObject(LargeObject lg, BaseConnection
> > > > >         conn)
> > > > >
> > > > > throws SQLException {
> > > > >
> > > > >                 this.lobj = lg;
> > > > >                 this.conn = conn;
> > > > >
> > > > >         }
> > > > >
> > > > >
> > > > >         @Override
> > > > >         public void close() throws SQLException {
> > > > >
> > > > >                 lobj.close();
> > > > >                 this.conn.commit();
> > > > >
> > > > >         }
> > > > >
> > > > >        public getInputStream() throws SQLException
> > > > >
> > > > >     {
> > > > >
> > > > >         return lobj.getInputStream();
> > > > >
> > > > >     }
> > > > >
> > > > > }
> > > > >
> > > > >
> > > > > As the comment says it all, I cannot do this: conn is private in
> > > > > LargeObjectManager. For now, the problem has been
> > > > > worked around by duplicating all the LargeObjectManager code, but
> > > > > this
> > >
> > > is
> > >
> > > > > obviously ugly and not
> > > > > maintainable. So my question is: have I missed some obvious solution
> > > > > ?
> > > > > Could a getter be added to
> > > > > the LargeObjectManager's connection ?
> > > > >
> > > > > Regards,
> > > > >
> > > > > Marc
> > > > >
> > > > >
> > > > > --
> > > > > Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
> > > > > To make changes to your subscription:
> > > > > http://www.postgresql.org/mailpref/pgsql-jdbc



Re: problem with LargeObject and commit

From
Marc Cousin
Date:
Great, thanks :)

On Tuesday 15 October 2013 06:54:48 Dave Cramer wrote:
> Marc,
>
> I just pushed it to github, thanks!
>
> Dave Cramer
>
> dave.cramer(at)credativ(dot)ca
> http://www.credativ.ca
>
> On Tue, Oct 15, 2013 at 6:48 AM, Dave Cramer <pg@fastcrypt.com> wrote:
> > Marc,
> >
> > Sorry I have been travelling. Just looked at it, It's fine albeit more
> > than just changing the visibility of the function.
> >
> >  Dave Cramer
> >
> > dave.cramer(at)credativ(dot)ca
> > http://www.credativ.ca
> >
> > On Mon, Oct 14, 2013 at 10:38 AM, Marc Cousin <cousinmarc@gmail.com>wrote:
> >> Hi,
> >>
> >> Did you give it a look ?
> >>
> >> Regards
> >>
> >> Marc
> >>
> >> On Sunday 06 October 2013 19:14:41 Marc Cousin wrote:
> >> > Hi,
> >> >
> >> > Attached is an attempt to do this (git diff on head).
> >> >
> >> > The code seems to work. It has been tested on the program we are
> >>
> >> migrating
> >>
> >> > from SQL Server, we can now stream LOBs directly to web browsers...
> >> >
> >> > Please note that I am not usually a Java developper. I did my best to
> >> > do
> >> > this correctly, and the Java way (as far as I know :) ). If some things
> >>
> >> are
> >>
> >> > not done the right way, or there is a smarter solution, tell me, I'll
> >> > improve this.
> >> >
> >> > Regards,
> >> >
> >> > Marc
> >> >
> >> > On Tuesday 01 October 2013 08:11:21 Dave Cramer wrote:
> >> > > Well I don't see any reason not to open up the LOM class to allow you
> >>
> >> to
> >>
> >> > > extend it.
> >> > >
> >> > > Can you send a git pull?
> >> > >
> >> > > Dave Cramer
> >> > >
> >> > > dave.cramer(at)credativ(dot)ca
> >> > > http://www.credativ.ca
> >> > >
> >> > > On Tue, Oct 1, 2013 at 3:52 AM, Marc Cousin <cousinmarc@gmail.com>
> >>
> >> wrote:
> >> > > > Sorry, I don't think I get it, so please correct me below if I
> >>
> >> didn't.
> >>
> >> > > > I currently use getInputStream() on my large object. How could I
> >>
> >> provide
> >>
> >> > > > a
> >> > > > connection to the extended InputStream ? I am calling a LargeObject
> >> > > > method.
> >> > > >
> >> > > > Or maybe extend LargeObject to have a new getInputStream method
> >>
> >> with a
> >>
> >> > > > connection provided, and then provide this connection to the
> >>
> >> extended
> >>
> >> > > > InputStream ?
> >> > > >
> >> > > > My initial idea was to not have to provide the connection as a
> >> > > > parameter,
> >> > > > so I
> >> > > > wanted to use the Lob Manager.
> >> > > >
> >> > > > Anyway, shouldn't streaming a large object in this special case be
> >> > > > easier
> >> > > > to
> >> > > > do ? The original code (from SQL Server) didn't have to do all
> >> > > > these
> >> > > > workarounds, and I'll get this objection from people that will have
> >>
> >> to
> >>
> >> > > > use
> >> > > > this code :)
> >> > > >
> >> > > > Just having conn as protected instead of private in the
> >> > > > LargeObjectManager
> >> > > > would have made it possible to me to do it simply: I could have
> >>
> >> provided
> >>
> >> > > > the
> >> > > > connection to my extended LargeObject, and from there end the
> >> > > > transaction
> >> > > > on
> >> > > > close().
> >> > > >
> >> > > > Regards.
> >> > > >
> >> > > > On Monday 30 September 2013 17:08:18 Dave Cramer wrote:
> >> > > > > Marc,
> >> > > > >
> >> > > > > You required the connection to create the LargeObjectManager, so
> >>
> >> why
> >>
> >> > > > > not
> >> > > > > just extend the InputStream and add the connection to that
> >> > > > > object.
> >> > > > > When you close the input stream, close the connection ????
> >> > > > >
> >> > > > > Dave Cramer
> >> > > > >
> >> > > > > dave.cramer(at)credativ(dot)ca
> >> > > > > http://www.credativ.ca
> >> > > > >
> >> > > > > On Mon, Sep 30, 2013 at 11:18 AM, Marc Cousin <
> >>
> >> cousinmarc@gmail.com>
> >>
> >> > > > wrote:
> >> > > > > > Hi,
> >> > > > > >
> >> > > > > > I'm having the following problem: I need to stream a LOB to a
> >>
> >> web
> >>
> >> > > > client,
> >> > > >
> >> > > > > > without consuming
> >> > > > > > a lot of memory. So I am returning getInputStream() to the
> >>
> >> caller.
> >>
> >> > > > > > This works perfectly, except that I have to keep a transaction
> >>
> >> open
> >>
> >> > > > > > to
> >> > > > > > return the large object
> >> > > > > > stream, and I have (or found) no way of closing the transaction
> >>
> >> when
> >>
> >> > > > the
> >> > > >
> >> > > > > > filestream is closed.
> >> > > > > >
> >> > > > > > So of course, if I did that, I would get a lot of IDLE in
> >> > > > > > transaction
> >> > > > > > sessions...
> >> > > > > >
> >> > > > > > So the logical way seemed to me to extend LargeObjectManager
> >> > > > > > and
> >> > > > > > LargeObject to have a
> >> > > > > > close method on my LargeObject that closes the transaction,
> >>
> >> meaning
> >>
> >> > > > that
> >> > > >
> >> > > > > > my LargeObject
> >> > > > > > should have a connection attribute. So I wanted to do something
> >>
> >> like
> >>
> >> > > > this
> >> > > >
> >> > > > > > (not working,
> >> > > > > > it's just for the sake of explanation):
> >> > > > > >
> >> > > > > >
> >> > > > > > public class StreamLargeObjectManager extends
> >>
> >> LargeObjectManager{
> >>
> >> > > > > >            public StreamLargeObject openStreamLargeObject(long
> >>
> >> oid,
> >>
> >> > > > > >            int
> >> > > > > >
> >> > > > > > mode)
> >> > > > > >
> >> > > > > >              throws SQLException
> >> > > > > >
> >> > > > > >            {
> >> > > > > >
> >> > > > > >              LargeObject lo = super.open(oid, mode);
> >> > > > > >              StreamLargeObject so = new StreamLargeObject(lo,
> >> > > > > >
> >> > > > > > super.getConn()); //cannot be done because no access to
> >>
> >> connection
> >>
> >> > > > > >              return so;
> >> > > > > >
> >> > > > > >            }
> >> > > > > >
> >> > > > > > }
> >> > > > > >
> >> > > > > >
> >> > > > > > and
> >> > > > > >
> >> > > > > > public class StreamLargeObject extends
> >> > > > > > org.postgresql.largeobject.LargeObject{
> >> > > > > >
> >> > > > > >         Connection conn;
> >> > > > > >
> >> > > > > >         LargeObject lobj;
> >> > > > > >
> >> > > > > >
> >> > > > > >         public StreamLargeObject(LargeObject lg, BaseConnection
> >> > > > > >         conn)
> >> > > > > >
> >> > > > > > throws SQLException {
> >> > > > > >
> >> > > > > >                 this.lobj = lg;
> >> > > > > >                 this.conn = conn;
> >> > > > > >
> >> > > > > >         }
> >> > > > > >
> >> > > > > >
> >> > > > > >         @Override
> >> > > > > >         public void close() throws SQLException {
> >> > > > > >
> >> > > > > >                 lobj.close();
> >> > > > > >                 this.conn.commit();
> >> > > > > >
> >> > > > > >         }
> >> > > > > >
> >> > > > > >        public getInputStream() throws SQLException
> >> > > > > >
> >> > > > > >     {
> >> > > > > >
> >> > > > > >         return lobj.getInputStream();
> >> > > > > >
> >> > > > > >     }
> >> > > > > >
> >> > > > > > }
> >> > > > > >
> >> > > > > >
> >> > > > > > As the comment says it all, I cannot do this: conn is private
> >> > > > > > in
> >> > > > > > LargeObjectManager. For now, the problem has been
> >> > > > > > worked around by duplicating all the LargeObjectManager code,
> >>
> >> but
> >>
> >> > > > > > this
> >> > > >
> >> > > > is
> >> > > >
> >> > > > > > obviously ugly and not
> >> > > > > > maintainable. So my question is: have I missed some obvious
> >>
> >> solution
> >>
> >> > > > > > ?
> >> > > > > > Could a getter be added to
> >> > > > > > the LargeObjectManager's connection ?
> >> > > > > >
> >> > > > > > Regards,
> >> > > > > >
> >> > > > > > Marc
> >> > > > > >
> >> > > > > >
> >> > > > > > --
> >> > > > > > Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
> >> > > > > > To make changes to your subscription:
> >> > > > > > http://www.postgresql.org/mailpref/pgsql-jdbc