Re: errors on restoring postgresql binary dump to glusterfs - Mailing list pgsql-general

On Mon, May 7, 2012 at 7:34 PM, Liang Ma <ma.satops@gmail.com> wrote:
> On Mon, May 7, 2012 at 12:54 PM, Magnus Hagander <magnus@hagander.net> wrote:
>> On Mon, May 7, 2012 at 5:02 PM, Liang Ma <ma.satops@gmail.com> wrote:
>>> On Fri, May 4, 2012 at 3:58 AM, Magnus Hagander <magnus@hagander.net> wrote:
>>>> On Mon, Apr 30, 2012 at 8:34 PM, Liang Ma <ma.satops@gmail.com> wrote:
>>>>> Hi There,
>>>>>
>>>>> While trying to restore a ~700GM binary dump by command
>>>>>
>>>>> pg_restore -d dbdata < sampledbdata-20120327.pgdump
>>>>>
>>>>> I encountered following errors repeatedly
>>>>>
>>>>> pg_restore: [archiver (db)] Error from TOC entry 2882463; 2613
>>>>> 10267347 BLOB 10267347 sdmcleod
>>>>> pg_restore: [archiver (db)] could not execute query: ERROR:
>>>>> unexpected data beyond EOF in block 500 of relation base/16386/11743
>>>>> HINT:  This has been seen to occur with buggy kernels; consider
>>>>> updating your system.
>>>>
>>>> Note the message right here...
>>>>
>>>> There may be further indications in the server log about what's wrong.
>>>>
>>>
>>> The server's logs in message file were clean.
>>
>> Then your logging is incorrectly configured, because it should *at
>> least* have the same message as the one that showed up in the client.
>>
>
> Oh, yes, the same error messages were logged in the postgresql log
> file but no further information. I thought you implied that there may
> be some indication in server's system logs, which I couldn't find any.

Well, there might be, I wasn't sure :-) I guess there wasn't.


>>>>> The server runs Ubuntu server 10.04 LTS with postgresql upgraded to
>>>>> version 9.1.3-1~lucid. The postgresql data directory is located in a
>>>>> glusterfs mounted directory to a replicated volume vol-2
>>>>
>>>> I assume you don't have more than one node actually *accessing* the
>>>> data directory at the same time, right?
>>>>
>>>
>>> Yes, you are right. I just set up this glusterfs and postgresql server
>>> with two nodes for testing purpose. There was no other gluster
>>> filesystem access activity at the time I tried to restore the
>>> postgresql dump. Do you know if postgresql recommends any other
>>> cluster filesystem, or it may not like cluster filesystem at all?
>>
>>
>> Did you have PostgreSQL started on both nodes? That is *not*
>> supported. If PostgreSQL only runs on one node at a time it should in
>> theory work, provided the cluster filesystem provides all the services
>> that a normal filesystem does, such as respecting fsync.
>>
>
> Postgresql are installed in both nodes, but only one node's postgresql
> data directory points to glusterfs filesystem. Another one's data
> directory is in its default location in the local ext4 filesystem.
> This is the one I used to prove the dump file can be restored without
> any problem when glusterfs is not involved.

ok. That should in theory be safe. Having two active notes against th
efilesystem is never safe.


> According to its introduction and document, glusterfs is supposed to
> appear as a normal filesystem when being mounted, although I don't
> know how well it respects things like fsync.

It certainly looks like it's failing at some point. So yeah, I'm
pretty sure you need to get in touch with the glusterfs folks -
hopefully you get a response from them soon.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/