On Thu, 31 May 2012, Jeff Janes wrote:
>>
>> No, idt_match is getting filled by multi-threaded copy() and then joined
>> with 4 other big tables like idt_phot. The result is then split into
>> partitions.
>
> That does make things more complicated. But you could you partition
> it at that level and then do the joins partition-wise?
Unfortunately the information to understand in which partition the data
needs to be put in is only available from the idt_match table. So I have
to do at least one join with idt_match. But I will experiment further
with ways to perform queries, I just stopped doing that because I saw
these problems with scalability.
> I don't have much experience at data partitioning (well, I do, but the
> experience is with partitioning in Perl with terabytes of flat files,
> not in PG :) ) but I think that once you have your partitioning keys
> you want to apply them the same way up and down the data set.
I'm not sure what you mean by "the same way up and down the data set".
Cheers, S
*****************************************************
Sergey E. Koposov, PhD, Research Associate
Institute of Astronomy, University of Cambridge
Madingley road, CB3 0HA, Cambridge, UK
Tel: +44-1223-337-551 Web: http://www.ast.cam.ac.uk/~koposov/
