Thread: arrays, inline to pointer

Hi,

I have some whopper tables for machine learning. One table has a handful
of 200-500 double precision arrays (representing feature vectors). It's
a 9.5 on a SSD (over USB3). Each table has 5-15M rows in them.

For each array I've added, and populated, any dealings with the table
has become way way slower. I can only assume this is because the array
data is inline in the datablock on disk that stores the row.

Is it possible to "have a pointer instead", or is the only workaround a
child table with vector only in it?

Best regards,
Marcus

For each array I've added, and populated, any dealings with the table has become way way slower. I can only assume this is because the array data is inline in the datablock on disk that stores the row.

any field on a table thats more than a few dozen bytes gets 'toasted' and stored out of line in special 'toast' tables.

-- 
john r pierce, recycling bits in santa cruz

In fact, disabling toast compression will probably improve the performance (the indirection will still take place). A float array is not usually very compressible anyway.