Thread: partition query on multiple cores
I have 8-core server, I wanted to ask whether a query can be divided for multiple processors or cores, if it could be what to do in postgresql
Thanks
Thanks
> I have 8-core server, I wanted to ask whether a query can be divided for > multiple processors or cores, if it could be what to do in postgresql No, at this time (and for the foreseeable future), a single query will run on a single core. --- Maciek Sakrejda | System Architect | Truviso 1065 E. Hillsdale Blvd., Suite 215 Foster City, CA 94404 (650) 242-3500 Main www.truviso.com
On 05/10/2011 10:06 AM, Maciek Sakrejda wrote: >> I have 8-core server, I wanted to ask whether a query can be divided for >> multiple processors or cores, if it could be what to do in postgresql > > No, at this time (and for the foreseeable future), a single query will > run on a single core. It can *kinda* be done. Take a look at GridSQL. It's really good for splitting up reporting-like queries that benefit from parallel access of large tables. It's not exactly Hadoop, but I ran a test on a single system with two separate instances of PostgreSQL, and a single query over those two nodes cut execution time in half. It's meant for server parallelism, so I wouldn't necessarily recommend splitting your data up across nodes on the same server. But it seems to deliver as promised when used in the right circumstances. -- Shaun Thomas OptionsHouse | 141 W. Jackson Blvd. | Suite 800 | Chicago IL, 60604 312-676-8870 sthomas@peak6.com ______________________________________________ See http://www.peak6.com/email_disclaimer.php for terms and conditions related to this email
Dne 10.5.2011 18:22, Shaun Thomas napsal(a): > On 05/10/2011 10:06 AM, Maciek Sakrejda wrote: > >>> I have 8-core server, I wanted to ask whether a query can be divided for >>> multiple processors or cores, if it could be what to do in postgresql >> >> No, at this time (and for the foreseeable future), a single query will >> run on a single core. > > It can *kinda* be done. Take a look at GridSQL. Or pgpool-II, that can give you something similar. http://pgpool.projects.postgresql.org/ regards Tomas
On Tue, May 10, 2011 at 12:22 PM, Shaun Thomas <sthomas@peak6.com> wrote:
Yes, GridSQL is useful even in multi-core scenarios on a single server for query parallelism. You can also use the same PostgreSQL instance (cluster), as the virtual node databases are named distinctly, which simplifies configuration.
On 05/10/2011 10:06 AM, Maciek Sakrejda wrote:It can *kinda* be done. Take a look at GridSQL. It's really good for splitting up reporting-like queries that benefit from parallel access of large tables. It's not exactly Hadoop, but I ran a test on a single system with two separate instances of PostgreSQL, and a single query over those two nodes cut execution time in half.I have 8-core server, I wanted to ask whether a query can be divided for
multiple processors or cores, if it could be what to do in postgresql
No, at this time (and for the foreseeable future), a single query will
run on a single core.
It's meant for server parallelism, so I wouldn't necessarily recommend splitting your data up across nodes on the same server. But it seems to deliver as promised when used in the right circumstances.
Yes, GridSQL is useful even in multi-core scenarios on a single server for query parallelism. You can also use the same PostgreSQL instance (cluster), as the virtual node databases are named distinctly, which simplifies configuration.
Mason
On Tue, May 10, 2011 at 2:57 PM, Tomas Vondra <tv@fuzzy.cz> wrote:
Dne 10.5.2011 18:22, Shaun Thomas napsal(a):> On 05/10/2011 10:06 AM, Maciek Sakrejda wrote:Or pgpool-II, that can give you something similar.
>
>>> I have 8-core server, I wanted to ask whether a query can be divided for
>>> multiple processors or cores, if it could be what to do in postgresql
>>
>> No, at this time (and for the foreseeable future), a single query will
>> run on a single core.
>
> It can *kinda* be done. Take a look at GridSQL.
http://pgpool.projects.postgresql.org/
Last time I tested parallelism in pgpool-II, I saw that if your query is fairly simple, pgpool-II will help. If it is more complex with joins and aggregates, GridSQL will typically outperform it. GridSQL pushes down joins as much as possible, minimizes row shipping, and parallelizes aggregates and grouping.
Mason Sharp