Postgres Pro
Downloads
Home   >   mailing lists

Thread: [BUGS] BUG #14709: inconsistent answers with foreign data wrappers to mysql

[BUGS] BUG #14709: inconsistent answers with foreign data wrappers to mysql

From
mark.manley@tapad.com
Date:
The following bug has been logged on the website:

Bug reference:      14709
Logged by:          Mark Manley
Email address:      mark.manley@tapad.com
PostgreSQL version: 9.6.2
Operating system:   CentOS 7.3
Description:

Greetings.

When using a foreign data wrapper from our 9.6.2 database to a MySQL 5.7
database, we get inconsistent results.  We're joining two tables from
another server:

dsp_users
dsp_accounts

This table is local in our Postgres server:

audiences

This query returns only the first record, which is using simple joins:

select audiences.* FROM audiences
INNER JOIN dsp_users ON dsp_users.id = audiences.created_by
INNER JOIN dsp_accounts ON dsp_accounts.id = audiences.account_id
WHERE audiences.name like '%Test%'

and I get only the first hit.

When I run the composite queries under it, removing the like clause from the
predicate, I get all the results I would expect:

1  |23842583561190673   |XXXXXXXX                  |sg1      |Test audience                      |            |1012
 |269        |2017-05-03 
15:04:57 |
2  |23842583562050673   |XXXXXXXX                  |sg1      |Test audience                      |            |1012
 |269        |2017-05-03 
15:25:59 |
10 |23842688223720004   |XXXXXXXX                  |sg1      |Audience DTAC
Test 1                 |            |1013       |307        |2017-06-09
16:40:54 |

with the clause:

1  |23842583561190673   |XXXXXXXX                  |sg1      |Test audience                      |            |1012
 |269        |2017-05-03 
15:04:57 |

If I rewrite the query to turn the foreign MySQL tables into a subselect,
then I get the correct results:

select a.* from audiences a
where exists (select 1 from audiences a, dsp_users du, dsp_accounts dawherea.created_by = du.idand a.account_id =
da.id)

1  |23842583561190673   |XXXXXXXX                  |sg1      |Test audience                      |            |1012
 |269        |2017-05-03 
15:04:57 |
2  |23842583562050673   |XXXXXXXX                  |sg1      |Test audience                      |            |1012
 |269        |2017-05-03 
15:25:59 |
10 |23842688223720004   |XXXXXXXX                  |sg1      |Audience DTAC
Test 1                 |            |1013       |307        |2017-06-09
16:40:54 |

As you can see, I get inconsistent results if I use a normal join versus a
subselect.  It's as though the like filter in the predicate is breaking.
This behaviour does not happen with using likes joining local tables only.


Anyway, wanted you guys to know.

Thanks!


--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Re: [BUGS] BUG #14709: inconsistent answers with foreign data wrappers to mysql

From
Tom Lane
Date:
mark.manley@tapad.com writes:
> When using a foreign data wrapper from our 9.6.2 database to a MySQL 5.7
> database, we get inconsistent results.

My first thought is that you're looking at a bug in the mysql FDW,
probably incorrect pushdown of conditions to the mysql server.  You
should enable logging on the mysql server and look at the queries
generated by the FDW, comparing those to EXPLAIN output to see which
conditions get checked locally as opposed to being sent to the remote.

If you can identify what the FDW is doing wrong, reporting that to
the FDW's authors might get some results.  (I don't know if they
read this bugs list.)
        regards, tom lane


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Re: [BUGS] BUG #14709: inconsistent answers with foreign data wrappers to mysql

From
Andrew Gierth
Date:
>>>>> "mark" == mark manley <mark.manley@tapad.com> writes:
mark> When using a foreign data wrapper from our 9.6.2 database to amark> MySQL 5.7 database, we get inconsistent
results. We're joiningmark> two tables from another server:
 

mysql_fdw is (and has _always_ been) broken for anything beyond the most
trivial queries:

https://github.com/EnterpriseDB/mysql_fdw/blob/master/mysql_fdw.c#L734

https://github.com/EnterpriseDB/mysql_fdw/issues/73

Since mysql_fdw is not part of postgresql you should take this up with
the maintainers. However, the linked issue has been outstanding for 18
months, so probably best not hold your breath.

-- 
Andrew (irc:RhodiumToad)


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Re: [BUGS] BUG #14709: inconsistent answers with foreign datawrappers to mysql

From
Rushabh Lathia
Date:
Hi,

I informed the maintainer of mysql_fdw and they will start looking into this issue.

Thanks,


On Mon, Jun 19, 2017 at 4:13 AM, Andrew Gierth <andrew@tao11.riddles.org.uk> wrote:
>>>>> "mark" == mark manley <mark.manley@tapad.com> writes:

 mark> When using a foreign data wrapper from our 9.6.2 database to a
 mark> MySQL 5.7 database, we get inconsistent results.  We're joining
 mark> two tables from another server:

mysql_fdw is (and has _always_ been) broken for anything beyond the most
trivial queries:

https://github.com/EnterpriseDB/mysql_fdw/blob/master/mysql_fdw.c#L734

https://github.com/EnterpriseDB/mysql_fdw/issues/73

Since mysql_fdw is not part of postgresql you should take this up with
the maintainers. However, the linked issue has been outstanding for 18
months, so probably best not hold your breath.

--
Andrew (irc:RhodiumToad)


--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs



--
Rushabh Lathia
www.EnterpriseDB.com

Re: [BUGS] BUG #14709: inconsistent answers with foreign datawrappers to mysql

From
Mark Manley
Date:
Thank you all.  :)


On Mon, Jun 19, 2017 at 3:57 AM, Rushabh Lathia <rushabh.lathia@gmail.com> wrote:
Hi,

I informed the maintainer of mysql_fdw and they will start looking into this issue.

Thanks,


On Mon, Jun 19, 2017 at 4:13 AM, Andrew Gierth <andrew@tao11.riddles.org.uk> wrote:
>>>>> "mark" == mark manley <mark.manley@tapad.com> writes:

 mark> When using a foreign data wrapper from our 9.6.2 database to a
 mark> MySQL 5.7 database, we get inconsistent results.  We're joining
 mark> two tables from another server:

mysql_fdw is (and has _always_ been) broken for anything beyond the most
trivial queries:

https://github.com/EnterpriseDB/mysql_fdw/blob/master/mysql_fdw.c#L734

https://github.com/EnterpriseDB/mysql_fdw/issues/73

Since mysql_fdw is not part of postgresql you should take this up with
the maintainers. However, the linked issue has been outstanding for 18
months, so probably best not hold your breath.

--
Andrew (irc:RhodiumToad)


--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs



--
Rushabh Lathia
www.EnterpriseDB.com

Re: [BUGS] BUG #14709: inconsistent answers with foreign datawrappers to mysql

From
Ahsan Hadi
Date:
It will be helpful if you can share a test case that can be used to verify our fix?

-- Ahsan


On Mon, Jun 19, 2017 at 3:57 PM, Mark Manley <mark.manley@tapad.com> wrote:
Thank you all.  :)


On Mon, Jun 19, 2017 at 3:57 AM, Rushabh Lathia <rushabh.lathia@gmail.com> wrote:
Hi,

I informed the maintainer of mysql_fdw and they will start looking into this issue.

Thanks,


On Mon, Jun 19, 2017 at 4:13 AM, Andrew Gierth <andrew@tao11.riddles.org.uk> wrote:
>>>>> "mark" == mark manley <mark.manley@tapad.com> writes:

 mark> When using a foreign data wrapper from our 9.6.2 database to a
 mark> MySQL 5.7 database, we get inconsistent results.  We're joining
 mark> two tables from another server:

mysql_fdw is (and has _always_ been) broken for anything beyond the most
trivial queries:

https://github.com/EnterpriseDB/mysql_fdw/blob/master/mysql_fdw.c#L734

https://github.com/EnterpriseDB/mysql_fdw/issues/73

Since mysql_fdw is not part of postgresql you should take this up with
the maintainers. However, the linked issue has been outstanding for 18
months, so probably best not hold your breath.

--
Andrew (irc:RhodiumToad)


--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs



--
Rushabh Lathia
www.EnterpriseDB.com




--
Ahsan Hadi
Snr Director Product Development
EnterpriseDB Corporation
The Enterprise Postgres Company

Phone: +92-51-8358874   
Mobile: +92-333-5162114

Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb

This e-mail message (and any attachment) is intended for the use of the individual or entity to whom it is addressed. This message contains information from EnterpriseDB Corporation that may be privileged, confidential, or exempt from disclosure under applicable law. If you are not the intended recipient or authorized to receive this for the intended recipient, any use, dissemination, distribution, retention, archiving, or copying of this communication is strictly prohibited. If you have received this e-mail in error, please notify the sender immediately by reply e-mail and delete this message.

Re: [BUGS] BUG #14709: inconsistent answers with foreign datawrappers to mysql

From
Ahsan Hadi
Date:
The fix for this issue is committed in mysql_fdw github repository, please see the following mysql_fdw github entry for details...
Thanks Mark for providing the test case.

On Mon, Jun 19, 2017 at 4:01 PM, Ahsan Hadi <ahsan.hadi@enterprisedb.com> wrote:
It will be helpful if you can share a test case that can be used to verify our fix?

-- Ahsan


On Mon, Jun 19, 2017 at 3:57 PM, Mark Manley <mark.manley@tapad.com> wrote:
Thank you all.  :)


On Mon, Jun 19, 2017 at 3:57 AM, Rushabh Lathia <rushabh.lathia@gmail.com> wrote:
Hi,

I informed the maintainer of mysql_fdw and they will start looking into this issue.

Thanks,


On Mon, Jun 19, 2017 at 4:13 AM, Andrew Gierth <andrew@tao11.riddles.org.uk> wrote:
>>>>> "mark" == mark manley <mark.manley@tapad.com> writes:

 mark> When using a foreign data wrapper from our 9.6.2 database to a
 mark> MySQL 5.7 database, we get inconsistent results.  We're joining
 mark> two tables from another server:

mysql_fdw is (and has _always_ been) broken for anything beyond the most
trivial queries:

https://github.com/EnterpriseDB/mysql_fdw/blob/master/mysql_fdw.c#L734

https://github.com/EnterpriseDB/mysql_fdw/issues/73

Since mysql_fdw is not part of postgresql you should take this up with
the maintainers. However, the linked issue has been outstanding for 18
months, so probably best not hold your breath.

--
Andrew (irc:RhodiumToad)


--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs



--
Rushabh Lathia
www.EnterpriseDB.com




--
Ahsan Hadi
Snr Director Product Development
EnterpriseDB Corporation
The Enterprise Postgres Company

Phone: +92-51-8358874   
Mobile: +92-333-5162114

Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb

This e-mail message (and any attachment) is intended for the use of the individual or entity to whom it is addressed. This message contains information from EnterpriseDB Corporation that may be privileged, confidential, or exempt from disclosure under applicable law. If you are not the intended recipient or authorized to receive this for the intended recipient, any use, dissemination, distribution, retention, archiving, or copying of this communication is strictly prohibited. If you have received this e-mail in error, please notify the sender immediately by reply e-mail and delete this message.



--
Ahsan Hadi
Snr Director Product Development
EnterpriseDB Corporation
The Enterprise Postgres Company

Phone: +92-51-8358874   
Mobile: +92-333-5162114

Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb

This e-mail message (and any attachment) is intended for the use of the individual or entity to whom it is addressed. This message contains information from EnterpriseDB Corporation that may be privileged, confidential, or exempt from disclosure under applicable law. If you are not the intended recipient or authorized to receive this for the intended recipient, any use, dissemination, distribution, retention, archiving, or copying of this communication is strictly prohibited. If you have received this e-mail in error, please notify the sender immediately by reply e-mail and delete this message.

Re: [BUGS] BUG #14709: inconsistent answers with foreign datawrappers to mysql

From
Mark Manley
Date:
Thank you.  I have confirmed your fix works:

iris_staging=# select audiences.* FROM audiences
INNER JOIN mwm_users ON mwm_users.id = audiences.created_by
INNER JOIN mwm_accounts ON mwm_accounts.id = audiences.account_id and audiences.name like '%name%';
 id |       foreign_audience_id       | advertising_platform_name | location |   name    |    description     | account
_id | created_by |          created_at
----+---------------------------------+---------------------------+----------+-----------+--------------------+--------
----+------------+-------------------------------
  1 | mock_facebook_audience(happy)-1 | FACEBOOK                  | nj1      | some name | some foreign descr |       1
012 |        269 | 2017-05-02 22:49:24.984224+00
  2 | mock_facebook_audience(happy)-2 | FACEBOOK                  | nj1      | some name | some foreign descr |       1
012 |        269 | 2017-05-02 22:49:49.771061+00
  3 | mock_facebook_audience(happy)-1 | FACEBOOK                  | nj1      | some name | some foreign descr |       1
012 |        269 | 2017-05-04 18:38:54.569716+00
(3 rows)

Thanks again!

On Wed, Jun 21, 2017 at 1:50 AM, Ahsan Hadi <ahsan.hadi@enterprisedb.com> wrote:
The fix for this issue is committed in mysql_fdw github repository, please see the following mysql_fdw github entry for details...
Thanks Mark for providing the test case.

On Mon, Jun 19, 2017 at 4:01 PM, Ahsan Hadi <ahsan.hadi@enterprisedb.com> wrote:
It will be helpful if you can share a test case that can be used to verify our fix?

-- Ahsan


On Mon, Jun 19, 2017 at 3:57 PM, Mark Manley <mark.manley@tapad.com> wrote:
Thank you all.  :)


On Mon, Jun 19, 2017 at 3:57 AM, Rushabh Lathia <rushabh.lathia@gmail.com> wrote:
Hi,

I informed the maintainer of mysql_fdw and they will start looking into this issue.

Thanks,


On Mon, Jun 19, 2017 at 4:13 AM, Andrew Gierth <andrew@tao11.riddles.org.uk> wrote:
>>>>> "mark" == mark manley <mark.manley@tapad.com> writes:

 mark> When using a foreign data wrapper from our 9.6.2 database to a
 mark> MySQL 5.7 database, we get inconsistent results.  We're joining
 mark> two tables from another server:

mysql_fdw is (and has _always_ been) broken for anything beyond the most
trivial queries:

https://github.com/EnterpriseDB/mysql_fdw/blob/master/mysql_fdw.c#L734

https://github.com/EnterpriseDB/mysql_fdw/issues/73

Since mysql_fdw is not part of postgresql you should take this up with
the maintainers. However, the linked issue has been outstanding for 18
months, so probably best not hold your breath.

--
Andrew (irc:RhodiumToad)


--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs



--
Rushabh Lathia
www.EnterpriseDB.com




--
Ahsan Hadi
Snr Director Product Development
EnterpriseDB Corporation
The Enterprise Postgres Company

Phone: +92-51-8358874   
Mobile: +92-333-5162114

Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb

This e-mail message (and any attachment) is intended for the use of the individual or entity to whom it is addressed. This message contains information from EnterpriseDB Corporation that may be privileged, confidential, or exempt from disclosure under applicable law. If you are not the intended recipient or authorized to receive this for the intended recipient, any use, dissemination, distribution, retention, archiving, or copying of this communication is strictly prohibited. If you have received this e-mail in error, please notify the sender immediately by reply e-mail and delete this message.



--
Ahsan Hadi
Snr Director Product Development
EnterpriseDB Corporation
The Enterprise Postgres Company

Phone: +92-51-8358874   
Mobile: +92-333-5162114

Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb

This e-mail message (and any attachment) is intended for the use of the individual or entity to whom it is addressed. This message contains information from EnterpriseDB Corporation that may be privileged, confidential, or exempt from disclosure under applicable law. If you are not the intended recipient or authorized to receive this for the intended recipient, any use, dissemination, distribution, retention, archiving, or copying of this communication is strictly prohibited. If you have received this e-mail in error, please notify the sender immediately by reply e-mail and delete this message.