Thread: Feature Request: Report additionally error value
Hi all.
I often fall into error like this:
DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR: timestamp out of range
CONTEXT: SQL function "accounting_ready" statement 1 [for Statement "SELECT COUNT( * ) FROM (WITH
target_date AS ( SELECT ?::timestamptz ),
target_order as (
SELECT
invoice_range as bill_range,
o.*
FROM ( SELECT * FROM "order_bt" WHERE sys_period @> sys_time() ) o
LEFT JOIN period prd on prd.id = o.period_id
LEFT JOIN accounting_ready(
.....
other 200 lines of query
Would be nice if here also will be reported error value.
It will shed more light on what is comming wrong
Also would be useful if PG point at query where this bad value was calculated or occur.
It this possible?
Thank you.
--
Best regards,
Eugen Konkov
I often fall into error like this:
DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR: timestamp out of range
CONTEXT: SQL function "accounting_ready" statement 1 [for Statement "SELECT COUNT( * ) FROM (WITH
target_date AS ( SELECT ?::timestamptz ),
target_order as (
SELECT
invoice_range as bill_range,
o.*
FROM ( SELECT * FROM "order_bt" WHERE sys_period @> sys_time() ) o
LEFT JOIN period prd on prd.id = o.period_id
LEFT JOIN accounting_ready(
.....
other 200 lines of query
Would be nice if here also will be reported error value.
It will shed more light on what is comming wrong
Also would be useful if PG point at query where this bad value was calculated or occur.
It this possible?
Thank you.
--
Best regards,
Eugen Konkov
On Sat, Nov 28, 2020 at 4:18 PM Eugen Konkov <kes-kes@yandex.ru> wrote:
Hi all.
I often fall into error like this:
DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR: timestamp out of range
CONTEXT: SQL function "accounting_ready" statement 1 [for Statement "SELECT COUNT( * ) FROM (WITH
target_date AS ( SELECT ?::timestamptz ),
target_order as (
SELECT
invoice_range as bill_range,
o.*
FROM ( SELECT * FROM "order_bt" WHERE sys_period @> sys_time() ) o
LEFT JOIN period prd on prd.id = o.period_id
LEFT JOIN accounting_ready(
.....
other 200 lines of query
Would be nice if here also will be reported error value.
In this specific situation timestamp input does report the problematic text when text is provided (timestamp_in) but both the output and "receive/binary" routines simply provide the reported error.
Unless you are doing some kind of ETL with that query I'd bet money whatever perl is sending along for that input parameter is being sent in binary and is incompatible with PostgreSQL's allowed timestamp range.
Otherwise, yes, sometimes you just need to debug your data (though usually data is in text format and the error includes the problematic string).
Also would be useful if PG point at query where this bad value was calculated or occur.
This is not the first time we've seen this request and it usually ends up getting stalled because its non-trivial to implement and thus isn't feasible for the benefit it brings. In short, the text input parsing routines are decoupled from the queries where they are used.
David J.
"David G. Johnston" <david.g.johnston@gmail.com> writes: > On Sat, Nov 28, 2020 at 4:18 PM Eugen Konkov <kes-kes@yandex.ru> wrote: >> I often fall into error like this: >> DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st >> execute failed: ERROR: timestamp out of range >> Would be nice if here also will be reported error value. > In this specific situation timestamp input does report the problematic text > when text is provided (timestamp_in) but both the output and > "receive/binary" routines simply provide the reported error. A quick look through the source code says that the places where we report that message without providing a value are mostly places where timestamp2tm or the like has failed. That means we *can't* produce a value that will mean anything to a human; the range checks on timestamps are closely associated with the limitations of the calendar conversion code. > Unless you are doing some kind of ETL with that query I'd bet money > whatever perl is sending along for that input parameter is being sent in > binary and is incompatible with PostgreSQL's allowed timestamp range. Yeah. It's not that easy to get an out-of-range timestamp into Postgres, so a garbage value being sent in binary seems like a pretty likely explanation. It's not the only explanation, but you'd have to be doing a fairly out-of-the-ordinary calculation, like # select now() + interval '1000000 years'; ERROR: timestamp out of range >> Also would be useful if PG point at query where this bad value was >> calculated or occur. > This is not the first time we've seen this request and it usually ends up > getting stalled because its non-trivial to implement and thus isn't > feasible for the benefit it brings. I do still have ambitions to make that happen, but you're right that it's a major undertaking. Don't hold your breath. regards, tom lane
Hello Tom,
>>> Also would be useful if PG point at query where this bad value was
>>> calculated or occur.
>> This is not the first time we've seen this request and it usually ends up
>> getting stalled because its non-trivial to implement and thus isn't
>> feasible for the benefit it brings.
> I do still have ambitions to make that happen, but you're right that it's
> a major undertaking. Don't hold your breath.
Another case posted below.
Here in database are values which could not construct correct daterange.
There are thousands of rows. It is hard to find even a way or make an assumption
how to get record which cause this error.
It will be very impressive if database could report the line at database where error is occur.
Via HINT for example.
If this is not possible, then current cursor position or raw data dump in debug mode.
Probably some memory buffers or so.
Any additional info would be helpful.
Thank you.
DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR: range lower bound must be less than or equal to range upper bound?CONTEXT: SQL function "accounting_ready" statement 1 [for Statement "SELECT COUNT( * ) FROM (( WITH?target_date AS ( SELECT ?::timestamptz ),?target_order as (? SELECT? usage_range as bill_range,? o.*? FROM ( SELECT * FROM "order_bt" WHERE sys_period @> sys_time() ) o? LEFT JOIN period prd on prd.id = o.period_id? LEFT JOIN accounting_ready(? CASE WHEN prd.INTERVAL = '1 mon' THEN date_trunc( 'month', o.docdate ) ELSE o.docdate END,? prd.INTERVAL, (SELECT * FROM target_date)? ) acc ON true? WHERE FALSE? OR ? = 0? OR ? = 1 AND o.id = ?? AND acc.usage AND EXISTS (? SELECT * FROM order_bt prev_order? WHERE sys_period @> sys_time() AND? prev_order.id = o.id AND prev_order.app_period && acc.usage_range? )? AND o.app_period && acc.usage_range? OR ? = 2 AND o.agreement_id = ? and o.period_id = ?? AND acc.usage AND EXISTS (? SELECT * FROM order_bt prev_order? WHERE sys_period @> sys_time() AND? prev_order.id = o.id AND prev_order.app_period && acc.usage_range? )? AND o.app_period && acc.usage_range? OR ? = 3?),?USAGE AS ( SELECT? (ocd.o).id as order_id,? (dense_rank() over (PARTITION BY o.agreement_id, o.id ORDER BY (ocd.ic).consumed_period )) as group_id,? (ocd.c).id as detail_id,? (ocd.c).service_type_id as service_type,? (ocd.c).resource_type_id as detail_type,? (ocd.c).amount as detail_amount,? (ocd.c).allocated_resource_id as resource_id,? null as resource_uuid,? null as resource_desc,? rt.unit as resource_unit,? -- count changes. Logic is next: How many times configration for this order is met at this period? count(*) OVER (PARTITION BY (ocd.o).id, (ocd.c).id ) as consumed_count,? (ocd.ic).consumed as consumed_days,? null as consumed_amount,? null as consumed_last,?? a.id as agreement_id,? coalesce( substring( a.docn from '^A?(.*?)(?:\s*-[^-]*)?$' ), a.docn ) as agreement,? a.docdate as docdate,?? pkg.id as package_id,? pkg.link_1c_id as package_1c_id,? coalesce( pkg.display, pkg.name ) as package,?? coalesce( st.display, st.name, rt.display, rt.name ) as sr_name,? COALESCE( (ocd.p).label, rt.LABEL ) as unit_label,?? coalesce( (ocd.c).sort_order, pd.sort_order ) as sort_order,?? ocd.item_price AS price,? -- We want to display QTY for resources too? coalesce( ocd.item_qty, (ocd.c).amount/rt.unit ) AS qty,? 0 AS month_length,? 0 AS days_count,? o.bill_range,? lower( (ocd.ic).consumed_period ) as consumed_from,? upper( (ocd.ic).consumed_period ) -interval '1sec' as consumed_till,??? ocd.item_suma,?? 0 as discount,? (sum( ocd.item_suma ) OVER( PARTITION BY (ocd.o).id, (ocd.ic).consumed_period ))::numeric( 10, 2) AS group_suma,? (sum( ocd.item_cost ) OVER( PARTITION BY (ocd.o).id, (ocd.ic).consumed_period ))::numeric( 10, 2) AS order_cost?FROM target_order o?LEFT JOIN order_cost_details( o.bill_range ) ocd? ON (ocd.o).id = o.id AND (ocd.ic).consumed_period && o.app_period? LEFT JOIN agreement a ON a.id = o.agreement_id? LEFT JOIN package pkg ON pkg.id = o.package_id? LEFT JOIN package_detail pd ON pd.package_id = (ocd.o).package_id? AND pd.resource_type_id IS NOT DISTINCT FROM (ocd.c).resource_type_id? AND pd.service_type_id IS NOT DISTINCT FROM (ocd.c).service_type_id? LEFT JOIN resource_type rt ON rt.id = (ocd.c).resource_type_id? LEFT JOIN service_type st on st.id = (ocd.c).service_type_id?)??SELECT *,? (group_suma/6) ::numeric( 10, 2 ) as group_nds,? (SELECT sum(x) from (SELECT sum( DISTINCT group_suma ) AS x FROM usage sub_u WHERE sub_u.agreement_id = usage.agreement_id GROUP BY agreement_id, order_id) t) as total_suma,? (SELECT sum(x) from (SELECT sum( DISTINCT (group_suma/6)::numeric( 10, 2 ) ) AS x FROM usage sub_u WHERE sub_u.agreement_id = usage.agreement_id GROUP BY agreement_id, order_id) t) as total_nds?FROM usage?where ? <> 3 OR consumed_count > 1?ORDER BY? /* put order first then allocated resource without Order */? agreement,? order_id,? bill_range,? group_id,? sort_order nulls last,? detail_type nulls last,? price desc nulls last,? detail_amount desc,? service_type nulls last,? detail_id?)?) "me"" with ParamValues: 1='2020-08-01', 2='2', 3='2', 4=undef, 5='2', 6='3493', 7='10', 8='2', 9='2'] at /home/kes/work/projects/tucha/monkeyman/lib/MaitreD/Controller/Cart.pm line 828
--
Best regards,
Eugen Konkov
>>> Also would be useful if PG point at query where this bad value was
>>> calculated or occur.
>> This is not the first time we've seen this request and it usually ends up
>> getting stalled because its non-trivial to implement and thus isn't
>> feasible for the benefit it brings.
> I do still have ambitions to make that happen, but you're right that it's
> a major undertaking. Don't hold your breath.
Another case posted below.
Here in database are values which could not construct correct daterange.
There are thousands of rows. It is hard to find even a way or make an assumption
how to get record which cause this error.
It will be very impressive if database could report the line at database where error is occur.
Via HINT for example.
If this is not possible, then current cursor position or raw data dump in debug mode.
Probably some memory buffers or so.
Any additional info would be helpful.
Thank you.
DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR: range lower bound must be less than or equal to range upper bound?CONTEXT: SQL function "accounting_ready" statement 1 [for Statement "SELECT COUNT( * ) FROM (( WITH?target_date AS ( SELECT ?::timestamptz ),?target_order as (? SELECT? usage_range as bill_range,? o.*? FROM ( SELECT * FROM "order_bt" WHERE sys_period @> sys_time() ) o? LEFT JOIN period prd on prd.id = o.period_id? LEFT JOIN accounting_ready(? CASE WHEN prd.INTERVAL = '1 mon' THEN date_trunc( 'month', o.docdate ) ELSE o.docdate END,? prd.INTERVAL, (SELECT * FROM target_date)? ) acc ON true? WHERE FALSE? OR ? = 0? OR ? = 1 AND o.id = ?? AND acc.usage AND EXISTS (? SELECT * FROM order_bt prev_order? WHERE sys_period @> sys_time() AND? prev_order.id = o.id AND prev_order.app_period && acc.usage_range? )? AND o.app_period && acc.usage_range? OR ? = 2 AND o.agreement_id = ? and o.period_id = ?? AND acc.usage AND EXISTS (? SELECT * FROM order_bt prev_order? WHERE sys_period @> sys_time() AND? prev_order.id = o.id AND prev_order.app_period && acc.usage_range? )? AND o.app_period && acc.usage_range? OR ? = 3?),?USAGE AS ( SELECT? (ocd.o).id as order_id,? (dense_rank() over (PARTITION BY o.agreement_id, o.id ORDER BY (ocd.ic).consumed_period )) as group_id,? (ocd.c).id as detail_id,? (ocd.c).service_type_id as service_type,? (ocd.c).resource_type_id as detail_type,? (ocd.c).amount as detail_amount,? (ocd.c).allocated_resource_id as resource_id,? null as resource_uuid,? null as resource_desc,? rt.unit as resource_unit,? -- count changes. Logic is next: How many times configration for this order is met at this period? count(*) OVER (PARTITION BY (ocd.o).id, (ocd.c).id ) as consumed_count,? (ocd.ic).consumed as consumed_days,? null as consumed_amount,? null as consumed_last,?? a.id as agreement_id,? coalesce( substring( a.docn from '^A?(.*?)(?:\s*-[^-]*)?$' ), a.docn ) as agreement,? a.docdate as docdate,?? pkg.id as package_id,? pkg.link_1c_id as package_1c_id,? coalesce( pkg.display, pkg.name ) as package,?? coalesce( st.display, st.name, rt.display, rt.name ) as sr_name,? COALESCE( (ocd.p).label, rt.LABEL ) as unit_label,?? coalesce( (ocd.c).sort_order, pd.sort_order ) as sort_order,?? ocd.item_price AS price,? -- We want to display QTY for resources too? coalesce( ocd.item_qty, (ocd.c).amount/rt.unit ) AS qty,? 0 AS month_length,? 0 AS days_count,? o.bill_range,? lower( (ocd.ic).consumed_period ) as consumed_from,? upper( (ocd.ic).consumed_period ) -interval '1sec' as consumed_till,??? ocd.item_suma,?? 0 as discount,? (sum( ocd.item_suma ) OVER( PARTITION BY (ocd.o).id, (ocd.ic).consumed_period ))::numeric( 10, 2) AS group_suma,? (sum( ocd.item_cost ) OVER( PARTITION BY (ocd.o).id, (ocd.ic).consumed_period ))::numeric( 10, 2) AS order_cost?FROM target_order o?LEFT JOIN order_cost_details( o.bill_range ) ocd? ON (ocd.o).id = o.id AND (ocd.ic).consumed_period && o.app_period? LEFT JOIN agreement a ON a.id = o.agreement_id? LEFT JOIN package pkg ON pkg.id = o.package_id? LEFT JOIN package_detail pd ON pd.package_id = (ocd.o).package_id? AND pd.resource_type_id IS NOT DISTINCT FROM (ocd.c).resource_type_id? AND pd.service_type_id IS NOT DISTINCT FROM (ocd.c).service_type_id? LEFT JOIN resource_type rt ON rt.id = (ocd.c).resource_type_id? LEFT JOIN service_type st on st.id = (ocd.c).service_type_id?)??SELECT *,? (group_suma/6) ::numeric( 10, 2 ) as group_nds,? (SELECT sum(x) from (SELECT sum( DISTINCT group_suma ) AS x FROM usage sub_u WHERE sub_u.agreement_id = usage.agreement_id GROUP BY agreement_id, order_id) t) as total_suma,? (SELECT sum(x) from (SELECT sum( DISTINCT (group_suma/6)::numeric( 10, 2 ) ) AS x FROM usage sub_u WHERE sub_u.agreement_id = usage.agreement_id GROUP BY agreement_id, order_id) t) as total_nds?FROM usage?where ? <> 3 OR consumed_count > 1?ORDER BY? /* put order first then allocated resource without Order */? agreement,? order_id,? bill_range,? group_id,? sort_order nulls last,? detail_type nulls last,? price desc nulls last,? detail_amount desc,? service_type nulls last,? detail_id?)?) "me"" with ParamValues: 1='2020-08-01', 2='2', 3='2', 4=undef, 5='2', 6='3493', 7='10', 8='2', 9='2'] at /home/kes/work/projects/tucha/monkeyman/lib/MaitreD/Controller/Cart.pm line 828
--
Best regards,
Eugen Konkov
I format query to read it more easy. See attachment
Tuesday, December 8, 2020, 3:43:13 PM, you wrote:
--
Best regards,
Eugen Konkov
Tuesday, December 8, 2020, 3:43:13 PM, you wrote:
Hello Tom, >>> Also would be useful if PG point at query where this bad value was >>> calculated or occur. >> This is not the first time we've seen this request and it usually ends up >> getting stalled because its non-trivial to implement and thus isn't >> feasible for the benefit it brings. > I do still have ambitions to make that happen, but you're right that it's > a major undertaking. Don't hold your breath. Another case posted below. Here in database are values which could not construct correct daterange. There are thousands of rows. It is hard to find even a way or make an assumption how to get record which cause this error. It will be very impressive if database could report the line at database where error is occur. Via HINT for example. If this is not possible, then current cursor position or raw data dump in debug mode. Probably some memory buffers or so. Any additional info would be helpful. Thank you. DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR: range lower bound must be less than or equal to range upper bound?CONTEXT: SQL function "accounting_ready" statement 1 [for Statement "SELECT COUNT( * ) FROM (( WITH?target_date AS ( SELECT ?::timestamptz ),?target_order as (? SELECT? usage_range as bill_range,? o.*? FROM ( SELECT * FROM "order_bt" WHERE sys_period @> sys_time() ) o? LEFT JOIN period prd on prd.id = o.period_id? LEFT JOIN accounting_ready(? CASE WHEN prd.INTERVAL = '1 mon' THEN date_trunc( 'month', o.docdate ) ELSE o.docdate END,? prd.INTERVAL, (SELECT * FROM target_date)? ) acc ON true? WHERE FALSE? OR ? = 0? OR ? = 1 AND o.id = ?? AND acc.usage AND EXISTS (? SELECT * FROM order_bt prev_order? WHERE sys_period @> sys_time() AND? prev_order.id = o.id AND prev_order.app_period && acc.usage_range? )? AND o.app_period && acc.usage_range? OR ? = 2 AND o.agreement_id = ? and o.period_id = ?? AND acc.usage AND EXISTS (? SELECT * FROM order_bt prev_order? WHERE sys_period @> sys_time() AND? prev_order.id = o.id AND prev_order.app_period && acc.usage_range? )? AND o.app_period && acc.usage_range? OR ? = 3?),?USAGE AS ( SELECT? (ocd.o).id as order_id,? (dense_rank() over (PARTITION BY o.agreement_id, o.id ORDER BY (ocd.ic).consumed_period )) as group_id,? (ocd.c).id as detail_id,? (ocd.c).service_type_id as service_type,? (ocd.c).resource_type_id as detail_type,? (ocd.c).amount as detail_amount,? (ocd.c).allocated_resource_id as resource_id,? null as resource_uuid,? null as resource_desc,? rt.unit as resource_unit,? -- count changes. Logic is next: How many times configration for this order is met at this period? count(*) OVER (PARTITION BY (ocd.o).id, (ocd.c).id ) as consumed_count,? (ocd.ic).consumed as consumed_days,? null as consumed_amount,? null as consumed_last,?? a.id as agreement_id,? coalesce( substring( a.docn from '^A?(.*?)(?:\s*-[^-]*)?$' ), a.docn ) as agreement,? a.docdate as docdate,?? pkg.id as package_id,? pkg.link_1c_id as package_1c_id,? coalesce( pkg.display, pkg.name ) as package,?? coalesce( st.display, st.name, rt.display, rt.name ) as sr_name,? COALESCE( (ocd.p).label, rt.LABEL ) as unit_label,?? coalesce( (ocd.c).sort_order, pd.sort_order ) as sort_order,?? ocd.item_price AS price,? -- We want to display QTY for resources too? coalesce( ocd.item_qty, (ocd.c).amount/rt.unit ) AS qty,? 0 AS month_length,? 0 AS days_count,? o.bill_range,? lower( (ocd.ic).consumed_period ) as consumed_from,? upper( (ocd.ic).consumed_period ) -interval '1sec' as consumed_till,??? ocd.item_suma,?? 0 as discount,? (sum( ocd.item_suma ) OVER( PARTITION BY (ocd.o).id, (ocd.ic).consumed_period ))::numeric( 10, 2) AS group_suma,? (sum( ocd.item_cost ) OVER( PARTITION BY (ocd.o).id, (ocd.ic).consumed_period ))::numeric( 10, 2) AS order_cost?FROM target_order o?LEFT JOIN order_cost_details( o.bill_range ) ocd? ON (ocd.o).id = o.id AND (ocd.ic).consumed_period && o.app_period? LEFT JOIN agreement a ON a.id = o.agreement_id? LEFT JOIN package pkg ON pkg.id = o.package_id? LEFT JOIN package_detail pd ON pd.package_id = (ocd.o).package_id? AND pd.resource_type_id IS NOT DISTINCT FROM (ocd.c).resource_type_id? AND pd.service_type_id IS NOT DISTINCT FROM (ocd.c).service_type_id? LEFT JOIN resource_type rt ON rt.id = (ocd.c).resource_type_id? LEFT JOIN service_type st on st.id = (ocd.c).service_type_id?)??SELECT *,? (group_suma/6) ::numeric( 10, 2 ) as group_nds,? (SELECT sum(x) from (SELECT sum( DISTINCT group_suma ) AS x FROM usage sub_u WHERE sub_u.agreement_id = usage.agreement_id GROUP BY agreement_id, order_id) t) as total_suma,? (SELECT sum(x) from (SELECT sum( DISTINCT (group_suma/6)::numeric( 10, 2 ) ) AS x FROM usage sub_u WHERE sub_u.agreement_id = usage.agreement_id GROUP BY agreement_id, order_id) t) as total_nds?FROM usage?where ? <> 3 OR consumed_count > 1?ORDER BY? /* put order first then allocated resource without Order */? agreement,? order_id,? bill_range,? group_id,? sort_order nulls last,? detail_type nulls last,? price desc nulls last,? detail_amount desc,? service_type nulls last,? detail_id?)?) "me"" with ParamValues: 1='2020-08-01', 2='2', 3='2', 4=undef, 5='2', 6='3493', 7='10', 8='2', 9='2'] at /home/kes/work/projects/tucha/monkeyman/lib/MaitreD/Controller/Cart.pm line 828 -- Best regards, Eugen Konkov |
--
Best regards,
Eugen Konkov