presto: Infinite loop during planning in 0.193
Hi,
I use presto 0.193 + HDP 2.5.3 + hive connector.
Infinite loop during planning for the following query occurs.
This error doesn’t occur in presto 0.192.
According to the 0.193 release note(https://prestodb.io/docs/current/release/release-0.193.html), Infinite loop has already been fixed, but it seems not to fix.
Fix an infinite loop during planning for queries containing non-trivial predicates.
I’m sorry not to create simple query which reproduces the issue…
SELECT
...
FROM
(
SELECT
...
FROM
(SELECT * FROM partitioned_table1 WHERE dt = date_format(date_add('day', -1, now()), '%Y%m%d') AND ...) aa
JOIN
(SELECT * FROM partitioned_table2 WHERE dt = date_format(date_add('day', -1, now()), '%Y%m%d')) bb
ON
aa.some_id = bb.id
JOIN
(SELECT * FROM partitioned_table3 WHERE dt = date_format(date_add('day', -1, now()), '%Y%m%d') AND ...) cc
ON
aa.some_id = cc.id
JOIN
(SELECT * FROM partitioned_table4 WHERE dt = date_format(date_add('day', -1, now()), '%Y%m%d') AND ...) dd
ON
aa.some_id = dd.some_id
CROSS JOIN
(
SELECT
...
FROM
non_partitioned_table
WHERE
yyyymmdd <= date_format(date_add('day', -1, now()), '%Y%m%d')
AND
(end_date_of_month_yn = 'Y' OR yyyymmdd = date_format(date_add('day', -1, now()), '%Y%m%d'))
) t
) m
JOIN
(
SELECT
substring(dt,1,6) AS yyyymm,
some_id,
sum(IF(platform='ALL', pv, 0)) AS pv_total
FROM
partitioned_table5
GROUP BY
substring(dt,1,6),
some_id
) a
ON
m.yyyymm = a.yyyymm
AND
m.some_id = a.some_id
LEFT JOIN
(
SELECT
substring(dt,1,6) AS yyyymm,
some_id,
sum(IF(platform='ALL', pv, 0)) AS pv_total
FROM
partitioned_table5
GROUP BY
substring(dt,1,6),
some_id
) b
ON
m.yyyymm_1m_ago = b.yyyymm
AND
m.some_id = b.some_id
LEFT JOIN
(
SELECT
substring(dt,1,6) AS yyyymm,
some_id,
count(*) AS cnt
FROM
partitioned_table6
WHERE
status = 1
GROUP BY
substring(dt,1,6),
some_id
) c
ON
m.yyyymm = c.yyyymm
AND
m.some_id = c.some_id
LEFT JOIN
(
SELECT
substring(dt,1,6) AS yyyymm,
some_id,
count(*) AS cnt
FROM
partitioned_table6
WHERE
status = 1
GROUP BY
substring(dt,1,6),
some_id
) d
ON
m.yyyymm_1m_ago = d.yyyymm
AND
m.some_id = d.some_id
LEFT JOIN
partitioned_table7 e
ON
m.yyyymmdd_end = e.dt
AND
m.some_id2 = e.some_id3
LEFT JOIN
partitioned_table7 f
ON
m.yyyymmdd_end_1m_ago = f.dt
AND
m.some_id2 = f.some_id3
WHERE
m.yyyymm = '201801'
Thanks
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 17 (13 by maintainers)
Yes! I confirmed martint:picklayout1 branch solved my problem although
PickTableLayoutForPredicate
metrics are not so different from 0.193.My tables have about 500 - 2000 partitions.
I believe that this will be merged soon and well as 0.194 will be soon released because of that.