presto: Infinite loop during planning in 0.193

Hi,

I use presto 0.193 + HDP 2.5.3 + hive connector.

Infinite loop during planning for the following query occurs.

This error doesn’t occur in presto 0.192.

According to the 0.193 release note(https://prestodb.io/docs/current/release/release-0.193.html), Infinite loop has already been fixed, but it seems not to fix.

Fix an infinite loop during planning for queries containing non-trivial predicates.

I’m sorry not to create simple query which reproduces the issue…

SELECT
...
FROM
(
    SELECT
...
    FROM
        (SELECT * FROM partitioned_table1 WHERE dt = date_format(date_add('day', -1, now()), '%Y%m%d') AND ...) aa
    JOIN
        (SELECT * FROM partitioned_table2 WHERE dt = date_format(date_add('day', -1, now()), '%Y%m%d')) bb
    ON
        aa.some_id = bb.id
    JOIN    
        (SELECT * FROM partitioned_table3 WHERE dt = date_format(date_add('day', -1, now()), '%Y%m%d') AND ...) cc
    ON
        aa.some_id = cc.id
    JOIN
        (SELECT * FROM partitioned_table4 WHERE dt = date_format(date_add('day', -1, now()), '%Y%m%d') AND ...) dd
    ON
        aa.some_id = dd.some_id
    CROSS JOIN
        (
            SELECT
...
            FROM
                non_partitioned_table
            WHERE
                yyyymmdd <= date_format(date_add('day', -1, now()), '%Y%m%d')
            AND
                (end_date_of_month_yn = 'Y' OR yyyymmdd = date_format(date_add('day', -1, now()), '%Y%m%d'))
        ) t
) m
JOIN
(
    SELECT
        substring(dt,1,6) AS yyyymm,
        some_id,
        sum(IF(platform='ALL', pv, 0)) AS pv_total
    FROM 
        partitioned_table5
    GROUP BY
        substring(dt,1,6),
        some_id
) a
ON
    m.yyyymm = a.yyyymm
AND
    m.some_id = a.some_id
LEFT JOIN
(
    SELECT
        substring(dt,1,6) AS yyyymm,
        some_id,
        sum(IF(platform='ALL', pv, 0)) AS pv_total
    FROM 
        partitioned_table5
    GROUP BY
        substring(dt,1,6),
        some_id
) b
ON
    m.yyyymm_1m_ago = b.yyyymm
AND
    m.some_id = b.some_id
LEFT JOIN
(
    SELECT
        substring(dt,1,6) AS yyyymm,
        some_id,
        count(*) AS cnt
    FROM
        partitioned_table6
    WHERE
        status = 1
    GROUP BY
        substring(dt,1,6),
        some_id
) c
ON
    m.yyyymm = c.yyyymm
AND
    m.some_id = c.some_id
LEFT JOIN
(
    SELECT
        substring(dt,1,6) AS yyyymm,
        some_id,
        count(*) AS cnt
    FROM
        partitioned_table6
    WHERE
        status = 1
    GROUP BY
        substring(dt,1,6),
        some_id
) d
ON
    m.yyyymm_1m_ago = d.yyyymm
AND
    m.some_id = d.some_id
LEFT JOIN
    partitioned_table7 e
ON
    m.yyyymmdd_end = e.dt
AND
    m.some_id2 = e.some_id3
LEFT JOIN
    partitioned_table7 f
ON
    m.yyyymmdd_end_1m_ago = f.dt
AND
    m.some_id2 = f.some_id3
WHERE
    m.yyyymm = '201801'

Thanks

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 17 (13 by maintainers)

Most upvoted comments

Yes! I confirmed martint:picklayout1 branch solved my problem although PickTableLayoutForPredicate metrics are not so different from 0.193.

My tables have about 500 - 2000 partitions.

I believe that this will be merged soon and well as 0.194 will be soon released because of that.