Home / Flink / Query compilation
Flink SQL: Currently the join key in Temporal Table Join can not be empty
org.apache.flink.table.api.ValidationExceptionCurrently the join key in Temporal Table Join can not be empty.Reproduced on Flink 2.3.0, · How
A FOR SYSTEM_TIME AS OF join has no equality between the stream and the versioned or lookup table. Flink needs at least one key equality to know which version or row to fetch; range conditions alone are not enough.
Causes · Fixes · Reproduce · Source
Common causes
- The ON clause only has non-equality conditions, for example d.id > o.cid or a BETWEEN.How to confirm: The ON clause of the FOR SYSTEM_TIME AS OF join contains no "=".
- The equality compares expressions instead of columns of the joined table, which Flink cannot use as a join key.How to confirm: The table side of the equality is wrapped in a function or cast.
How to fix it
Add an equality on a plain column of the joined table; keep range conditions as additional filters.
SELECT o.id, d.name
FROM orders AS o
JOIN dim FOR SYSTEM_TIME AS OF o.proc AS d
ON d.id = o.cid AND d.valid_from <= o.order_time;Reproduce it
Triggered on Flink 2.3.0 in a local Docker container, using the SQL client:
CREATE TABLE orders (id INT, cid INT, proc AS PROCTIME()) WITH ('connector' = 'datagen', 'number-of-rows' = '5');
CREATE TABLE dim (id INT, name STRING) WITH ('connector' = 'filesystem', 'path' = 'file:///tmp/dim', 'format' = 'csv');
SELECT o.id, d.name FROM orders AS o JOIN dim FOR SYSTEM_TIME AS OF o.proc AS d ON d.id > o.cid;Output:
org.apache.flink.table.api.ValidationException: Currently the join key in Temporal Table Join can not be empty.[0mWhere the message comes from
LogicalCorrelateToJoinFromTemporalTableRule.scalainapache/flinkatrelease-2.3.0- Message checked in Flink 2.3.0