hudi: [SUPPORT] Trino can't read tables created by Flink Hudi conector
I am creating a hudi table using Flink Hudi connector
CREATE TABLE flink.flink_hudi_hms3 (
uuid VARCHAR(20),
name VARCHAR(10),
age INT,
ts TIMESTAMP(3),
`partition` VARCHAR(20)
)
PARTITIONED BY (`partition`)
WITH (
'connector' = 'hudi',
'path' = 'abfs://flink@test.dfs.core.windows.net/hudi/t1hms3',
'table.type' = 'COPY_ON_WRITE',
'hive_sync.enable' = 'true',
'hive_sync.mode' = 'hms',
'hive_sync.metastore.uris' = 'thrift://hive-metastore:9083'
);
I Insert data into the table
INSERT INTO flink_hudi_hms3 VALUES
('id1','Danny',23,TIMESTAMP '1970-01-01 00:00:01','par1'),
('id2','Stephen',33,TIMESTAMP '1970-01-01 00:00:02','par1'),
('id3','Julian',53,TIMESTAMP '1970-01-01 00:00:03','par2'),
('id4','Fabian',31,TIMESTAMP '1970-01-01 00:00:04','par2'),
('id5','Sophia',18,TIMESTAMP '1970-01-01 00:00:05','par3'),
('id6','Emma',20,TIMESTAMP '1970-01-01 00:00:06','par3'),
('id7','Bob',44,TIMESTAMP '1970-01-01 00:00:07','par4'),
('id8','Han',56,TIMESTAMP '1970-01-01 00:00:08','par4');
I can see the registered table in Trino, but when I try to read data from the table, I get the following error
io.trino.spi.TrinoException: line 1:8: SELECT * not allowed from relation that has no columns
at io.trino.sql.analyzer.SemanticExceptions.semanticException(SemanticExceptions.java:48)
at io.trino.sql.analyzer.SemanticExceptions.semanticException(SemanticExceptions.java:43)
at io.trino.sql.analyzer.StatementAnalyzer$Visitor.analyzeSelectAllColumns(StatementAnalyzer.java:4249)
at io.trino.sql.analyzer.StatementAnalyzer$Visitor.analyzeSelect(StatementAnalyzer.java:4185)
at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuerySpecification(StatementAnalyzer.java:2913)
at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuerySpecification(StatementAnalyzer.java:479)
at io.trino.sql.tree.QuerySpecification.accept(QuerySpecification.java:155)
at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:496)
at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:504)
at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuery(StatementAnalyzer.java:1470)
at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuery(StatementAnalyzer.java:479)
at io.trino.sql.tree.Query.accept(Query.java:107)
at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:496)
at io.trino.sql.analyzer.StatementAnalyzer.analyze(StatementAnalyzer.java:458)
at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:79)
at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:71)
at io.trino.execution.SqlQueryExecution.analyze(SqlQueryExecution.java:268)
at io.trino.execution.SqlQueryExecution.<init>(SqlQueryExecution.java:203)
at io.trino.execution.SqlQueryExecution$SqlQueryExecutionFactory.createQueryExecution(SqlQueryExecution.java:850)
at io.trino.dispatcher.LocalDispatchQueryFactory.lambda$createDispatchQuery$0(LocalDispatchQueryFactory.java:149)
at io.trino.$gen.Trino_0_410_0_1_0_5_2____20230812_012523_2.call(Unknown Source)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Show create table in Trino returns no details on the table.
trino:flink> show create table flink_hudi_hms3;
Create Table
-------------------------------------------
CREATE TABLE hive.flink.flink_hudi_hms3 (
)
(1 row)
What am I missing here? I tried reading this from Trino Hive catalog above, then I use Hudi connector - I get the error
USE
trino:flink> select * from flink_hudi_hms3;
Query 20230812_014715_00026_st296 failed: Not a Hudi table: flink.flink_hudi_hms3
About this issue
- Original URL
- State: open
- Created a year ago
- Comments: 23 (11 by maintainers)
Hudi could work with Hadoop 3.3.2.
Yes
DOC is here: https://github.com/apache/hudi/pull/9453
I didn’t write that in hudi website, will fire a fix for it.
You are using the Flink Hive catalog, the table are actually created by the hive catalog. Actually we have a separate Hudi hive catalog instead, the syntax looks like:
The error log in JM indicates a missing calcite-core jar, you can fix it by adding it to the classpath.