hudi: [SUPPORT] Trino can't read tables created by Flink Hudi conector

I am creating a hudi table using Flink Hudi connector

CREATE TABLE flink.flink_hudi_hms3 (
  uuid VARCHAR(20),
  name VARCHAR(10),
  age INT,
  ts TIMESTAMP(3),
  `partition` VARCHAR(20)
)
PARTITIONED BY (`partition`)
WITH (
  'connector' = 'hudi',
  'path' =  'abfs://flink@test.dfs.core.windows.net/hudi/t1hms3',
  'table.type' = 'COPY_ON_WRITE',  
  'hive_sync.enable' = 'true',    
  'hive_sync.mode' = 'hms',     
  'hive_sync.metastore.uris' = 'thrift://hive-metastore:9083' 
);

I Insert data into the table

INSERT INTO  flink_hudi_hms3 VALUES
  ('id1','Danny',23,TIMESTAMP '1970-01-01 00:00:01','par1'),
  ('id2','Stephen',33,TIMESTAMP '1970-01-01 00:00:02','par1'),
  ('id3','Julian',53,TIMESTAMP '1970-01-01 00:00:03','par2'),
  ('id4','Fabian',31,TIMESTAMP '1970-01-01 00:00:04','par2'),
  ('id5','Sophia',18,TIMESTAMP '1970-01-01 00:00:05','par3'),
  ('id6','Emma',20,TIMESTAMP '1970-01-01 00:00:06','par3'),
  ('id7','Bob',44,TIMESTAMP '1970-01-01 00:00:07','par4'),
  ('id8','Han',56,TIMESTAMP '1970-01-01 00:00:08','par4');

I can see the registered table in Trino, but when I try to read data from the table, I get the following error

io.trino.spi.TrinoException: line 1:8: SELECT * not allowed from relation that has no columns
	at io.trino.sql.analyzer.SemanticExceptions.semanticException(SemanticExceptions.java:48)
	at io.trino.sql.analyzer.SemanticExceptions.semanticException(SemanticExceptions.java:43)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.analyzeSelectAllColumns(StatementAnalyzer.java:4249)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.analyzeSelect(StatementAnalyzer.java:4185)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuerySpecification(StatementAnalyzer.java:2913)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuerySpecification(StatementAnalyzer.java:479)
	at io.trino.sql.tree.QuerySpecification.accept(QuerySpecification.java:155)
	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:496)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:504)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuery(StatementAnalyzer.java:1470)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuery(StatementAnalyzer.java:479)
	at io.trino.sql.tree.Query.accept(Query.java:107)
	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:496)
	at io.trino.sql.analyzer.StatementAnalyzer.analyze(StatementAnalyzer.java:458)
	at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:79)
	at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:71)
	at io.trino.execution.SqlQueryExecution.analyze(SqlQueryExecution.java:268)
	at io.trino.execution.SqlQueryExecution.<init>(SqlQueryExecution.java:203)
	at io.trino.execution.SqlQueryExecution$SqlQueryExecutionFactory.createQueryExecution(SqlQueryExecution.java:850)
	at io.trino.dispatcher.LocalDispatchQueryFactory.lambda$createDispatchQuery$0(LocalDispatchQueryFactory.java:149)
	at io.trino.$gen.Trino_0_410_0_1_0_5_2____20230812_012523_2.call(Unknown Source)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

Show create table in Trino returns no details on the table.

trino:flink> show create table flink_hudi_hms3;
               Create Table
-------------------------------------------
 CREATE TABLE hive.flink.flink_hudi_hms3 (

 )
(1 row)

What am I missing here? I tried reading this from Trino Hive catalog above, then I use Hudi connector - I get the error

USE
trino:flink> select * from  flink_hudi_hms3;
Query 20230812_014715_00026_st296 failed: Not a Hudi table: flink.flink_hudi_hms3 

About this issue

  • Original URL
  • State: open
  • Created a year ago
  • Comments: 23 (11 by maintainers)

Most upvoted comments

Hudi could work with Hadoop 3.3.2.

do you mean the Flink lib directory in the JM/TM

Yes

I didn’t write that in hudi website, will fire a fix for it.

HMS props for the Hudi table creating using Flink SQL

You are using the Flink Hive catalog, the table are actually created by the hive catalog. Actually we have a separate Hudi hive catalog instead, the syntax looks like:

  CREATE CATALOG hoodie_catalog
  WITH (
    'type'='hudi',
    'catalog.path' = '${catalog root path}',
    'hive.conf.dir' = '${hive-site.xml dir}',
    'mode'='hms'
  );

The error log in JM indicates a missing calcite-core jar, you can fix it by adding it to the classpath.