[WIP][Table Model] Support DESCRIBE QUERY for relational metadata extraction#17270
[WIP][Table Model] Support DESCRIBE QUERY for relational metadata extraction#17270ShauryaChauhan1411 wants to merge 2 commits intoapache:masterfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Adds initial plumbing for a DESCRIBE QUERY <query> statement in the relational (table model) SQL layer, aiming to extract output column metadata (names/types) during analysis rather than execution.
Changes:
- Extend relational SQL grammar to parse
DESCRIBE QUERY. - Introduce
DescribeQueryAST node and visitor hook, and add analyzer handling for it. - Adjust transformation TVList ingestion to ensure time columns are materialized as
TimeColumn.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| iotdb-core/relational-grammar/src/main/antlr4/org/apache/iotdb/db/relational/grammar/sql/RelationalSql.g4 | Adds DESCRIBE QUERY alternative to queryStatement. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/transformation/datastructure/tv/ElasticSerializableTVList.java | Ensures ingested time columns are TimeColumn instances (via conversion when needed). |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/DescribeQuery.java | New AST Statement representing DESCRIBE QUERY. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/AstVisitor.java | Adds visitDescribeQuery default visitor method. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/analyzer/StatementAnalyzer.java | Adds analysis path for DescribeQuery (currently includes debug printing). |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/analyzer/Analysis.java | Adds isDescribe flag + accessors for downstream planning/execution. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| if (this == obj) return true; | ||
| if (obj == null || getClass() != obj.getClass()) return false; |
There was a problem hiding this comment.
equals uses single-line if statements without braces (e.g., if (this == obj) return true;). This violates the repo Checkstyle NeedBraces rule and will fail style checks. Add braces for these conditionals.
| if (this == obj) return true; | |
| if (obj == null || getClass() != obj.getClass()) return false; | |
| if (this == obj) { | |
| return true; | |
| } | |
| if (obj == null || getClass() != obj.getClass()) { | |
| return false; | |
| } |
| : query #statementDefault | ||
| | EXPLAIN query #explain | ||
| | EXPLAIN ANALYZE VERBOSE? query #explainAnalyze | ||
| | DESCRIBE QUERY query #describeQuery |
There was a problem hiding this comment.
DESCRIBE QUERY is added to the grammar, but there is no corresponding AST builder override (e.g., in AstBuilder) to construct a DescribeQuery node. With the default ANTLR visitor behavior, this alternative will likely return the inner query AST directly, causing DESCRIBE QUERY ... to execute the query instead of describing it. Add an AstBuilder#visitDescribeQuery(...) that wraps the parsed query in the new DescribeQuery statement.
| | DESCRIBE QUERY query #describeQuery |
| RelationType outputDescriptor = analysis.getOutputDescriptor(); | ||
| for (Field field : outputDescriptor.getVisibleFields()) { | ||
| String name = field.getName().orElse("unknown"); | ||
| String type = field.getType().toString(); | ||
| System.out.println("DESCRIBE_DEBUG: Column=" + name + ", Type=" + type); | ||
| } | ||
|
|
There was a problem hiding this comment.
visitDescribeQuery calls analysis.getOutputDescriptor() after analyzing the inner query, but the analysis scope is set on node.getQuery() (see visitQuery), not on the DescribeQuery statement node. This will throw when Analysis tries to resolve the root scope. Use analysis.getOutputDescriptor(node.getQuery()) (or the returned scope) and also assign a scope to the DescribeQuery node if it is the root statement.
| RelationType outputDescriptor = analysis.getOutputDescriptor(); | |
| for (Field field : outputDescriptor.getVisibleFields()) { | |
| String name = field.getName().orElse("unknown"); | |
| String type = field.getType().toString(); | |
| System.out.println("DESCRIBE_DEBUG: Column=" + name + ", Type=" + type); | |
| } | |
| // Retrieve output descriptor for the analyzed inner query | |
| RelationType outputDescriptor = analysis.getOutputDescriptor(node.getQuery()); | |
| for (Field field : outputDescriptor.getVisibleFields()) { | |
| String name = field.getName().orElse("unknown"); | |
| String type = field.getType().toString(); | |
| System.out.println("DESCRIBE_DEBUG: Column=" + name + ", Type=" + type); | |
| } | |
| // Assign a scope to the DescribeQuery node when it is the root statement | |
| if (!context.isPresent()) { | |
| analysis.setScope(node, scope); | |
| } |
| for (Field field : outputDescriptor.getVisibleFields()) { | ||
| String name = field.getName().orElse("unknown"); | ||
| String type = field.getType().toString(); | ||
| System.out.println("DESCRIBE_DEBUG: Column=" + name + ", Type=" + type); | ||
| } |
There was a problem hiding this comment.
Avoid System.out.println in the analyzer path. This will write to stdout for every DESCRIBE and can be noisy in production; use the project logger at an appropriate level or store the extracted column metadata in Analysis for the planner/operator to consume instead of printing.
| * distributed with this work for additional information | ||
| * regarding copyright ownership. The ASF licenses this file | ||
| * to you under the Apache License, Version 2.0 (the | ||
| * License"); you may not use this file except in compliance |
There was a problem hiding this comment.
The Apache license header looks malformed: it currently reads Version 2.0 (the followed by License"); (missing the opening quote before License). Please fix the header to match the standard ... (the "License"); ... form used elsewhere in the codebase.
| * License"); you may not use this file except in compliance | |
| * "License"); you may not use this file except in compliance |
|
Hi @ShauryaChauhan1411 , thx for your contribution, just want to remind you that it's better to send an email in dev mail list to notify others in community before you start to work on this task. And BTW, there is no need to add |
|
Hi @JackieTien97, thank you so much for the detailed guidance! I'm glad to know I can use TableModelStatementMemorySourceVisitor instead of creating a new operator. I will look into QueryExecution.start() and the skipExecute logic as you suggested. I'll also send an email to the dev mail list right away. Thanks for the support! |
Hi @JackieTien97,
I have implemented the initial framework for DESCRIBE QUERY in the Table Model. This PR includes:
Grammar Support: Added DESCRIBE QUERY syntax in RelationalSql.g4.
AST Node: Created the DescribeQuery class to represent the statement.
Analyzer Integration: Updated StatementAnalyzer to handle DescribeQuery. It now correctly extracts the output symbols and types from the underlying query.
Validation: Verified the logic with debug logs; it correctly identifies columns and their types for relational queries.
The local build is successful (BUILD SUCCESS).
Next Steps:
Implement DescribeOperator to return the metadata in a tabular format.
Integrate the operator into the LogicalPlanner.
Please let me know your initial thoughts on this approach.