Skip to content

[WIP][Table Model] Support DESCRIBE QUERY for relational metadata extraction#17270

Open
ShauryaChauhan1411 wants to merge 2 commits intoapache:masterfrom
ShauryaChauhan1411:describe-query-feature
Open

[WIP][Table Model] Support DESCRIBE QUERY for relational metadata extraction#17270
ShauryaChauhan1411 wants to merge 2 commits intoapache:masterfrom
ShauryaChauhan1411:describe-query-feature

Conversation

@ShauryaChauhan1411
Copy link
Contributor

Hi @JackieTien97,

I have implemented the initial framework for DESCRIBE QUERY in the Table Model. This PR includes:

Grammar Support: Added DESCRIBE QUERY syntax in RelationalSql.g4.

AST Node: Created the DescribeQuery class to represent the statement.

Analyzer Integration: Updated StatementAnalyzer to handle DescribeQuery. It now correctly extracts the output symbols and types from the underlying query.

Validation: Verified the logic with debug logs; it correctly identifies columns and their types for relational queries.

The local build is successful (BUILD SUCCESS).

Next Steps:

  1. Implement DescribeOperator to return the metadata in a tabular format.

  2. Integrate the operator into the LogicalPlanner.

Please let me know your initial thoughts on this approach.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds initial plumbing for a DESCRIBE QUERY <query> statement in the relational (table model) SQL layer, aiming to extract output column metadata (names/types) during analysis rather than execution.

Changes:

  • Extend relational SQL grammar to parse DESCRIBE QUERY.
  • Introduce DescribeQuery AST node and visitor hook, and add analyzer handling for it.
  • Adjust transformation TVList ingestion to ensure time columns are materialized as TimeColumn.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
iotdb-core/relational-grammar/src/main/antlr4/org/apache/iotdb/db/relational/grammar/sql/RelationalSql.g4 Adds DESCRIBE QUERY alternative to queryStatement.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/transformation/datastructure/tv/ElasticSerializableTVList.java Ensures ingested time columns are TimeColumn instances (via conversion when needed).
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/DescribeQuery.java New AST Statement representing DESCRIBE QUERY.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/AstVisitor.java Adds visitDescribeQuery default visitor method.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/analyzer/StatementAnalyzer.java Adds analysis path for DescribeQuery (currently includes debug printing).
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/analyzer/Analysis.java Adds isDescribe flag + accessors for downstream planning/execution.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +58 to +59
if (this == obj) return true;
if (obj == null || getClass() != obj.getClass()) return false;
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

equals uses single-line if statements without braces (e.g., if (this == obj) return true;). This violates the repo Checkstyle NeedBraces rule and will fail style checks. Add braces for these conditionals.

Suggested change
if (this == obj) return true;
if (obj == null || getClass() != obj.getClass()) return false;
if (this == obj) {
return true;
}
if (obj == null || getClass() != obj.getClass()) {
return false;
}

Copilot uses AI. Check for mistakes.
: query #statementDefault
| EXPLAIN query #explain
| EXPLAIN ANALYZE VERBOSE? query #explainAnalyze
| DESCRIBE QUERY query #describeQuery
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DESCRIBE QUERY is added to the grammar, but there is no corresponding AST builder override (e.g., in AstBuilder) to construct a DescribeQuery node. With the default ANTLR visitor behavior, this alternative will likely return the inner query AST directly, causing DESCRIBE QUERY ... to execute the query instead of describing it. Add an AstBuilder#visitDescribeQuery(...) that wraps the parsed query in the new DescribeQuery statement.

Suggested change
| DESCRIBE QUERY query #describeQuery

Copilot uses AI. Check for mistakes.
Comment on lines +859 to +865
RelationType outputDescriptor = analysis.getOutputDescriptor();
for (Field field : outputDescriptor.getVisibleFields()) {
String name = field.getName().orElse("unknown");
String type = field.getType().toString();
System.out.println("DESCRIBE_DEBUG: Column=" + name + ", Type=" + type);
}

Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

visitDescribeQuery calls analysis.getOutputDescriptor() after analyzing the inner query, but the analysis scope is set on node.getQuery() (see visitQuery), not on the DescribeQuery statement node. This will throw when Analysis tries to resolve the root scope. Use analysis.getOutputDescriptor(node.getQuery()) (or the returned scope) and also assign a scope to the DescribeQuery node if it is the root statement.

Suggested change
RelationType outputDescriptor = analysis.getOutputDescriptor();
for (Field field : outputDescriptor.getVisibleFields()) {
String name = field.getName().orElse("unknown");
String type = field.getType().toString();
System.out.println("DESCRIBE_DEBUG: Column=" + name + ", Type=" + type);
}
// Retrieve output descriptor for the analyzed inner query
RelationType outputDescriptor = analysis.getOutputDescriptor(node.getQuery());
for (Field field : outputDescriptor.getVisibleFields()) {
String name = field.getName().orElse("unknown");
String type = field.getType().toString();
System.out.println("DESCRIBE_DEBUG: Column=" + name + ", Type=" + type);
}
// Assign a scope to the DescribeQuery node when it is the root statement
if (!context.isPresent()) {
analysis.setScope(node, scope);
}

Copilot uses AI. Check for mistakes.
Comment on lines +860 to +864
for (Field field : outputDescriptor.getVisibleFields()) {
String name = field.getName().orElse("unknown");
String type = field.getType().toString();
System.out.println("DESCRIBE_DEBUG: Column=" + name + ", Type=" + type);
}
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid System.out.println in the analyzer path. This will write to stdout for every DESCRIBE and can be noisy in production; use the project logger at an appropriate level or store the extracted column metadata in Analysis for the planner/operator to consume instead of printing.

Copilot uses AI. Check for mistakes.
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* License"); you may not use this file except in compliance
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Apache license header looks malformed: it currently reads Version 2.0 (the followed by License"); (missing the opening quote before License). Please fix the header to match the standard ... (the "License"); ... form used elsewhere in the codebase.

Suggested change
* License"); you may not use this file except in compliance
* "License"); you may not use this file except in compliance

Copilot uses AI. Check for mistakes.
@JackieTien97
Copy link
Contributor

Hi @ShauryaChauhan1411 , thx for your contribution, just want to remind you that it's better to send an email in dev mail list to notify others in community before you start to work on this task.

And BTW, there is no need to add DescribeOperator, you can refer to TableModelStatementMemorySourceVisitor, and you should skip execution(including logical plan) after analyze phase. The first skipExecute call in QueryExecution.start() should return true, then the result should be constructed through constructResultForMemorySource.

@ShauryaChauhan1411
Copy link
Contributor Author

Hi @JackieTien97, thank you so much for the detailed guidance! I'm glad to know I can use TableModelStatementMemorySourceVisitor instead of creating a new operator. I will look into QueryExecution.start() and the skipExecute logic as you suggested. I'll also send an email to the dev mail list right away. Thanks for the support!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants