Skip to content

[regression] simplify iceberg table cache schema-change checks#61066

Open
xylaaaaa wants to merge 2 commits intoapache:masterfrom
xylaaaaa:simplify-iceberg-table-cache-schema-cases
Open

[regression] simplify iceberg table cache schema-change checks#61066
xylaaaaa wants to merge 2 commits intoapache:masterfrom
xylaaaaa:simplify-iceberg-table-cache-schema-cases

Conversation

@xylaaaaa
Copy link
Contributor

@xylaaaaa xylaaaaa commented Mar 5, 2026

Summary

  • simplify schema-change section in test_iceberg_table_cache
  • keep ADD COLUMN, RENAME COLUMN, ALTER COLUMN TYPE
  • remove DROP COLUMN subcase to reduce heavy external Spark calls

Why

  • keep key schema-cache coverage (including column-count increase path)
  • reduce externregression runtime with a moderate cut instead of removing all column add/drop checks

Impact

  • spark_iceberg calls reduced from 48 to about 44 in this case

Test

  • not run in local environment (pipeline validation expected)

Copilot AI review requested due to automatic review settings March 5, 2026 07:19
@Thearas
Copy link
Contributor

Thearas commented Mar 5, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Simplifies the Iceberg external table cache regression test by reducing the number of schema-evolution scenarios executed via Spark, aiming to keep representative cache-behavior coverage while lowering external-call cost in externregression.

Changes:

  • Removed schema-change subcases for external ADD COLUMN and DROP COLUMN.
  • Kept schema-change coverage for external RENAME COLUMN and ALTER COLUMN TYPE with cache vs no-cache verification.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 241 to 245
// Test 2.2: ALTER COLUMN TYPE
logger.info("--- Test 2.2: External ALTER COLUMN TYPE ---")
spark_iceberg "DROP TABLE IF EXISTS demo.${testDb}.test_alter_type"
spark_iceberg "CREATE TABLE demo.${testDb}.test_alter_type (id INT, value INT) USING iceberg"
spark_iceberg "INSERT INTO demo.${testDb}.test_alter_type VALUES (1, 100)"
Copy link

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the ALTER COLUMN TYPE test, the cached-schema assertion uses contains("int"), but the expected new type bigint also contains the substring int. This can make the test pass even if the cache is already refreshed (or not caching). Use an exact comparison (e.g., equalsIgnoreCase("int")) or explicitly assert that the type is not bigint for the cached/no-refresh case (and similarly tighten the initial schema check if you want it to fail fast).

Copilot uses AI. Check for mistakes.
@xylaaaaa
Copy link
Contributor Author

xylaaaaa commented Mar 5, 2026

run buildall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants