Enhance unnest support in decoupled mode #17550
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| return makeIdentityMapping(input); | ||
| } | ||
|
|
||
| protected TrimResult dummyProject(int fieldCount, RelNode input, |
Check notice
Code scanning / CodeQL
Missing Override annotation
| return result(input, mapping); | ||
| } | ||
|
|
||
| public TrimResult trimFields( |
Check notice
Code scanning / CodeQL
Deprecated method or constructor invocation Note test
sql/src/main/java/org/apache/druid/sql/calcite/planner/DruidRelFieldTrimmer.java
Fixed
Show fixed
Hide fixed
sql/src/main/java/org/apache/druid/sql/calcite/planner/DruidRelFieldTrimmer.java
Fixed
Show fixed
Hide fixed
github-actions
bot
added
Area - Batch Ingestion
Area - MSQ
…mer-unnestfieldtype
| import java.util.List; | ||
| import java.util.Set; | ||
|
|
||
| public class DruidRelFieldTrimmer extends RelFieldTrimmer |
There was a problem hiding this comment.
A javadoc for this class would be really useful
There was a problem hiding this comment.
added a few lines; main goal of the class is the same as RelFieldTrimmer - so I've added a link to it in the apidoc as all of those also applies here
| .columnTypes(ColumnType.LONG) | ||
| .context(defaultScanQueryContext( | ||
| queryContext, | ||
| RowSignature.builder().add("v0", ColumnType.LONG).build() | ||
| RowSignature.builder().add("__time", ColumnType.LONG).build() |
There was a problem hiding this comment.
I did not understand why the column name changes here. I don't see any other change to the test
There was a problem hiding this comment.
no columns are needed from the left hand side; but Calcite has some tweaks here and there to avoid relnodes with 0 columns.
This PR suppresses the introduction of a column with the 0 value by the Fieldtrimmer
but in one of the rules there is also an unconditional projection of the 1st columns
which causes this change.
We should be able to handle the case of empty columns - I wanted to dig into that more deeply ; as it seems like there are also some rule combinations which may lead to an empty column set...(but forgot the testcase)
I think that in general Calcite should be able to handle these things - and we should only make
corrections in the execution engine if it causes issues.
sql/src/test/java/org/apache/druid/sql/calcite/CalciteJoinQueryTest.java
Show resolved
Hide resolved
| @@ -92,7 +92,8 @@ enum Modes | |||
| SORT_REMOVE_CONSTANT_KEYS_CONFLICT(DruidException.class, "not enough rules"), | |||
| REQUIRE_TIME_CONDITION(CannotBuildQueryException.class, "requireTimeCondition is enabled"), | |||
| UNNEST_INLINED(Exception.class, "Missing conversion is Uncollect"), | |||
| UNNEST_RESULT_MISMATCH(AssertionError.class, "(Result count mismatch|column content mismatch)"); | |||
| UNNEST_RESULT_MISMATCH(AssertionError.class, "(Result count mismatch|column content mismatch)"), | |||
| RESULT_MISMATCH_NATIVE_UNNEST_INCORRECT_RESULTS(Throwable.class, "(Result count mismatch|column content mismatch|ARRAY)"); | |||
There was a problem hiding this comment.
Is this required? Won't it be covered by the previous one?
LogicalCorrelateandUnnest-s