singhpk234 commented on issue #7502:
URL: https://github.com/apache/iceberg/issues/7502#issuecomment-1532501988
This is an interesting observation, can you please elaborate more on steps to re-produce for example how is `failures` table generated, and attach complete plans ?
One interesting obv is the same view i.e view_failure_events has diff plans in diff subtrees.
Subtree 1 :
```
:- SubqueryAlias s
:- SubqueryAlias s
: +- SubqueryAlias view_failure_events
: +- SubqueryAlias view_failure_events
: +- View (`view_failure_events`, [event_id#554])
: +- View (`view_failure_events`, [event_id#554])
: +- Deduplicate [event_id#554]
: +- Deduplicate [event_id#554]
: +- Project [event_id#554]
: +- Project [event_id#554]
: +- Project [failure_id#555, profile_id#553, event_id#554, weight#556, time_to_fail#557]
: +- Project [failure_id#555, profile_id#553, event_id#554, weight#556, time_to_fail#557]
: +- Join Inner, (failure_id#555 = failure_id#538)
: +- Join Inner, (failure_id#555 = failure_id#538)
: :- Filter (profile_id#553 = 57e18aee-6038-4557-b42d-042739a2ccad)
: :- Filter (profile_id#553 = 57e18aee-6038-4557-b42d-042739a2ccad)
: : +- SubqueryAlias iceberg.demo.__bdg_ano_event_failure
: : +- SubqueryAlias iceberg.demo.__bdg_ano_event_failure
: : +- RelationV2[profile_id#553, event_id#554, failure_id#555, weight#556, time_to_fail#557] iceberg.demo.__bdg_ano_event_failure
: : +- RelationV2[profile_id#553, event_id#554, failure_id#555, weight#556, time_to_fail#557] iceberg.demo.__bdg_ano_event_failure
: +- Project [failure_id#538]
: +- Project [failure_id#538]
: +- Filter ((profile_id#536 = 57e18aee-6038-4557-b42d-042739a2ccad) AND (failure_date#539 >= cast(2023-02-06 as date)))
: +- Filter ((profile_id#536 = 57e18aee-6038-4557-b42d-042739a2ccad) AND (failure_date#539 >= cast(2023-02-06 as date)))
: +- SubqueryAlias iceberg.demo.core__fct_failures
```
Subtree 2 :
```
+- SubqueryAlias s
+- SubqueryAlias view_failure_events
+- SubqueryAlias view_failure_events
! +- View (`view_failure_events`, [event_id#554,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,
failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538]) +- View (`view_failure_events`, [event_id#554,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538
,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538,failure_id#538...,failure_id#538])
+- Deduplicate [event_id#554]
+- Deduplicate [event_id#554]
! +- Project [event_id#554, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, ... 75 more fields]
+- Project [event_id#554, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, ... 76 more fields]
! +- Project [failure_id#555, profile_id#553, event_id#554, weight#556, time_to_fail#557, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, ... 79 more fields]
+- Project [failure_id#555, profile_id#553, event_id#554, weight#556, time_to_fail#557, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, failure_id#538, ... 80 more fields]
+- Join Inner, (failure_id#555 = failure_id#538)
+- Join Inner, (failure_id#555 = failure_id#538)
:- Filter (profile_id#553 = 57e18aee-6038-4557-b42d-042739a2ccad)
:- Filter (profile_id#553 = 57e18aee-6038-4557-b42d-042739a2ccad)
: +- SubqueryAlias iceberg.demo.__bdg_ano_event_failure
: +- SubqueryAlias iceberg.demo.__bdg_ano_event_failure
: +- RelationV2[profile_id#553, event_id#554, failure_id#555, weight#556, time_to_fail#557, _spec_id#563, _partition#564, _file#565, _pos#566L, _deleted#567] iceberg.demo.__bdg_ano_event_failure
: +- RelationV2[profile_id#553, event_id#554, failure_id#555, weight#556, time_to_fail#557, _spec_id#563, _partition#564, _file#565, _pos#566L, _deleted#567] iceberg.demo.__bdg_ano_event_failure
+- Project [failure_id#538, _spec_id#547, _partition#548, _file#549, _pos#550L, _deleted#551]
+- Project [failure_id#538, _spec_id#547, _partition#548, _file#549, _pos#550L, _deleted#551]
...
...
```
can you please do a select * and df explain on the view `view_failure_events` with extended mode as true in explain ?
spark.sql("select * from view_failure_events).explain(true);
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org