You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2024/05/12 09:46:17 UTC
[I] Ah, I think you are asking about getting `LogicalPlan` from `Arc` [datafusion]
alamb opened a new issue, #10465:
URL: https://github.com/apache/datafusion/issues/10465
Ah, I think you are asking about getting `LogicalPlan` from `Arc<LogicalPlan>`
One way to do it today is using `unwrap_arc` (source below) which will only copy when necessary
```rust
// the builder accepts LogicalPlan only, so a clone needed
let mut builder = LogicalPlanBuilder::from(unwrap_arc(plan));
// making an aggregate plan
builder.aggregate(..);
// this will call Arc::new() on the LogicalPlan contained.
```
https://github.com/apache/datafusion/blob/1eff714ef8356dc305047386ba250b62bed6a795/datafusion/expr/src/logical_plan/tree_node.rs#L368
Separately, adding an `LogicalPlanBuilder::from(input_plan_arc)` would be a great idea. I'll file a ticket about doing so
_Originally posted by @alamb in https://github.com/apache/datafusion/pull/10460#discussion_r1597600173_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Implement `LogicalPlanBuilder::from` for `Arc` [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #10465: Implement `LogicalPlanBuilder::from` for `Arc<LogicalPlan>`
URL: https://github.com/apache/datafusion/issues/10465
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Implement `LogicalPlanBuilder::from` for `Arc` [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #10465:
URL: https://github.com/apache/datafusion/issues/10465#issuecomment-2106377308
That is an interesting point @ClSlaid
So perhaps you are suggesting `LogicalPlanBuilder` has an `Arc<LogicalPlan>` as its internal state (rather than a `LogicalPlan`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Implement `LogicalPlanBuilder::from` for `Arc` [datafusion]
Posted by "ClSlaid (via GitHub)" <gi...@apache.org>.
ClSlaid commented on issue #10465:
URL: https://github.com/apache/datafusion/issues/10465#issuecomment-2106192785
I suggest not call `unwrap_arc` until the built `LogicalPlan` really requires the ownership.
In most occasions, `LogicalPlan` contains an `input: Arc<LogicalPlan>`, and sometimes we will have to construct `LogicalPlanBuilder` from it. This will lead to a clone.
Meanwhile, most of the `LogicalPlan` constructed by the builder places the plan in an `Arc` again. This then causes another clone.
There must be a way to seamlessly moving the `input: Arc<LogicalPlan>` into rewritten plans, and it would be no clone in most occasions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Implement `LogicalPlanBuilder::from` for `Arc` [datafusion]
Posted by "ClSlaid (via GitHub)" <gi...@apache.org>.
ClSlaid commented on issue #10465:
URL: https://github.com/apache/datafusion/issues/10465#issuecomment-2106599908
> So perhaps you are suggesting `LogicalPlanBuilder` has an `Arc<LogicalPlan>` as its internal state (rather than a `LogicalPlan`)
This is surely the most straight forward way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Implement `LogicalPlanBuilder::from` for `Arc` [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #10465:
URL: https://github.com/apache/datafusion/issues/10465#issuecomment-2107222678
Great idea @ClSlaid
Thanks to @AbrarNitk we have a first version of `LogicalPlanBuilder::from(arc_input)` in https://github.com/apache/datafusion/pull/10466 🙏
I think we should merge that PR and then make a second PR (and maybe a second ticket) about switching the internal representation from https://github.com/apache/datafusion/blob/8cc92a921f3b0296caf035edb66a4b3c422f084d/datafusion/expr/src/logical_plan/builder.rs#L98-L101
To
```rust
pub struct LogicalPlanBuilder {
plan: Arc<LogicalPlan>,
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Implement `LogicalPlanBuilder::from` for `Arc` [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #10465:
URL: https://github.com/apache/datafusion/issues/10465#issuecomment-2107579559
Filed https://github.com/apache/datafusion/issues/10485 to track the internal representation change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org