You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "matthewmturner (via GitHub)" <gi...@apache.org> on 2024/03/11 14:04:40 UTC
[I] Add `name` method to `ExecutionPlan` trait [arrow-datafusion]
matthewmturner opened a new issue, #9558:
URL: https://github.com/apache/arrow-datafusion/issues/9558
### Is your feature request related to a problem or challenge?
In the telemetry we produce on query executions we sometimes need to know the name of `ExecutionPlan` nodes (i.e. `ParquetExec`). Right now we do something like the following to extract the name:
```
// Get the one-line representation of the ExecutionPlan, something like this:
// ParquetExec: file_groups=[...], ...
let mut buf = String::new();
write!(&mut buf, "{}", displayable(plan).one_line()).map_err(|e| {
DataFusionError::Internal(format!("Error while collecting metrics: {e}"))
})?;
let node_type = match buf.split_once(':') {
None => {
warn!("execution plan has unexpected display format: {buf}");
return Ok(true);
}
Some((name, _)) => name.to_string(),
};
```
From what I have seen the name of `ExecutionPlan` is always known at compile time so I think this is unnecessary work and I am hoping to provide a simpler way to get access to this information.
### Describe the solution you'd like
Could we add something like the following to `ExecutionPlan`
```
pub trait ExecutionPlan: Debug + DisplayAs + Send + Sync {
const NAME: &'static str;
fn name(&self) -> &'static str {
return Self::NAME;
}
impl ExecutionPlan for ParquetExec {
const NAME = "ParquetExec";
// Rest of `ParquetExec` is unchanged
....
}
```
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add `name` method to `ExecutionPlan` trait [arrow-datafusion]
Posted by "SteveLauC (via GitHub)" <gi...@apache.org>.
SteveLauC commented on issue #9558:
URL: https://github.com/apache/arrow-datafusion/issues/9558#issuecomment-1988550306
We can probably impl this method using td::any::type_name:
Something like
fn name(&self) -> &str {
let full_name = std::any::type_name::<Self>();
let start_idx = full_name.rfind(':').unwrap() + 1;
&full_name[start_idx..]
}
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add `name` method to `ExecutionPlan` trait [arrow-datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #9558:
URL: https://github.com/apache/arrow-datafusion/issues/9558#issuecomment-2015193954
I think adding a `name()` function to ExecutionPlan sounds like a good idea to me
Requiring a `&static str` would likely be inconvenient for some use cases, but given the API doesn't currently exist I think it would be fine if you wanted to add it.
Note I think you could probably just add a single function and make the change to the trait backwards compatible, for example
```rust
pub trait ExecutionPlan: Debug + DisplayAs + Send + Sync {
/// return the name of this execution plan's type
fn name(&self) -> &'static str { "UNKNOWN" }
...
}
```
The `UserDefinedLogicalNode` has a `fn name(&self) -> &str`: https://docs.rs/datafusion/latest/datafusion/logical_expr/trait.UserDefinedLogicalNode.html#tymethod.name
Which is another possiblity
Another option for an API that avoids string copying could be
```rust
fn name(&self) -> Arc<str> ;
```
To still allow dynamicc names
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add `name` method to `ExecutionPlan` trait [arrow-datafusion]
Posted by "matthewmturner (via GitHub)" <gi...@apache.org>.
matthewmturner commented on issue #9558:
URL: https://github.com/apache/arrow-datafusion/issues/9558#issuecomment-2015051872
@alamb im curious if you have thoughts on this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add `name` method to `ExecutionPlan` trait [arrow-datafusion]
Posted by "matthewmturner (via GitHub)" <gi...@apache.org>.
matthewmturner commented on issue #9558:
URL: https://github.com/apache/arrow-datafusion/issues/9558#issuecomment-2018014421
> Note I think you could probably just add a single function and make the change to the trait backwards compatible, for example
>
> ```rust
> pub trait ExecutionPlan: Debug + DisplayAs + Send + Sync {
> /// return the name of this execution plan's type
> fn name(&self) -> &'static str { "UNKNOWN" }
> ...
> }
> ```
I think thats a great idea.
> Another option for an API that avoids string copying could be
>
> ```rust
> fn name(&self) -> Arc<str> ;
> ```
>
> To still allow dynamicc names
Do you have or know of existing use cases for dynamic names? I had in mind that these names would always be known at compile time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add `name` method to `ExecutionPlan` trait [arrow-datafusion]
Posted by "matthewmturner (via GitHub)" <gi...@apache.org>.
matthewmturner commented on issue #9558:
URL: https://github.com/apache/arrow-datafusion/issues/9558#issuecomment-1988579473
I was hoping to avoid having to do any compute since I believe the name is always known at compile time. Of course my proposal would make this a breaking change and maybe thats not wanted for this type of change. If so, then certainly we could do something as a provided method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add `name` method to `ExecutionPlan` trait [arrow-datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #9558: Add `name` method to `ExecutionPlan` trait
URL: https://github.com/apache/arrow-datafusion/issues/9558
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add `name` method to `ExecutionPlan` trait [arrow-datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #9558:
URL: https://github.com/apache/arrow-datafusion/issues/9558#issuecomment-2018176724
> Do you have or know of existing use cases for dynamic names? I had in mind that these names would always be known at compile time.
I do not know of any use cases for dynamic names
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org