You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2026/09/24 09:25:27 UTC
[I] Native UDFs: config to disable or restrict library loading [datafusion-comet]
andygrove opened a new issue, #6175:
URL: https://github.com/apache/datafusion-comet/issues/6175
`CometNativeUDF.register` loads any shared library at any path the caller names, on the driver and
then on every executor, with the full privileges of the JVM process. There is no configuration an
operator can set to turn the feature off or to limit which paths may be loaded.
Options:
- `spark.comet.nativeUdf.enabled`, checked in `register` on the driver and in the planner on the
executor, so a plan built elsewhere cannot bypass it. Whether it defaults to on (the feature is
already opt-in by API call) or off (loading native code should need an operator's consent) is
the main question for this issue.
- `spark.comet.nativeUdf.allowedPaths`, a list of directory prefixes. Paths would be canonicalized
before the check so `..` and symlinks cannot escape it.
The executor-side check matters as much as the driver-side one: the library path travels in the
`NativeScalarUdf` proto, so the executor must not trust that the driver checked it.
Follow-up to #4459.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Native UDFs: config to disable or restrict library loading [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #6175: Native UDFs: config to disable or restrict library loading
URL: https://github.com/apache/datafusion-comet/issues/6175
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org