You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "timsaucer (via GitHub)" <gi...@apache.org> on 2024/07/27 13:26:35 UTC
[PR] Add missing exports for wrapper modules [datafusion-python]
timsaucer opened a new pull request, #782:
URL: https://github.com/apache/datafusion-python/pull/782
# Which issue does this PR close?
This addresses part of https://github.com/apache/datafusion-python/issues/767 but does not close the issue. We still should add wrapper classes for some functions.
# Rationale for this change
Since we have a large change to the python interface, this PR ensures that every exposed class, function, and attribute in the internal module also exists in the corresponding location in the wrapper module. Additionally it adds a python unit test to guarantee that anything in the future exposed in the rust code must have a corresponding entry in the python wrappers.
# What changes are included in this PR?
This PR exports the internal classes that were missing after the python wrapper PR #750 . I do not believe any of these were strictly necessary through all of my testing, but with this PR we have greater confidence that we will not introduce a breaking change to existing customers. For example, if a user is currently importing one of the classes in object store, this PR ensures that import will still work the same.
Also included is a correction on the rust side for the class naming capitalization.
Lastly, added a unit test to check to see if all of the exports exist. This will help future proof the repository so that when we do expose a new feature we must add they corresponding python wrappers.
# Are there any user-facing changes?
This PR should be opaque to the users.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add missing exports for wrapper modules [datafusion-python]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove merged PR #782:
URL: https://github.com/apache/datafusion-python/pull/782
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add missing exports for wrapper modules [datafusion-python]
Posted by "Michael-J-Ward (via GitHub)" <gi...@apache.org>.
Michael-J-Ward commented on code in PR #782:
URL: https://github.com/apache/datafusion-python/pull/782#discussion_r1693992444
##########
python/datafusion/tests/test_wrapper_coverage.py:
##########
@@ -0,0 +1,49 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import datafusion
+import datafusion.functions
+import datafusion.object_store
+import datafusion.substrait
+
+
+def missing_exports(internal_obj, wrapped_obj) -> None:
+ for attr in dir(internal_obj):
+ assert attr in dir(wrapped_obj)
+
+ internal_attr = getattr(internal_obj, attr)
+ wrapped_attr = getattr(wrapped_obj, attr)
+
+ assert wrapped_attr is not None if internal_attr is not None else True
+
+ if attr in ["__self__", "__class__"]:
+ continue
+ if isinstance(internal_attr, list):
+ assert isinstance(wrapped_attr, list)
+ for val in internal_attr:
+ assert val in wrapped_attr
+ elif hasattr(internal_attr, "__dict__"):
+ missing_exports(internal_attr, wrapped_attr)
+
+
+def test_datafusion_missing_exports() -> None:
Review Comment:
Love this test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org