[GitHub] [arrow] westonpace commented on a diff in pull request #33909: GH-33899: [C++] Add NamedTapRel relation as a Substrait extension

["westonpace (via GitHub)" <gi...@apache.org>]

westonpace commented on code in PR #33909:
URL: https://github.com/apache/arrow/pull/33909#discussion_r1093867742


##########
cpp/src/arrow/engine/substrait/options.h:
##########
@@ -83,6 +84,14 @@ class ARROW_ENGINE_EXPORT ExtensionProvider {
 
 ARROW_ENGINE_EXPORT std::shared_ptr<ExtensionProvider> default_extension_provider();
 
+struct ARROW_ENGINE_EXPORT NamedTapNodeOptions : public compute::ExecNodeOptions {
+  NamedTapNodeOptions(const std::string& name, std::shared_ptr<Schema> schema)
+      : name(name), schema(std::move(schema)) {}
+
+  std::string name;
+  std::shared_ptr<Schema> schema;
+};

Review Comment:
   If you return exec node options (similar to named tap provider) then you could bypass the need to encode properties into the name (you wouldn't even really need `kind`):
   
   ```
   # in python
   def tap_provider(name):
     if name == 'one':
       return TeeNodeOptions('/tmp/dataset_one')
     elif name == 'two':
       return TeeNodeOptions('/tmp/dataset_two')
     else:
       raise Error(...)
   ```
   
   or you could move the name encoding / decoding into python
   
   ```
   # in python
   def tap_provider(name)
     path = get_path_from_name(name)
     return TeeNodeOptions(path)
   ```
   
   or you could still use the kind mapping
   
   ```
   # in python
   def tap_provider(name)
     kind = get_kind_from_name(name)
     if kind == 'tee':
      return TeeNodeOptions(path)...
   ```
   
   However, this feature is still pretty experimental so I don't mind sticking with kind<->factory name mapping for now if that is what you would prefer.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org