You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "zyratlo (via GitHub)" <gi...@apache.org> on 2026/01/13 04:40:55 UTC

[D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

GitHub user zyratlo created a discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

## Problem Summary
Currently, modifying values used inside a Python UDF (for example, hyperparameters in an ML training algorithm) requires users to directly modify the UDF's source code. This presents a barrier for non-coding users, who often work with workflows created by other developers and may not feel comfortable editing code. We have received this feedback from many collaborators already.

It would be beneficial to allow workflow developers to expose configurable parameters outside of the UDF source code. This would enable non-technical users to adjust these values through a separate interface without needing to modify or understand the source code.

GitHub link: https://github.com/apache/texera/discussions/4154

----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org


Re: [D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

Posted by "kunwp1 (via GitHub)" <gi...@apache.org>.
GitHub user kunwp1 added a comment to the discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

The current R UDF doesn't support multiple ports, and we will have a separate effort to add this support. The idea should be the same as option 1.

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15490792

----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org


Re: [D] Supporting non-coding users in configuring parameters in Python UDF code [texera]

Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
GitHub user aglinxinyuan added a comment to the discussion: Supporting non-coding users in configuring parameters in Python UDF code

Currently, we have a Text Input operator that outputs each line of input as a tuple, producing a table like the following:
| line | 
| ------------- | 
| string1 | 
| string2 | 

Each table produced by this operator has a single column (named line by default) and multiple rows, where each row corresponds to one line in the text input.

We propose introducing a new operator called Parameter Input, which outputs key–value pairs in the following format:

| key | value | 
| ------------- | ------------- | 
| key1 | value1 |
| key2 | value2 |

For simplicity, both key and value will initially be treated as strings. Although values could conceptually support multiple types (e.g., string, int, double), our current system requires all tuples within the same table to share the same schema. This means that if one value is a string, all values must also be strings. To avoid unnecessary complexity in the initial implementation, we will restrict all values to strings for now.

The plan is to implement the Parameter Input operator first. After that, we can decide how to support file paths—either by extending this operator to handle file paths or by introducing a separate operator specifically for file path inputs.


GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15577187

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
GitHub user Yicong-Huang edited a comment on the discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

I am afraid I would give a -1 for the current proposal.

I want to repeat a previously mentioned idea/concept before we dive into support this feature. In my mind, UDFs are designed for developers. The user of UDF has to understand the code, and because of this, changing a value/parameter in the editor is the easiest method for those users. Non-coding users modifying a value on UDFs is not a recommended user scenario. Instead, the owner/developer of the UDF should go through a "publish" process to convert a UDF to a builtin operator. The new builtin operator should have predefined fields on the property panel, so that non-coding users can modify on the property panel. The property panel is the defined interface where the developer to control what parameters for a non-coding user can/should modify. Our existing plotting operators are great examples of Python UDFs converted to builtin operators. They are easy to use and can only be modified in the defined way. (i.e., a non-coding user cannot arbitrarily change an unwanted field mistakenly).

With this understanding, I am a bit against the proposed idea in this discussion. There are a few reasons:
1. The proposed idea requires non-coding users to define parameters in an upstream operator and pass it to UDF. This is a bit hard to understand and counter intuitive: why am I editing a property of one operator on another operator? 
2. This adds additional complexity:
     - it would be hard for a developer to write: The UDF code would be more complex as it needs to add handling logic for `tuple['parameters']` column. 
     - it would be hard for a developer to maintain. suppose as a developer of UDF, I need to add/remove a parameter, I would need to modify multiple places.
     - it would be hard for a user to use/debug: suppose a UDF does not work as expected and there are a few possible operators passing parameters into this UDF. It would be hard for me as a user to trace the change. 
3. There is no abstract/interface guarantee. Anyone (developer or user) would be able to change code/parameter, and users may make mistakes when changing the code/parameter. Alternatively, if developers can "publish as builtin operator", they can define the parameters and range/scope of the parameters. And the user can only change the defined parameters in a controlled range, so less mistakes.


GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15511624

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Supporting non-coding users in configuring parameters in Python UDF code [texera]

Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
GitHub user aglinxinyuan added a comment to the discussion: Supporting non-coding users in configuring parameters in Python UDF code

In the current system, the Python code is passed from the Frontend → JVM → PVM as a string, allowing us to inject the assignment statement fileName = "movies.csv" into the code on the JVM side.

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15916362

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Supporting non-coding users in configuring parameters in Python UDF code [texera]

Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: Supporting non-coding users in configuring parameters in Python UDF code

Thanks.  Let's have a meeting with me and @aglinxinyuan  and report the plan back.

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15940008

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Supporting non-coding users in configuring parameters in Python UDF code [texera]

Posted by "zyratlo (via GitHub)" <gi...@apache.org>.
GitHub user zyratlo edited a discussion: Supporting non-coding users in configuring parameters in Python UDF code

## Problem Summary
Currently, modifying values used inside a Python UDF (for example, hyperparameters in an ML training algorithm) requires users to directly modify the UDF's source code. This presents a barrier for non-coding users, who often work with workflows created by other developers and may not feel comfortable editing code. We have received this feedback from many collaborators already.

It would be beneficial to allow workflow developers to expose configurable parameters outside of the UDF source code. This would enable non-technical users to adjust these values through a separate interface without needing to modify or understand the source code.

## Possible Solutions

1. Introduce a dedicated input operator for users to provide values, which are passed into the UDF operator
2. Extend the UDF properties panel to include a space for users to provide parameter values

GitHub link: https://github.com/apache/texera/discussions/4154

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
GitHub user aglinxinyuan edited a comment on the discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

Sorry, I don’t understand the plan. First, the title “Feature Proposal” may be confusing, because Option 1 is not actually proposing a new feature. It is simply a way to use the existing system, and no new functionality needs to be added. I understand that Yicong may be against this idea, but we should not restrict users from using the system in a particular way. 

The JVM can send parameters to the PVM as control messages; however, I don’t think sending parameters via control messages contradicts either Option 1 or Option 2. Moreover, sending parameters as control messages does not address or resolve any of the downsides of Option 1 or Option 2 that Yicong mentioned.

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15513932

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
GitHub user Yicong-Huang edited a comment on the discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

For the second option, we previously implemented it already and ended up removed it. The main reason is the users who need to change the parameter would need to 1) define the value name in code, 2) define the value on the property panel. It is not easy to use compared to edit the code directly once. After many users complaining, we removed the feature.

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15511657

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

I also believe option 1 is better.  Here are more insights from the perspective of the engine.

In the Amber engine, the UDF's Python code is sent from JVM to PVM as a **string** to be executed on the PVM.

In option 1, the user-provided parameters are passed from the top-left operator as two **data tuples** to the Python UDF operator.  In the UDF, the Python code for port 0, which runs on PVM, parses the data tuples sent from JVM to retrieve the two parameters.  The Python code for port 1, which also runs on the same PVM, uses the two parameters in its computation.

In option 2, the parameters are provided by the user through the frontend fields of the Python UDF (as part of its property panel).  For these parameters to be obtained by the Python code on PVM, we need to send them from JVM to PVM as a control message, which is doable but not easy.  We could avoid sending a control message by using a "Python code template" on JVM and plugging in the user's parameters, but that's NOT how our engine generates the Python code.

Therefore, option 1 is cleaner and easier to adopt.

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15481647

----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org


Re: [D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
GitHub user aglinxinyuan added a comment to the discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

Do you mean @kunwp1 ?

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15513881

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Supporting non-coding users in configuring parameters in Python UDF code [texera]

Posted by "zyratlo (via GitHub)" <gi...@apache.org>.
GitHub user zyratlo added a comment to the discussion: Supporting non-coding users in configuring parameters in Python UDF code

Thanks for your feedback, I've adjusted the title to more closely match our discussion.

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15521168

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

@Yicong-Huang and I had a discussion about these two design.  Yicong is suggesting the JVM can send the parameters to PVM as control messages.  @aglinxinyuan Do you think if that's possible and easy?  If needed, we can have a discussion and report the results here. 

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15513854

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
GitHub user aglinxinyuan added a comment to the discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

Sorry, I don’t fully understand the plan. First, the title “Feature Proposal” may be confusing, because Option 1 is not actually proposing a new feature. It is simply a way to use the existing system, and no new functionality needs to be added.

I understand that Yicong may be against this idea, but we should not restrict users from using the system in a particular way. The JVM can send parameters to the PVM as control messages; however, this is already part of the system design. I don’t think sending parameters via control messages contradicts either Option 1 or Option 2.

Moreover, sending parameters as control messages does not address or resolve any of the downsides of Option 1 or Option 2 that Yicong mentioned.

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15513932

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Supporting non-coding users in configuring parameters in Python UDF code [texera]

Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
GitHub user carloea2 added a comment to the discussion: Supporting non-coding users in configuring parameters in Python UDF code

<img width="1594" height="601" alt="image" src="https://github.com/user-attachments/assets/771f2c85-1442-4c9a-98cf-f5c41e8d2707" />

@chenlica Here’s what I have working now, end to end, for **Option 2**.

### What the user writes (Python)

Users declare UI parameters once in `open()`, and then use the typed value directly:

```python
from pytexera import *
from typing import Iterator, Optional

class ProcessTupleOperator(UDFOperatorV2):

    @overrides
    def open(self):
        # declare UiParameters once, store the parsed runtime values
        self.value1 = self.UiParameter(name="param1", type=AttributeType.DOUBLE).value
        self.value2 = self.UiParameter(name="param2", type=AttributeType.INT).value
        self.value3 = self.UiParameter(name="param3", type=AttributeType.STRING).value
        self.value4 = self.UiParameter(name="param4", type=AttributeType.TIMESTAMP).value

    @overrides
    def process_tuple(self, tuple_: Tuple, port: int) -> Iterator[Optional[TupleLike]]:
        print(self.value1)
        print(self.value2)
        print(self.value3)
        print(self.value4)
        yield tuple_
```

### What shows up in the UI

From those `self.UiParameter(...)` lines, the property panel automatically generates a **Parameters** section with one row per parameter:

* **Name** + **Type** are read-only (so they stay consistent with the code)
* **Value** is editable (so users can change runtime values without touching the script)

### How the values get into Python

When the workflow runs, we inject the UI values into the UDF and the base class applies them right before `open()` executes. That way, when the user calls `UiParameter(...).value`, they get the current value from the UI.

### I think this is a good approach:

* It’s **safer than passing raw strings around** because the `AttributeType` declaration is actually enforcing the schema as any other property in other operators. We parse and enforce types (int/double/timestamp/etc.) instead of leaving it to user code.
* The UI stays in sync with the script, and users can tweak values without editing code.

### Next improvements

This can be extended pretty naturally with:

* **default values** (so params can be optional and still have sensible behavior)
* more validation (required, ranges, patterns)
* better widgets (timestamp picker, file picker, etc.)

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15938613

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Supporting non-coding users in configuring parameters in Python UDF code [texera]

Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: Supporting non-coding users in configuring parameters in Python UDF code

This description is very clear.  I am thinking about a design in which a value for a key can be provided using a pop-up window to specify a full file path such as "/chenli@gmail.com/imdb-chenli-dataset/v1/imdb-chenli.CSV", similar to the interface of the "CSV File Scan" operator.  The specified value is still of a string type, so this design is not contradicting to the one proposed by @aglinxinyuan .

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15577353

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

Posted by "zyratlo (via GitHub)" <gi...@apache.org>.
GitHub user zyratlo edited a discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

## Problem Summary
Currently, modifying values used inside a Python UDF (for example, hyperparameters in an ML training algorithm) requires users to directly modify the UDF's source code. This presents a barrier for non-coding users, who often work with workflows created by other developers and may not feel comfortable editing code. We have received this feedback from many collaborators already.

It would be beneficial to allow workflow developers to expose configurable parameters outside of the UDF source code. This would enable non-technical users to adjust these values through a separate interface without needing to modify or understand the source code.

## Possible Solutions

1. Introduce a dedicated input operator for users to provide values, which are passed into the UDF operator
2. Extend the UDF properties panel to include a space for users to provide parameter values

GitHub link: https://github.com/apache/texera/discussions/4154

----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org


Re: [D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

@zyratlo Please use a real example with screenshots to show the first solution.

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15481470

----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org


Re: [D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
GitHub user aglinxinyuan edited a comment on the discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

Sorry, I don’t understand the plan. First, the title “Feature Proposal” may be confusing, because Option 1 is not actually proposing a new feature. It is simply a way to use the existing system, and no new functionality needs to be added.

I understand that Yicong may be against this idea, but we should not restrict users from using the system in a particular way. The JVM can send parameters to the PVM as control messages; however, this is already part of the system design. I don’t think sending parameters via control messages contradicts either Option 1 or Option 2.

Moreover, sending parameters as control messages does not address or resolve any of the downsides of Option 1 or Option 2 that Yicong mentioned.

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15513932

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

Posted by "zyratlo (via GitHub)" <gi...@apache.org>.
GitHub user zyratlo added a comment to the discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

Sure. Here is a simple workflow to demonstrate how we can expose parameters using a Text Input Operator.

<img width="441" height="272" alt="image" src="https://github.com/user-attachments/assets/05ab9840-d6cc-4e01-83e2-af5972224791" />

The Python UDF takes a CSV file as input and performs an ML training algorithm. In this example, the developer has chosen to make the `test_size` and `random_seed` parameters configurable. To support this, an additional `Text Input` operator has been added as input to the UDF. The source code of the UDF must take into account both input ports, which is shown in the below screenshot:

<img width="1452" height="920" alt="image" src="https://github.com/user-attachments/assets/3a3f59c1-b104-470d-ae5b-8d8f1d12bf32" />

In the code, the `if` block executes on port 0, which is the parameter input. This code sets the class variable `parameters` to hold the values for `test_size` and `random_seed`. The `else` block will execute on any other port, so it runs with the CSV file input and will use the inputted values for the two parameters.

To ensure the parameters are processed first before the rest of the code, we ensure that the `File Scan Port` has a dependency on port 0 (since input ports are numbered top-down beginning at 0). This will mean that the `File Scan Port` will not be processed until the `Parameter Port` has finished:

<img width="499" height="330" alt="image" src="https://github.com/user-attachments/assets/503747d1-0319-4f9e-b7ea-4e845bf1f10c" />

Now, when the non-coding user wants to modify the parameters, they can simply edit the text in the `Text Input` operator instead of directly changing the UDF code.

<img width="552" height="330" alt="image" src="https://github.com/user-attachments/assets/995c5d22-8318-4396-9208-68fb8804c525" />

In this example, the developer assumes the formatting and order of the parameters will be consistent. However, it is easy to convert this to be dynamic parsing instead.

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15481568

----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org


Re: [D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
GitHub user aglinxinyuan edited a comment on the discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

Sorry, I don’t understand the plan. First, the title “Feature Proposal” may be confusing, because Option 1 is not actually proposing a new feature. It is simply a way to use the existing system, and no new functionality needs to be added.

I understand that Yicong may be against this idea, but we should not restrict users from using the system in a particular way. The JVM can send parameters to the PVM as control messages; however, I don’t think sending parameters via control messages contradicts either Option 1 or Option 2.

Moreover, sending parameters as control messages does not address or resolve any of the downsides of Option 1 or Option 2 that Yicong mentioned.

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15513932

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
GitHub user aglinxinyuan deleted a comment on the discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

Do you mean @kunwp1 ?

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15513881

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
GitHub user Yicong-Huang added a comment to the discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

For the second option, we previously implemented it already and ended up removed it. The main reason is the users who need to change the parameter would need to 1) define the value name in code, 2) define the value on the property panel. It is not easy to use. After many users complaining, we removed the feature.

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15511657

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Supporting non-coding users in configuring parameters in Python UDF code [texera]

Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: Supporting non-coding users in configuring parameters in Python UDF code

@carloea2 Please describe your design of option 2 per our discussion.

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15920111

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Supporting non-coding users in configuring parameters in Python UDF code [texera]

Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: Supporting non-coding users in configuring parameters in Python UDF code

@aglinxinyuan What you described is another design of solution 2.  We discussed it and agreed with it.

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15920109

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Supporting non-coding users in configuring parameters in Python UDF code [texera]

Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
GitHub user carloea2 added a comment to the discussion: Supporting non-coding users in configuring parameters in Python UDF code

https://github.com/apache/texera/pull/4268

This will be the PR to add this feature.

GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-16031711

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org


Re: [D] Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code [texera]

Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
GitHub user Yicong-Huang added a comment to the discussion: Feature Proposal: Extend Python UDF operators to allow users to modify parameters without editing the code

I am afraid I would give a -1 for the current proposal.

I want to repeat a previously mentioned idea/concept before we dive into support this feature. In my mind, UDFs are designed for developers. The user of UDF has to understand the code, and because of this, changing a value/parameter in the editor is the easiest method for those users. For non-coding users who would like to modify a value is not a recommended user scenario. Instead, the owner/developer of the UDF should go through a "publish" process to convert a UDF to a builtin operator. The new builtin operator should have predefined fields on the property panel, so that non-coding users can modify on the property panel. The property panel is the defined interface where the developer to control what parameters for a non-coding user can/should modify. Our existing plotting operators are great examples of Python UDFs converted to builtin operators. They are easy to use and can only be modified in the defined way. (i.e., a non-coding user cannot arbitrarily change an unwanted field mi
 stakenly).

With this understanding, I am a bit against the proposed idea in this discussion. There are a few reasons:
1. The proposed idea requires non-coding users to define parameters in an upstream operator and pass it to UDF. This is a bit hard to understand and counter intuitive: why am I editing a property of one operator on another operator? 
2. This adds additional complexity:
     - it would be hard for a developer to write: The UDF code would be more complex as it needs to add handling logic for `tuple['parameters']` column. 
     - it would be hard for a developer to maintain. suppose as a developer of UDF, I need to add/remove a parameter, I would need to modify multiple places.
     - it would be hard for a user to use/debug: suppose a UDF does not work as expected and there are a few possible operators passing parameters into this UDF. It would be hard for me as a user to trace the change. 
3. There is no abstract/interface guarantee. Anyone (developer or user) would be able to change code/parameter, and users may make mistakes when changing the code/parameter. Alternatively, if developers can "publish as builtin operator", they can define the parameters and range/scope of the parameters. And the user can only change the defined parameters in a controlled range, so less mistakes.


GitHub link: https://github.com/apache/texera/discussions/4154#discussioncomment-15511624

----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org