You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by jihoonson <gi...@git.apache.org> on 2015/03/27 10:19:53 UTC
[GitHub] tajo pull request: TAJO-1344: Python UDF support
GitHub user jihoonson opened a pull request:
https://github.com/apache/tajo/pull/473
TAJO-1344: Python UDF support
To support Python's dynamic typing, I added AnyDatum which can contain a datum of any types.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jihoonson/tajo-2 TAJO-1344
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/tajo/pull/473.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #473
----
commit 349b1c23eb9653a7ef0206e9c2a94178953e134f
Author: Hyunsik Choi <hy...@apache.org>
Date: 2015-03-12T09:53:45Z
Initial prototype.
commit b8596e88c58840d2c80c09af604c156ea0073039
Author: Hyunsik Choi <hy...@apache.org>
Date: 2015-03-12T09:54:02Z
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into python_udf
commit 3d4b2b9c5a425d4be027b6fc139f8c469c19a74d
Author: Jihoon Son <ji...@apache.org>
Date: 2015-03-23T12:41:51Z
Merge branch 'python_udf' of https://github.com/hyunsik/tajo into TAJO-1344
commit 104fa93d4e4fb8752679c94ea06c0357692a95d5
Author: Jihoon Son <ji...@apache.org>
Date: 2015-03-23T13:59:15Z
TAJO-1344
commit 9241d1ef5a95479772cf192f942905347c3729a4
Author: Jihoon Son <ji...@apache.org>
Date: 2015-03-24T08:05:31Z
TAJO-1344
commit 7808685f735c56ffb035971630a8865be37e4c7a
Author: Jihoon Son <ji...@apache.org>
Date: 2015-03-25T01:29:28Z
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1344
commit b3d09b82c0288ef6e9c6e85715c50bf5578eea0b
Author: Jihoon Son <ji...@apache.org>
Date: 2015-03-25T10:03:48Z
TAJO-1344
commit 06d4a27e65cc75aec2ed1239cdf2215dcf99c00d
Author: Jihoon Son <ji...@apache.org>
Date: 2015-03-25T13:34:40Z
TAJO-1344
commit 44afbe25f2021b939aa4498a6791e0f268b6d5e7
Author: Jihoon Son <ji...@apache.org>
Date: 2015-03-26T06:34:27Z
TAJO-1344
commit dae49bdbd6d13ff1503168591260569863683e3f
Author: Jihoon Son <ji...@apache.org>
Date: 2015-03-26T10:00:52Z
TAJO-1344
commit 88a2fd2da98b78f52d7b2288a68a2e5ac67ad757
Author: Jihoon Son <ji...@apache.org>
Date: 2015-03-26T13:32:24Z
TAJO-1344
commit b615a696ab42c75fd4832592772c7219b6a4dccc
Author: Jihoon Son <ji...@apache.org>
Date: 2015-03-26T14:22:10Z
TAJO-1344
commit 99ec66a0747413fb2e43a3b15dc142fb4a943b41
Author: Jihoon Son <ji...@apache.org>
Date: 2015-03-26T14:44:11Z
TAJO-1344
commit 3457b7830e12f8a8273b9d76f35198bacbee8f90
Author: Jihoon Son <ji...@apache.org>
Date: 2015-03-27T02:09:07Z
Remove OptionalFunctionContext
commit d7fa88a3feb96cc4bad800f77a07ac126fecac2e
Author: Jihoon Son <ji...@apache.org>
Date: 2015-03-27T03:07:08Z
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1344
commit 4741be6bd7faf96ed1cc8d22e430388db84c3e4e
Author: Jihoon Son <ji...@apache.org>
Date: 2015-03-27T03:20:09Z
TAJO-1344
commit 4a5570597b85f71af630d3c318de87aa03737ae7
Author: Jihoon Son <ji...@apache.org>
Date: 2015-03-27T09:01:40Z
TAJO-1344
commit 1d7dab11eb73be96e0c979de118e7e4e7321eaac
Author: Jihoon Son <ji...@apache.org>
Date: 2015-03-27T09:14:47Z
TAJO-1344
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1344: Python UDF support
Posted by jihoonson <gi...@git.apache.org>.
Github user jihoonson commented on the pull request:
https://github.com/apache/tajo/pull/473#issuecomment-90789940
I'll open another PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1344: Python UDF support
Posted by jihoonson <gi...@git.apache.org>.
Github user jihoonson commented on the pull request:
https://github.com/apache/tajo/pull/473#issuecomment-87158298
I'll investigate the test failure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1344: Python UDF support
Posted by jihoonson <gi...@git.apache.org>.
Github user jihoonson commented on the pull request:
https://github.com/apache/tajo/pull/473#issuecomment-87258850
The test failure looks to be related with the version of Jython.
I fixed it and verified with ```mvn clean install -Pparallel-test,hcatalog-0.12.0 -Dmaven.fork.count=2```.
In addition, I fixed findbugs warnings.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1344: Python UDF support
Posted by hyunsik <gi...@git.apache.org>.
Github user hyunsik commented on the pull request:
https://github.com/apache/tajo/pull/473#issuecomment-87281813
The patch looks good to me.
I have one suggestion. You make new EvalNode Type ```PYTHON_FUNCTION```. But, it may be not matched to the design of EvalNode, and it will require duplicated codes for EvalNode visitors.
You seem to do because Python function involcation is different from existing Java function. I think that the best solution is to separate EvalNode and Function Invocation code. In more detail, you should implement some function invocation interface, and separate java function invocation and python function invocation. Each function invocation implementation can be initialized at the first function call, or you can use EvalNode::Bind(), being addressed in TAJO-1436.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1344: Python UDF support
Posted by jihoonson <gi...@git.apache.org>.
Github user jihoonson commented on the pull request:
https://github.com/apache/tajo/pull/473#issuecomment-87399705
@hyunsik, thanks for the nice suggestion. It is definitely a better solution.
However, I have a different opinion on detailed implementation.
The first option that initialize each function invocation implementation at the first function call is deprecated, so it is not a good choice.
And the second option using EvalNode::Bind() seems to initialize schema-dependent variables. Even though we still can change the specification of TAJO-1436, the current one looks good. Since relations have different schemas, schema-dependent variables should be initialized for each schema.
However, function invocation implementation does not depend on relations' schema. I think that it would be better to initialize when an instance of ```FunctionEval``` is created.
What do you think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1344: Python UDF support
Posted by jinossy <gi...@git.apache.org>.
Github user jinossy commented on the pull request:
https://github.com/apache/tajo/pull/473#issuecomment-86959283
Great! Really useful feature
I'll review this weekend.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1344: Python UDF support
Posted by jihoonson <gi...@git.apache.org>.
Github user jihoonson commented on the pull request:
https://github.com/apache/tajo/pull/473#issuecomment-86956557
I've added comments and documentations.
Please review the patch.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1344: Python UDF support
Posted by jihoonson <gi...@git.apache.org>.
Github user jihoonson closed the pull request at:
https://github.com/apache/tajo/pull/473
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---