You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@predictionio.apache.org by chanlee514 <gi...@git.apache.org> on 2017/06/28 20:17:56 UTC
[GitHub] incubator-predictionio pull request #401: [PIO-72] Fix class loading for pio...
GitHub user chanlee514 opened a pull request:
https://github.com/apache/incubator-predictionio/pull/401
[PIO-72] Fix class loading for pio-shell
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chanlee514/incubator-predictionio develop
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-predictionio/pull/401.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #401
----
commit ab89fbc10e72887b4a5abc0914e1ed414c5c3f1f
Author: Chan Lee <ch...@gmail.com>
Date: 2017-05-26T06:22:47Z
[PIO-72] Fix class loading for pio-shell
commit b4576130d4bad07c83fa4926cc1b48f43c2247a9
Author: Chan Lee <ch...@gmail.com>
Date: 2017-05-26T10:04:44Z
Fix error for loading classpath on travis
commit 5be171512c0ffdc0299a4d87a8da4c8cc19c03a7
Author: Chan Lee <ch...@gmail.com>
Date: 2017-06-27T17:13:30Z
Merge remote-tracking branch 'origin/develop' into develop
commit b5d334056af3397984985d2f88a53da1944662a1
Author: Chan Lee <ch...@gmail.com>
Date: 2017-06-27T19:45:56Z
Fix bug on class loading for pio
commit c71eb37988c825ae4758e34a3816e404c01771c7
Author: Chan Lee <ch...@gmail.com>
Date: 2017-06-27T20:50:28Z
Fix classloading bug for travis
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-predictionio issue #401: [PIO-72] Fix class loading for pio-shell
Posted by mars <gi...@git.apache.org>.
Github user mars commented on the issue:
https://github.com/apache/incubator-predictionio/pull/401
Hi @BrianOn99,
Adding that `--jars` option to `pio-shell` command is the right solution, and then the "No suitable driver found" error can be solved by adding the Postgres driver to your PredictionIO install:
1. download [Postgres JDBC driver](https://jdbc.postgresql.org/download.html) (probably the newest one for Java 8)
2. put it in the PredictionIO distribution's `lib/` directory (this directory is sibling to the `bin/` directory where the `pio` command is located; any jars in that directory are automatically added to the classpath for `pio` commands)
We're working on releasing 0.12!
---
[GitHub] incubator-predictionio issue #401: [PIO-72] Fix class loading for pio-shell
Posted by BrianOn99 <gi...@git.apache.org>.
Github user BrianOn99 commented on the issue:
https://github.com/apache/incubator-predictionio/pull/401
Good news to hear that 0.12 will be released soon!
---
[GitHub] incubator-predictionio issue #401: [PIO-72] Fix class loading for pio-shell
Posted by mars <gi...@git.apache.org>.
Github user mars commented on the issue:
https://github.com/apache/incubator-predictionio/pull/401
Back in May, we fixed an intermittent class loading problem by [making a change to stabilize the classpath](https://github.com/mars/incubator-predictionio/commit/9ecc77628aba347454073e9919096a8fc8e0b952) in our fork of incubator-predictionio. Would you be open to adding that to this PR as well?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-predictionio pull request #401: [PIO-72] Fix class loading for pio...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-predictionio/pull/401
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-predictionio issue #401: [PIO-72] Fix class loading for pio-shell
Posted by chanlee514 <gi...@git.apache.org>.
Github user chanlee514 commented on the issue:
https://github.com/apache/incubator-predictionio/pull/401
@mars Certainly. I've already merged it for my branch [here](https://github.com/chanlee514/incubator-predictionio/commits/pr-372). I still haven't had time to look into the shade rule error for Heroku UR. I'll update the PR once this has been resolved.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-predictionio issue #401: [PIO-72] Fix class loading for pio-shell
Posted by mars <gi...@git.apache.org>.
Github user mars commented on the issue:
https://github.com/apache/incubator-predictionio/pull/401
Yes @BrianOn99, I do believe [class loading for pio-shell is fixed](https://github.com/apache/incubator-predictionio/blob/develop/bin/pio-shell#L59) for the next release, or if you `make-distribution.sh` on main `develop` branch, you'll get these fixes now.
---
[GitHub] incubator-predictionio issue #401: [PIO-72] Fix class loading for pio-shell
Posted by dszeto <gi...@git.apache.org>.
Github user dszeto commented on the issue:
https://github.com/apache/incubator-predictionio/pull/401
LGTM. Thanks @chanlee514 @mars !
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-predictionio issue #401: [PIO-72] Fix class loading for pio-shell
Posted by BrianOn99 <gi...@git.apache.org>.
Github user BrianOn99 commented on the issue:
https://github.com/apache/incubator-predictionio/pull/401
I am using predicitonio 0.11 with postgres, and meet this issue `java.lang.ClassNotFoundException: jdbc.StorageClient` when using `pio-shell` when trying the tutorial https://predictionio.incubator.apache.org/datacollection/eventmodel/
As 0.12 is not released, I tried to apply this patch to 0.11 by appending `pio-data-jdbc-assembly-0.11.0-incubating.jar` to the `--jars` flag in `pio-shell` file, but then another error pops up
```
java.sql.SQLException: No suitable driver found for jdbc:postgresql://localhost/pio
at java.sql.DriverManager.getConnection(DriverManager.java:689)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
```
Is it necessary to patch `Common.scala` and recompile?
Or is there another way to do the same thing as `PEventStore.aggregateProperties` as shown in the tutorial without `pio-shell`?
I really hope to have some help to be shown in the tutorial otherwise other newcomers like me will be confused.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-predictionio issue #401: [PIO-72] Fix class loading for pio-shell
Posted by BrianOn99 <gi...@git.apache.org>.
Github user BrianOn99 commented on the issue:
https://github.com/apache/incubator-predictionio/pull/401
@mars Thanks for the help. Yes the jdbc driver is not loaded correctly, though I have the jar downloaded. I need to add `export CLASSPATH` like this in pio-shell
```
if [[ "$1" == "--with-spark" ]]
then
echo "Starting the PIO shell with the Apache Spark Shell."
# Get paths of assembly jars to pass to spark-shell
. ${PIO_HOME}/bin/compute-classpath.sh
shift
export CLASSPATH
${SPARK_HOME}/bin/spark-shell --jars ${ASSEMBLY_JAR},/PredictionIO-0.11.0-incubating/lib/spark/pio-data-jdbc-assembly-0.11.0-incubating.jar $@
else
```
and then every thing works.
I note that `export CLASSPATH` exist in `bin/pio-class`, so in my environment `pio status`, `pio train`, etc, can work, but `pio-shell` does not call `bin/pio-class` so `CLASSPATH` is not exported. Is this fixed upstream or my setting is wrong?
---