You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by DmytroShkvyra <gi...@git.apache.org> on 2017/02/09 19:45:44 UTC
[GitHub] zeppelin pull request #2002: [ZEPPELIN-1928]: Get a rid of preinstalled exte...
GitHub user DmytroShkvyra opened a pull request:
https://github.com/apache/zeppelin/pull/2002
[ZEPPELIN-1928]: Get a rid of preinstalled external dependencies (nodejs and npm) for maven build
### What is this PR for?
Get a rid of preinstalled external dependencies (nodejs and npm) for maven build
- We do not need install nodejs npm and libfontconfig anymore
- zeppelin-web tests will run via maven build
- Fix documentation
### What type of PR is it?
[Bug Fix | Improvement | Documentation]
### Todos
N/A
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1928
### How should this be tested?
1. Create new empty instance of (Centos7|Ubuntu|Windows) in VirtualBox
2. Install JDK and Maven (also export JAVA_HOME and M2_HOME etc.)
3. Go to project directory and run
mvn clean package -Pbuild-distr
### Screenshots (if appropriate)
see travis build https://travis-ci.org/DmytroShkvyra/zeppelin/builds/200104715
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Zeppelin ........................................... SUCCESS [ 53.338 s]
[INFO] Zeppelin: Interpreter .............................. SUCCESS [ 37.829 s]
[INFO] Zeppelin: Zengine .................................. SUCCESS [ 25.306 s]
[INFO] Zeppelin: Display system apis ...................... SUCCESS [ 26.981 s]
[INFO] Zeppelin: Spark dependencies ....................... SUCCESS [01:21 min]
[INFO] Zeppelin: Spark .................................... SUCCESS [ 33.265 s]
[INFO] Zeppelin: Markdown interpreter ..................... SUCCESS [ 3.576 s]
[INFO] Zeppelin: Angular interpreter ...................... SUCCESS [ 1.225 s]
[INFO] Zeppelin: Shell interpreter ........................ SUCCESS [ 2.526 s]
[INFO] Zeppelin: Livy interpreter ......................... SUCCESS [03:03 min]
[INFO] Zeppelin: HBase interpreter ........................ SUCCESS [ 17.911 s]
[INFO] Zeppelin: Apache Pig Interpreter ................... SUCCESS [ 13.064 s]
[INFO] Zeppelin: PostgreSQL interpreter ................... SUCCESS [ 4.306 s]
[INFO] Zeppelin: JDBC interpreter ......................... SUCCESS [ 6.681 s]
[INFO] Zeppelin: File System Interpreters ................. SUCCESS [ 4.661 s]
[INFO] Zeppelin: Flink .................................... SUCCESS [ 30.573 s]
[INFO] Zeppelin: Apache Ignite interpreter ................ SUCCESS [ 13.729 s]
[INFO] Zeppelin: Kylin interpreter ........................ SUCCESS [ 2.170 s]
[INFO] Zeppelin: Python interpreter ....................... SUCCESS [ 3.098 s]
[INFO] Zeppelin: Lens interpreter ......................... SUCCESS [ 16.616 s]
[INFO] Zeppelin: Apache Cassandra interpreter ............. SUCCESS [01:11 min]
[INFO] Zeppelin: Elasticsearch interpreter ................ SUCCESS [ 12.687 s]
[INFO] Zeppelin: BigQuery interpreter ..................... SUCCESS [ 24.615 s]
[INFO] Zeppelin: Alluxio interpreter ...................... SUCCESS [ 10.031 s]
[INFO] Zeppelin: Scio ..................................... SUCCESS [ 54.854 s]
[INFO] Zeppelin: web Application .......................... SUCCESS [39:47 min]
[INFO] Zeppelin: Server ................................... SUCCESS [ 36.567 s]
[INFO] Zeppelin: Packaging distribution ................... SUCCESS [23:54 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:16 h
[INFO] Finished at: 2017-02-09T11:07:53+00:00
[INFO] Final Memory: 204M/1102M
[INFO] ------------------------------------------------------------------------
### Questions:
* Does the licenses files need update? - No
* Is there breaking changes for older versions? - No
* Does this needs documentation? - Update of docs are part of PR
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/DmytroShkvyra/zeppelin ZEPPELIN-1928
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2002.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2002
----
commit d09f9b3f8c540d31997484849fd8980e8945784d
Author: DmytroShkvyra <ds...@gmail.com>
Date: 2017-02-09T18:29:05Z
[ZEPPELIN-1928] Get a rid of preinstalled external dependencies (nodejs and npm) for maven build
commit 337bc89b13584fbfa0a508b1321229cb9b42fce0
Author: DmytroShkvyra <ds...@gmail.com>
Date: 2017-02-09T18:45:05Z
[ZEPPELIN-1928] add rat exclusion
commit eec761fb2815f49243dee698f8aff0273d812c37
Author: DmytroShkvyra <ds...@gmail.com>
Date: 2017-02-09T19:41:01Z
[ZEPPELIN-1928] add execution permition for add_dependencies.sh
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
Hi @1ambda, when we install nodejs modules locally(only for project) we do not have access to them from command line outside. So we need install these modules globally. I'm not so plural with nodejs to export locally installed modules. I have added installation these modules only for compatibility with old style of development:
$ yarn build
$ yarn test
\u041e\u0442\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u043e \u0441 \u0431\u0435\u0441\u043f\u0440\u043e\u0432\u043e\u0434\u043d\u043e\u0433\u043e \u0443\u0441\u0442\u0440\u043e\u0439\u0441\u0442\u0432\u0430 BlackBerry�
-----Original Message-----
From: Hoon <no...@github.com>
Date: Sat, 11 Feb 2017 20:15:14
To: apache/zeppelin<ze...@noreply.github.com>
Reply-To: apache/zeppelin <re...@reply.github.com>
Cc: Dmytro Shkvyra<ds...@gmail.com>; Mention<me...@noreply.github.com>
Subject: Re: [apache/zeppelin] [ZEPPELIN-1928]: Get a rid of preinstalled
external dependencies (nodejs and npm) for maven build (#2002)
1ambda commented on this pull request.
> # install required depepdencies and bower packages (only once)
-$ npm install -g yarn
-$ yarn install
+```
+$ npm install -g yarn bower npm-run-all rimraf grunt webpack karma karma-phantomjs-launcher
@DmytroShkvyra We don't need to install these packages as global. All packages installed locally (`node_modules`) can be used in `scripts` tag in `package.json` as command like `grunt`, `webpack`
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/apache/zeppelin/pull/2002
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
@1ambda Yes, this different from build, but cause the same. Helium installed npm with frondend-maven-plugin.
BTW. Many contributors cant use windows because this is proprietary and costs some money.
I though zeppelin is cross-platform system. ;-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
@1ambda
>Could you share the content of /vagrant/zeppelin/local-repo/helium-bundle/npm-debug.log? when node/node node_modules/webpack/bin/webpack.js --display-error-details --json failed. (Gist will be enough)
>I would like to check. It seems that helium bundling failed due to network is offline not because of to max stack size.
https://issues.apache.org/jira/secure/attachment/12852597/npm-debug.log
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
Installation of hidden dependencies was removed. (accordingly to reviewers comments).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
Failed tests:
IgniteSqlInterpreterTest.testSql:95 expected:<SUCCESS> but was:<ERROR>
This test do not related with PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by AtulKumVerma <gi...@git.apache.org>.
Github user AtulKumVerma commented on the issue:
https://github.com/apache/zeppelin/pull/2002
i am not able to get data from hbase using spark hbase connector,
any one did this so plz let me know all of its hbase,spark and other
configuation.
i got Error...
java.io.IOException: java.lang.reflect.InvocationTargetException
at
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
at
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
at
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
at
org.apache.spark.sql.execution.datasources.hbase.HBaseConnectionCache$$anonfun$getConnection$1.apply(HBaseConnectionCache.scala:141)
at
org.apache.spark.sql.execution.datasources.hbase.HBaseConnectionCache$$anonfun$getConnection$1.apply(HBaseConnectionCache.scala:141)
at
org.apache.spark.sql.execution.datasources.hbase.HBaseConnectionCache$$anonfun$1.apply(HBaseConnectionCache.scala:132)
at
org.apache.spark.sql.execution.datasources.hbase.HBaseConnectionCache$$anonfun$1.apply(HBaseConnectionCache.scala:130)
at
scala.collection.mutable.MapLike$class.getOrElseUpdate(MapLike.scala:194)
at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:80)
at
org.apache.spark.sql.execution.datasources.hbase.HBaseConnectionCache$.getConnection(HBaseConnectionCache.scala:130)
at
org.apache.spark.sql.execution.datasources.hbase.HBaseConnectionCache$.getConnection(HBaseConnectionCache.scala:141)
at
org.apache.spark.sql.execution.datasources.hbase.RegionResource.init(HBaseResources.scala:93)
at
org.apache.spark.sql.execution.datasources.hbase.ReferencedResource$class.liftedTree1$1(HBaseResources.scala:57)
at
org.apache.spark.sql.execution.datasources.hbase.ReferencedResource$class.acquire(HBaseResources.scala:54)
at
org.apache.spark.sql.execution.datasources.hbase.RegionResource.acquire(HBaseResources.scala:88)
at
org.apache.spark.sql.execution.datasources.hbase.ReferencedResource$class.releaseOnException(HBaseResources.scala:74)
at
org.apache.spark.sql.execution.datasources.hbase.RegionResource.releaseOnException(HBaseResources.scala:88)
at
org.apache.spark.sql.execution.datasources.hbase.RegionResource.<init>(HBaseResources.scala:108)
at
org.apache.spark.sql.execution.datasources.hbase.HBaseTableScanRDD.getP......
---
*Thanks & Regards*
*Er. Atul VermaStudent of CSE department.*
On Wed, Sep 20, 2017 at 11:46 AM, liang <no...@github.com> wrote:
> Should I need to modify some files? @1ambda <https://github.com/1ambda>
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/zeppelin/pull/2002#issuecomment-330755633>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AeFWGwGc1sTwoIM4-XPm53V1FfRIm1CIks5skK3TgaJpZM4L8lLl>
> .
>
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
@1ambda
> I think we don't need to worry about max call stack size since it occurs in unusual env like windows, linux on vm.
> Zeppelin is not the thing like JDK. There is no reason to support windows build. Working is enough.
> Even nodejs fails to support all platforms. (e.g max call stack size).
The problem not in OS or NodeJS. Problem is that I and my colleagues are backend developers whom want develop interpreters and integrate zeppelin (to Hadoop e.g. and other cluster technologies) and the do not know anything about UI development (in common). They use usually vms for testing and development. And they use maven as the most mature build tool. I and my collegues don't want learn nodejs and how to solve it's bugs. I have more than 15 years experience as developer. Last time I have developed UI 7 years ago. Only this gave me possibility find solution fast. To be honest my experience with nodejs was not so sweet.
Let's give to all developers possibility contribute to zeppelin! :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/2002
@DmytroShkvyra Should we support build env such as windows on vm?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/2002
@DmytroShkvyra That's cool. You are trying to modify something without understanding. Anyway, We have 2 problems.
1. local development
- you must install yarn if you want to use `yarn` in terminal. there is no exception.
2. `mvn package`
- convert yarn to npm instead of this kind of big change.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/2002
@DmytroShkvyra Really sorry if you feel like that.
But IMO
- you didn't try almost all possible alternatives
- you didn't try to understand my comments at all. I repeatedly said script tags is not the problem.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/2002
Regrading to **max call stack size** what @DmytroShkvyra mentioned in above comment (for other people)
- https://github.com/npm/npm/issues/9953
- https://github.com/npm/npm/issues/10776#issuecomment-208791872
> Symlinks are only supported in POSIX filesystem i.e. ext3 or ext4, not on Windows filesytems i.e. NTFS, FAT. This is a serious bug while using docker. Just switch to ext4 filesystem and everything will run as expected. So I think that is why on windows it recursively tries to suppress the creation of symlinks using the flag, whose result is unidentified by the (windows) filesystem, results in stackoverflow (Maximum call stack size exceeded).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/2002
@DmytroShkvyra
- I think we don't need to worry about **max call stack size** since it occurs in unusual env like windows, *linux on vm*.
- Zeppelin is not the thing like JDK. There is no reason to support windows build. Working is enough.
- Even nodejs fails to support all platforms. (e.g **max call stack size**).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/2002
So, what you want to solve is **building zeppelin in vms which can't have enough symlinks**? right?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
@1ambda BTW I have reverted my changes in README.md for build and test from CL and add section about build and test with maven like this example:
```
mvn test -Dmaven.clean.skip=true -Dmaven.compile.skip=true
```
Output
```
[INFO] Running "replace:dist" (replace) task
[INFO] >> 36 replacements in 37 files.
[INFO]
[INFO] Running "cacheBust:taskName" (cacheBust) task
[INFO] >> Busted: dist/index.html
[INFO]
[INFO] Done, without errors.
[INFO]
[INFO]
[INFO] Execution Time (2017-02-13 18:24:21 UTC)
[INFO] concurrent:dist 6.8s \u2587 2%
[INFO] copy:dist 4m 56.9s \u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587\u2587 85%
[INFO] uglify:generated 15.1s \u2587\u2587 4%
[INFO] replace:dist 23.7s \u2587\u2587\u2587 7%
[INFO] Total 5m 50.3s
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 06:59 min
[INFO] Finished at: 2017-02-13T18:30:11+00:00
[INFO] Final Memory: 18M/221M
[INFO] ------------------------------------------------------------------------
[vagrant@c7 zeppelin-web]$ mvn test -Dmaven.clean.skip=true -Dmaven.compile.skip=true
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
@1ambda No, I want make "normal|classical" maven project which do not need preinstall everybody nodejs and their packages by hand. More over - yarn works faster than npm
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
@1ambda Seems to Ignite and MR filed. They not related to zeppelin-web.
Is problem in proxy-configuration for bower? I could add support for this in POM.xml via -D parameters. But I think we could open another jira for this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
@1ambda
>If you want to use node modules in terminal, you have to install them globally. (e.g $ yarn install)
>But in case of grunt (and karma so on..), we don't need to do that. Since they are used in scripts >(package.json). That's what i meant.
I dont care about use modules in terminal. I want to run
```
mvn package
```
and execute clean->process-resources->test->package phases of maven automatically, without errors and manual maintenance of local environment.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by AtulKumVerma <gi...@git.apache.org>.
Github user AtulKumVerma commented on the issue:
https://github.com/apache/zeppelin/pull/2002
./bin/spark-shell --jars /home/ist/jarfile/shc-core-1.1.1-2.1-s_2.11.jar,/home/iile/hbase-spark-2.0.0-alpha-1.jar,/home/ist/jarfile/hbase-client-1.0.1.1.jar,/home/ist/jarfile/hbase-0.94.2.jar,/home/ist/jarfile/hbase-common-0.98.3-hadoop1.jar
import org.apache.spark.sql._
import org.apache.spark.sql.datasources.hbase._
import org.apache.spark.sql.execution.datasources.hbase._
import org.apache.hadoop._
import org.apache.hadoop.hbase.util._
val sqlContext = new SQLContext(sc)
def catalog = s"""{
|"table":{"namespace":"default", "name":"table1"},
|"rowkey":"key",
|"columns":{
|"col0":{"cf":"rowkey", "col":"key", "type":"string"},
|"col1":{"cf":"cf1", "col":"col1", "type":"string"}
|}
|}""".stripMargin
def withCatalog(cat: String): DataFrame = {
sqlContext
.read
.options(Map(HBaseTableCatalog.tableCatalog->cat))
.format("org.apache.spark.sql.execution.datasources.hbase")
.load()
}
val df = withCatalog(catalog)
df.printSchema()
df.show()
Caused by: java.lang.reflect.InvocationTargetException: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.util.Addressing.getIpAddress()Ljava/net/InetAddress;
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
... 111 more
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.util.Addressing.getIpAddress()Ljava/net/InetAddress;
at org.apache.hadoop.hbase.client.ClientIdGenerator.getIpAddressBytes(ClientIdGenerator.java:83)
at org.apache.hadoop.hbase.client.ClientIdGenerator.generateClientId(ClientIdGenerator.java:43)
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
I have had similar issues. The main problem with this plugin. It can run scripts in package.json via shell (postinstall, pretest etc) but not provide that they accessible for current shell session.
\u041e\u0442\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u043e \u0441 \u0431\u0435\u0441\u043f\u0440\u043e\u0432\u043e\u0434\u043d\u043e\u0433\u043e \u0443\u0441\u0442\u0440\u043e\u0439\u0441\u0442\u0432\u0430 BlackBerry�
-----Original Message-----
From: Hoon <no...@github.com>
Date: Thu, 09 Feb 2017 17:20:23
To: apache/zeppelin<ze...@noreply.github.com>
Reply-To: apache/zeppelin <re...@reply.github.com>
Cc: Dmytro Shkvyra<ds...@gmail.com>; Author<au...@noreply.github.com>
Subject: Re: [apache/zeppelin] [ZEPPELIN-1928]: Get a rid of preinstalled
external dependencies (nodejs and npm) for maven build (#2002)
FYI - https://github.com/eirslett/frontend-maven-plugin/issues/562
--
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/zeppelin/pull/2002#issuecomment-278830036
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
@1ambda I repeat again (very slowly):
1. We cant install npm with frontend-maven-plugin locally. It throws a lot of errors like this (zeppelin.log) this is attempt install npm with frontend-maven-plugin in runtime. Errors with pom.xml are simmilar:
```
WARN [2017-02-09 18:20:34,011] ({main} Helium.java[loadConf]:119) - /vagrant/zeppelin/conf/helium.json does not exists
ERROR [2017-02-09 18:20:35,296] ({main} Helium.java[getAllPackageInfo]:184) - Directory '/vagrant/zeppelin/local-repo/helium-registry-cache/https:/s3.amazonaws.com/helium-package' could not be created
java.io.IOException: Directory '/vagrant/zeppelin/local-repo/helium-registry-cache/https:/s3.amazonaws.com/helium-package' could not be created
at org.apache.commons.io.FileUtils.openOutputStream(FileUtils.java:363)
at org.apache.commons.io.FileUtils.writeStringToFile(FileUtils.java:1928)
at org.apache.commons.io.FileUtils.writeStringToFile(FileUtils.java:1962)
at org.apache.zeppelin.helium.HeliumOnlineRegistry.writeToCache(HeliumOnlineRegistry.java:130)
at org.apache.zeppelin.helium.HeliumOnlineRegistry.getAll(HeliumOnlineRegistry.java:101)
at org.apache.zeppelin.helium.Helium.getAllPackageInfo(Helium.java:173)
at org.apache.zeppelin.helium.Helium.getBundlePackagesToBundle(Helium.java:307)
at org.apache.zeppelin.server.ZeppelinServer.<init>(ZeppelinServer.java:136)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.createSingletonInstance(CXFNonSpringJaxrsServlet.java:382)
at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.createApplicationInstance(CXFNonSpringJaxrsServlet.java:454)
at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.createServerFromApplication(CXFNonSpringJaxrsServlet.java:432)
at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.init(CXFNonSpringJaxrsServlet.java:93)
at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:616)
at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:396)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:871)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:163)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
at org.eclipse.jetty.server.Server.start(Server.java:387)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
at org.eclipse.jetty.server.Server.doStart(Server.java:354)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:185)
ERROR [2017-02-09 18:20:53,185] ({main} ZeppelinServer.java[<init>]:138) - Running 'npm run bundle --registry=http://registry.npmjs.org/' in /vagrant/zeppelin/local-repo/helium-bundle
> zeppelin-helium-bundle@ bundle /vagrant/zeppelin/local-repo/helium-bundle
> node/node node_modules/webpack/bin/webpack.js --display-error-details --json
module.js:471
throw err;
^
Error: Cannot find module '/vagrant/zeppelin/local-repo/helium-bundle/node_modules/webpack/bin/webpack.js'
at Function.Module._resolveFilename (module.js:469:15)
at Function.Module._load (module.js:417:25)
at Module.runMain (module.js:604:10)
at run (bootstrap_node.js:394:7)
at startup (bootstrap_node.js:149:9)
at bootstrap_node.js:509:3
npm ERR! Linux 3.10.0-514.2.2.el7.x86_64
npm ERR! argv "/vagrant/zeppelin/local-repo/helium-bundle/node/node" "/vagrant/zeppelin/local-repo/helium-bundle/node/node_modules/npm/bin/npm-cli.js" "run" "bundle" "--registry=http://registry.npmjs.org/"
npm ERR! node v6.9.1
npm ERR! npm v3.10.8
npm ERR! code ELIFECYCLE
npm ERR! zeppelin-helium-bundle@ bundle: `node/node node_modules/webpack/bin/webpack.js --display-error-details --json`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the zeppelin-helium-bundle@ bundle script 'node/node node_modules/webpack/bin/webpack.js --display-error-details --json'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the zeppelin-helium-bundle package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node/node node_modules/webpack/bin/webpack.js --display-error-details --json
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs zeppelin-helium-bundle
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls zeppelin-helium-bundle
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR! /vagrant/zeppelin/local-repo/helium-bundle/npm-debug.log
java.io.IOException: Running 'npm run bundle --registry=http://registry.npmjs.org/' in /vagrant/zeppelin/local-repo/helium-bundle
> zeppelin-helium-bundle@ bundle /vagrant/zeppelin/local-repo/helium-bundle
> node/node node_modules/webpack/bin/webpack.js --display-error-details --json
module.js:471
throw err;
^
Error: Cannot find module '/vagrant/zeppelin/local-repo/helium-bundle/node_modules/webpack/bin/webpack.js'
at Function.Module._resolveFilename (module.js:469:15)
at Function.Module._load (module.js:417:25)
at Module.runMain (module.js:604:10)
at run (bootstrap_node.js:394:7)
at startup (bootstrap_node.js:149:9)
at bootstrap_node.js:509:3
npm ERR! Linux 3.10.0-514.2.2.el7.x86_64
npm ERR! argv "/vagrant/zeppelin/local-repo/helium-bundle/node/node" "/vagrant/zeppelin/local-repo/helium-bundle/node/node_modules/npm/bin/npm-cli.js" "run" "bundle" "--registry=http://registry.npmjs.org/"
npm ERR! node v6.9.1
npm ERR! npm v3.10.8
npm ERR! code ELIFECYCLE
npm ERR! zeppelin-helium-bundle@ bundle: `node/node node_modules/webpack/bin/webpack.js --display-error-details --json`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the zeppelin-helium-bundle@ bundle script 'node/node node_modules/webpack/bin/webpack.js --display-error-details --json'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the zeppelin-helium-bundle package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node/node node_modules/webpack/bin/webpack.js --display-error-details --json
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs zeppelin-helium-bundle
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls zeppelin-helium-bundle
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR! /vagrant/zeppelin/local-repo/helium-bundle/npm-debug.log
at org.apache.zeppelin.helium.HeliumBundleFactory.buildBundle(HeliumBundleFactory.java:210)
at org.apache.zeppelin.helium.HeliumBundleFactory.buildBundle(HeliumBundleFactory.java:105)
at org.apache.zeppelin.server.ZeppelinServer.<init>(ZeppelinServer.java:136)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.createSingletonInstance(CXFNonSpringJaxrsServlet.java:382)
at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.createApplicationInstance(CXFNonSpringJaxrsServlet.java:454)
at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.createServerFromApplication(CXFNonSpringJaxrsServlet.java:432)
at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.init(CXFNonSpringJaxrsServlet.java:93)
at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:616)
at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:396)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:871)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:163)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
at org.eclipse.jetty.server.Server.start(Server.java:387)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
at org.eclipse.jetty.server.Server.doStart(Server.java:354)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:185)
INFO [2017-02-09 18:20:55,830] ({main} InterpreterFactory.java[init]:247) - InterpreterSettingRef name ignite
```
I have written about causes this behaviour - (limit of amount symlinks in Linux and recursive style of package installation with npm and limit of stack size for JS engines in different OSes).
See https://github.com/npm/npm/issues/9953
> you didn't try to understand my comments at all. I repeatedly said script tags is not the problem.
I didn't say that these scripts "are problem". I just say that:
this:
```
"postinstall": "bower install --silent",
```
and this:
```
"pretest": "npm install karma-phantomjs-launcher",
```
Why?
Because when frontend-maven-plugin execute something like this:
```
<execution>
<id>yarn add bower</id>
<goals>
<goal>yarn</goal>
</goals>
<configuration>
<arguments>add bower --no-lockfile --no-bin-links</arguments>
</configuration>
</execution>
```
or like this:
```
<execution>
<id>npm npm install</id>
<goals>
<goal>npm</goal>
</goals>
<configuration>
<arguments>install</arguments>
</configuration>
</execution>
```
it triggered **postinstall** and naturally it fails
Similar situation with
```
mvn test
```
It triggered **pretest** with same result.
Why we should use maven for build????
Because we can build all sub-modules by one command instead of run build and test each sub module.
**CONCLUSION**: We should avoid use tags like "postinstall" and "pretest"
Do you agree?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/2002
FYI - https://github.com/eirslett/frontend-maven-plugin/issues/562
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/2002
@DmytroShkvyra
I think the log you'v just posted is quite different. It's not for build.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
All these packages are required and used in scripts of package.json. I have tested this (run mvn build at clean vbox). And when I tried run
$ npm install -g yarn
$ yarn install
It failed because scripts from package.json couldn't resolve these packages.
Please note: I in this case I would add also following command:
$ apt-get install -y nodejs
\u041e\u0442\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u043e \u0441 \u0431\u0435\u0441\u043f\u0440\u043e\u0432\u043e\u0434\u043d\u043e\u0433\u043e \u0443\u0441\u0442\u0440\u043e\u0439\u0441\u0442\u0432\u0430 BlackBerry�
-----Original Message-----
From: Lee moon soo <no...@github.com>
Date: Sat, 11 Feb 2017 00:26:51
To: apache/zeppelin<ze...@noreply.github.com>
Reply-To: apache/zeppelin <re...@reply.github.com>
Cc: Dmytro Shkvyra<ds...@gmail.com>; Author<au...@noreply.github.com>
Subject: Re: [apache/zeppelin] [ZEPPELIN-1928]: Get a rid of preinstalled
external dependencies (nodejs and npm) for maven build (#2002)
Leemoonsoo commented on this pull request.
> # install required depepdencies and bower packages (only once)
-$ npm install -g yarn
-$ yarn install
+```
+$ npm install -g yarn bower npm-run-all rimraf grunt webpack karma karma-phantomjs-launcher
I'm don't think 'rimraf grunt webpack karma karma-phantomjs-launcher' are really a required package installed globally for development. Shell we remove them?
Also shell we not use 'npm-run-all' package?
After this PR, every existing developer will face error, such as
```
$ npm-run-all --parallel dev:server dev:watch
sh: npm-run-all: command not found
error Command failed with exit code 127.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
```
and all developer need to read README.md again and install necessary package. which is big impact compare to benefit that we can get from npm-run-all here.
What do you think?
--
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/zeppelin/pull/2002#pullrequestreview-21381858
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #2002: [ZEPPELIN-1928]: Get a rid of preinstalled exte...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra closed the pull request at:
https://github.com/apache/zeppelin/pull/2002
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/2002
@DmytroShkvyra Please read and try #2016
### Details
- yarn installed by frontend-maven-plugin can't run commands described in scripts tag (package.json)
- but npm is able to execute them.
- so, it's bug of the maven plugin.
- the most simple and appropriate way to solve this build failure is, use `npm` instead of `yarn`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
Incompatible with https://github.com/apache/zeppelin/pull/2016
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
BTW @Leemoonsoo , I agree with you that install libfontconfig with shell script is graceful solution. Unfortunately, in phantomjs package for linux absent this dependency and we can't build zeppelin at empty linux box without installation by hand all hidden dependences. I think it would better change phantomjs to another one.
\u041e\u0442\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u043e \u0441 \u0431\u0435\u0441\u043f\u0440\u043e\u0432\u043e\u0434\u043d\u043e\u0433\u043e \u0443\u0441\u0442\u0440\u043e\u0439\u0441\u0442\u0432\u0430 BlackBerry�
-----Original Message-----
From: Lee moon soo <no...@github.com>
Date: Sat, 11 Feb 2017 22:40:54
To: apache/zeppelin<ze...@noreply.github.com>
Reply-To: apache/zeppelin <re...@reply.github.com>
Cc: Dmytro Shkvyra<ds...@gmail.com>; Mention<me...@noreply.github.com>
Subject: Re: [apache/zeppelin] [ZEPPELIN-1928]: Get a rid of preinstalled
external dependencies (nodejs and npm) for maven build (#2002)
Leemoonsoo commented on this pull request.
> @@ -0,0 +1,2 @@
+#!/bin/bash
+( sudo apt-get install libfontconfig -y || sudo yum install fontconfig -y ) || exit 0;
I'm not sure it's best idea install package on the system without asking user. Shell we remove this file?
> @@ -120,34 +152,77 @@
</execution>
<execution>
- <id>yarn install</id>
- <goals>
- <goal>yarn</goal>
- </goals>
- <configuration>
- <arguments>install --no-lockfile</arguments>
- </configuration>
+ <id>yarn add bower</id>
+ <goals>
+ <goal>yarn</goal>
+ </goals>
+ <configuration>
+ <!--ignore-scripts-->
+ <arguments>add bower --no-lockfile --no-bin-links</arguments>
we don't need to run
'yarn add bower'
'yarn add karma-phantomjs-launcher'
and we can just run command defined in 'script' field in 'package.json'.
like
'yarn install'
'yarn build'
'yarn test'
we don't want to define the same command in the pom.xml that 'package.json' already has in 'script' field, such as
'bower install'
'webpack build'
'grunt pre-webpack-dist'
'grunt post-webpack-dist'
'karma test'
I think all changes made in 'executions' section of 'frontend-maven-plugin' are unnecessary and should be reverted.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/apache/zeppelin/pull/2002#pullrequestreview-21400884
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by lwmonster <gi...@git.apache.org>.
Github user lwmonster commented on the issue:
https://github.com/apache/zeppelin/pull/2002
Now it is 2017-09-20, and I cloned the master branch, and it still have error:
```
[INFO] [4/4] Building fresh packages...
[INFO] $ bower install --silent
[ERROR] [{
[ERROR] "code": "ESUDO",
[ERROR] "details": "Since bower is a user command, there is no need to execute it with superuser permissions.\nIf you're having permission errors when using bower without sudo, pleas
e spend a few minutes learning more about how your system should work and make any necessary repairs.\n\nhttp://www.joyent.com/blog/installing-node-and-npm\nhttps://gist.github.com/isa
acs/579814\n\nYou can however run a command with sudo using --allow-root option",
[ERROR] "id": "ESUDO",
[ERROR] "level": "error",
[ERROR] "data": {},
[ERROR] "message": "Cannot be run with sudo",
[ERROR] "stacktrace": "Error: Cannot be run with sudo\n at createError (/root/software/zeppelin-master/zeppelin-web/node_modules/bower/lib/util/createError.js:4:15)\n at rootCh
eck (/root/software/zeppelin-master/zeppelin-web/node_modules/bower/lib/util/rootCheck.js:26:24)\n at Object.<anonymous> (/root/software/zeppelin-master/zeppelin-web/node_modules/bo
wer/lib/bin/bower.js:33:1)\n at Module._compile (module.js:570:32)\n at Object.Module._extensions..js (module.js:579:10)\n at Module.load (module.js:487:32)\n at tryModuleL
oad (module.js:446:12)\n at Function.Module._load (module.js:438:3)\n at Module.require (module.js:497:17)\n at require (internal/module.js:20:19)"
[ERROR] }]
[ERROR] error Command failed with exit code 1.
[INFO] info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Zeppelin ........................................... SUCCESS [ 2.472 s]
[INFO] Zeppelin: Interpreter .............................. SUCCESS [ 5.529 s]
[INFO] Zeppelin: Display system apis ...................... SUCCESS [ 9.437 s]
[INFO] Zeppelin: Python interpreter ....................... SUCCESS [01:20 min]
[INFO] Zeppelin: Spark .................................... SUCCESS [ 19.324 s]
[INFO] Zeppelin: Zengine .................................. SUCCESS [ 6.182 s]
[INFO] Zeppelin: Spark dependencies ....................... SUCCESS [ 39.089 s]
[INFO] Zeppelin: Groovy interpreter ....................... SUCCESS [ 0.269 s]
[INFO] Zeppelin: Markdown interpreter ..................... SUCCESS [ 0.482 s]
[INFO] Zeppelin: Angular interpreter ...................... SUCCESS [ 0.292 s]
[INFO] Zeppelin: Shell interpreter ........................ SUCCESS [ 0.392 s]
[INFO] Zeppelin: Livy interpreter ......................... SUCCESS [ 2.483 s]
[INFO] Zeppelin: HBase interpreter ........................ SUCCESS [ 10.656 s]
[INFO] Zeppelin: Apache Pig Interpreter ................... SUCCESS [ 6.431 s]
[INFO] Zeppelin: JDBC interpreter ......................... SUCCESS [ 0.862 s]
[INFO] Zeppelin: File System Interpreters ................. SUCCESS [ 0.906 s]
[INFO] Zeppelin: Flink .................................... SUCCESS [ 4.536 s]
[INFO] Zeppelin: Apache Ignite interpreter ................ SUCCESS [ 1.260 s]
[INFO] Zeppelin: Kylin interpreter ........................ SUCCESS [ 0.659 s]
[INFO] Zeppelin: Lens interpreter ......................... SUCCESS [ 6.755 s]
[INFO] Zeppelin: Apache Cassandra interpreter ............. SUCCESS [ 35.240 s]
[INFO] Zeppelin: Elasticsearch interpreter ................ SUCCESS [ 2.318 s]
[INFO] Zeppelin: BigQuery interpreter ..................... SUCCESS [ 3.289 s]
[INFO] Zeppelin: Alluxio interpreter ...................... SUCCESS [ 1.814 s]
[INFO] Zeppelin: Scio ..................................... SUCCESS [ 26.458 s]
[INFO] Zeppelin: web Application .......................... FAILURE [01:04 min]
[INFO] Zeppelin: Server ................................... SKIPPED
[INFO] Zeppelin: Jupyter Support .......................... SKIPPED
[INFO] Zeppelin: Packaging distribution ................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 05:32 min
[INFO] Finished at: 2017-09-20T05:09:38Z
[INFO] Final Memory: 519M/2922M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.3:yarn (yarn install) on project zeppelin-web: Failed to run task: 'yarn install --no-lockfile' failed. (error code 1) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.3:yarn (yarn install) on project zeppelin-web: Failed to run task
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
```
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by lwmonster <gi...@git.apache.org>.
Github user lwmonster commented on the issue:
https://github.com/apache/zeppelin/pull/2002
Should I need to modify some files? @1ambda
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
@1ambda
>That's cool. You are trying to modify something without understanding. Anyway, We have 2 problems.
> local development
> you must install yarn if you want to use yarn in terminal. there is no exception.
> mvn package
> convert yarn to npm instead of this kind of big change.
1. I don't change anything that impact **local** development. You can see
https://github.com/apache/zeppelin/pull/2002/files#diff-606da5c2abcc66b817845ab483a4f1cd
2. If you really read comments in ZEPPELIN-1928 may be read that npm failed because "Max stack size" when we install this locally. So use yarn for build with maven is good idea.
My OS ():
```
[vagrant@c7 zeppelin-web]$ cat /etc/*-release
CentOS Linux release 7.3.1611 (Core)
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
CentOS Linux release 7.3.1611 (Core)
CentOS Linux release 7.3.1611 (Core)
```
P.S. When first assignee have asked for clarification nobody answer him. So say "That's cool. You are trying to modify something without understanding" was a little bit rude because I have tried almost all possible alternatives.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/2002
@DmytroShkvyra
If you want to use node modules in terminal, you have to install them globally. (e.g `$ yarn install`)
But in case of grunt (and karma so on..), we don't need to do that. Since they are used in `scripts` (package.json). That's what i meant.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by lwmonster <gi...@git.apache.org>.
Github user lwmonster commented on the issue:
https://github.com/apache/zeppelin/pull/2002
I installed nodejs and npm before package zeppelin, but another error occurs, a timeout occurs; someone says that it is the yarn problem, and I use the yarn-1.0.2 got a similar error as below, lots of people meet the same problem, can we don't use yarn?
```
[INFO]
[INFO] --- frontend-maven-plugin:1.3:yarn (yarn install) @ zeppelin-web ---
[INFO] Running 'yarn install --no-lockfile --network-concurrency 1' in /root/software/zeppelin-0.7.2/zeppelin-web
[INFO] yarn install v1.0.2
[INFO] [1/5] Validating package.json...
[INFO] [2/5] Resolving packages...
[WARNING] warning There appears to be trouble with your network connection. Retrying...
[WARNING] warning There appears to be trouble with your network connection. Retrying...
[WARNING] warning There appears to be trouble with your network connection. Retrying...
[WARNING] warning There appears to be trouble with your network connection. Retrying...
[WARNING] warning There appears to be trouble with your network connection. Retrying...
[WARNING] warning There appears to be trouble with your network connection. Retrying...
[ERROR] error An unexpected error occurred: "https://registry.yarnpkg.com/clean-css: ETIMEDOUT".
[INFO] info If you think this is a bug, please open a bug report with the information provided in "/root/software/zeppelin-0.7.2/zeppelin-web/yarn-error.log".
[INFO] info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
[WARNING] warning There appears to be trouble with your network connection. Retrying...
[WARNING] warning minimatch@0.3.0: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
[WARNING] warning There appears to be trouble with your network connection. Retrying...
[WARNING] warning There appears to be trouble with your network connection. Retrying...
[ERROR] Error: https://registry.yarnpkg.com/async: ETIMEDOUT
[ERROR] at Timeout._onTimeout (/root/software/zeppelin-0.7.2/zeppelin-web/node/yarn/dist/lib/cli.js:118034:19)
[ERROR] at ontimeout (timers.js:365:14)
[ERROR] at Timeout._onTimeout (/root/software/zeppelin-0.7.2/zeppelin-web/node/yarn/dist/lib/cli.js:118034:19)
[ERROR] at ontimeout (timers.js:365:14)
[ERROR] at tryOnTimeout (timers.js:237:5)
[ERROR] at Timer.listOnTimeout (timers.js:207:5)
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Zeppelin ........................................... SUCCESS [ 2.181 s]
[INFO] Zeppelin: Interpreter .............................. SUCCESS [ 2.417 s]
[INFO] Zeppelin: Zengine .................................. SUCCESS [ 2.840 s]
[INFO] Zeppelin: Display system apis ...................... SUCCESS [ 0.644 s]
```
---
[GitHub] zeppelin issue #2002: [ZEPPELIN-1928]: Get a rid of preinstalled external de...
Posted by DmytroShkvyra <gi...@git.apache.org>.
Github user DmytroShkvyra commented on the issue:
https://github.com/apache/zeppelin/pull/2002
Hi @Leemoonsoo, fontconfig only need to run pahantomjs browser during test phase of maven build. If you don't want run tests and install fontconfig you can pass -DSkip.test option to maven
\u041e\u0442\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u043e \u0441 \u0431\u0435\u0441\u043f\u0440\u043e\u0432\u043e\u0434\u043d\u043e\u0433\u043e \u0443\u0441\u0442\u0440\u043e\u0439\u0441\u0442\u0432\u0430 BlackBerry�
-----Original Message-----
From: Lee moon soo <no...@github.com>
Date: Sat, 11 Feb 2017 22:40:54
To: apache/zeppelin<ze...@noreply.github.com>
Reply-To: apache/zeppelin <re...@reply.github.com>
Cc: Dmytro Shkvyra<ds...@gmail.com>; Mention<me...@noreply.github.com>
Subject: Re: [apache/zeppelin] [ZEPPELIN-1928]: Get a rid of preinstalled
external dependencies (nodejs and npm) for maven build (#2002)
Leemoonsoo commented on this pull request.
> @@ -0,0 +1,2 @@
+#!/bin/bash
+( sudo apt-get install libfontconfig -y || sudo yum install fontconfig -y ) || exit 0;
I'm not sure it's best idea install package on the system without asking user. Shell we remove this file?
> @@ -120,34 +152,77 @@
</execution>
<execution>
- <id>yarn install</id>
- <goals>
- <goal>yarn</goal>
- </goals>
- <configuration>
- <arguments>install --no-lockfile</arguments>
- </configuration>
+ <id>yarn add bower</id>
+ <goals>
+ <goal>yarn</goal>
+ </goals>
+ <configuration>
+ <!--ignore-scripts-->
+ <arguments>add bower --no-lockfile --no-bin-links</arguments>
we don't need to run
'yarn add bower'
'yarn add karma-phantomjs-launcher'
and we can just run command defined in 'script' field in 'package.json'.
like
'yarn install'
'yarn build'
'yarn test'
we don't want to define the same command in the pom.xml that 'package.json' already has in 'script' field, such as
'bower install'
'webpack build'
'grunt pre-webpack-dist'
'grunt post-webpack-dist'
'karma test'
I think all changes made in 'executions' section of 'frontend-maven-plugin' are unnecessary and should be reverted.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/apache/zeppelin/pull/2002#pullrequestreview-21400884
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---