You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@zeppelin.apache.org by "paul8263 (via GitHub)" <gi...@apache.org> on 2025/02/12 06:44:24 UTC
[PR] [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.1.17.0 [zeppelin]
paul8263 opened a new pull request, #4908:
URL: https://github.com/apache/zeppelin/pull/4908
### What is this PR for?
Currently in Zeppelin HBase interpreter does not support HBase 2.x ruby syntax.
This patch add a profile which switches jruby version to 9.1.17.0, which supports HBase 2.x
This closes #ZEPPELIN-6163
### What type of PR is it?
Improvement
### Todos
### What is the Jira issue?
* Open an issue on Jira https://issues.apache.org/jira/browse/ZEPPELIN/
* Put link here, and add [ZEPPELIN-*Jira number*] in PR title, eg. [ZEPPELIN-533]
[ZEPPELIN-6163](https://issues.apache.org/jira/browse/ZEPPELIN-6163)
### How should this be tested?
* Strongly recommended: add automated unit tests for any new or changed behavior
* Outline any manual steps to test the PR here.
Compiled and tested manually.
### Screenshots (if appropriate)
### Questions:
* Does the license files need to update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "Reamer (via GitHub)" <gi...@apache.org>.
Reamer commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r2012297965
##########
hbase/src/main/java/org/apache/zeppelin/hbase/HbaseInterpreter.java:
##########
@@ -143,30 +145,25 @@ public Scheduler getScheduler() {
HbaseInterpreter.class.getName() + this.hashCode());
}
- @Override
- public List<InterpreterCompletion> completion(String buf, int cursor,
- InterpreterContext interpreterContext) {
- return null;
+ private String getScriptFileName(String paragraphId) {
+ return String.format("%s%s.txt", getScriptDir(), paragraphId);
}
- private static String getSystemDefault(
- String envName,
- String propertyName,
- String defaultValue) {
-
- if (envName != null && !envName.isEmpty()) {
- String envValue = System.getenv().get(envName);
- if (envValue != null) {
- return envValue;
- }
+ private String getScriptDir() {
+ String tmpProperty = System.getProperty("java.io.tmpdir");
Review Comment:
I'm not a fan of fixed strings. We should use a method directly from Java to create a temp directory or file.
The path must of course be memorized in order to remove it again later.
Files.createTempFile or Files.createTempDirectory should be useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Bumps jruby to 9.2.13.0 to support hbase 2.x [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1955513663
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
Hi @pan3793
Zeppelin 0.12.0 could now work with HBase 2.0.0 after I restored hbase/pom.xml of Zeppelin 0.12.0 to 0.11.2 and upgraded jruby to 9.2.13.0.
In Zeppelin HBase Interpreter, there are some dependencies short.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.1.17.0 [zeppelin]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1952242945
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
Could you please leave some brief comments here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.1.17.0 [zeppelin]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1952074401
##########
hbase/pom.xml:
##########
@@ -61,5 +61,13 @@
</plugin>
</plugins>
</build>
+ <profiles>
+ <profile>
+ <id>hbase-2.x</id>
Review Comment:
HBase 1.x is EOL, see HBASE-27286, I think we don't need a profile, just adapt to HBase 2.x
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.1.17.0 [zeppelin]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1952333942
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
this looks like jruby/jline version mismatch, not dep miss.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Bumps jruby to 9.2.13.0 to support hbase 2.x [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1954201913
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
Hi @pan3793 ,
I tested the latest Zeppelin 0.12.0 with HBase 2.0.0 and jruby 9.2.13.0. Got the exception like(executing 'list' and 'scan' respectively):
```
list
TABLE
ERROR: undefined method `getAdmin' for nil:NilClass
List all user tables in hbase. Optional regular expression parameter could
be used to filter the output. Examples:
hbase> list
hbase> list 'abc.*'
hbase> list 'ns:abc.*'
hbase> list 'ns:.*'
Took 0.0286 seconds
```
And:
```
scan
ERROR: undefined method `getTable' for nil:NilClass
Did you mean? get_table
Scan a table; pass table name and optionally a dictionary of scanner
specifications. Scanner specifications may include one or more of:
TIMERANGE, FILTER, LIMIT, STARTROW, STOPROW, ROWPREFIXFILTER, TIMESTAMP,
MAXLENGTH or COLUMNS, CACHE or RAW, VERSIONS, ALL_METRICS or METRICS
```
It might indicate that some of the methods are not reachable. I investigated further by comparing the output '$LOADED_FEATURES.each { |str| puts str }' in 0.11.2 and 0.12.0. Surprisingly they are the same.
The output is:
```
java.rb
jruby/util.rb
enumerator.rb
rational.rb
complex.rb
thread.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/unicode_normalize.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rbconfig.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/compatibility.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/defaults.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/deprecate.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/errors.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/exceptions.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/basic_specification.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/stub_specification.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/text.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/user_interaction.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification_policy.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/util/list.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/platform.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/version.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/requirement.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jars/post_install_hook.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jar_install_post_install_hook.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/defaults/jruby.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/util.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/dependency.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_gem.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/monitor.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_warn.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/defaults/operating_system.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/path_support.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/version.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/core_ext/name_error.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/levenshtein.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/jaro_winkler.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/delegate.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/name_error_checkers/class_name_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/name_error_checkers/variable_name_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/name_error_checkers.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/method_name_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/key_error_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/null_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/formatters/plain_formatter.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/readline/version.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/readline.jar
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/readline.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/completion.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/pathname.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jruby.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/admin.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/table.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/taskmonitor.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/quotas.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/security.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/visibility_labels.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/hbase.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/replication_admin.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/rsgroup_admin.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase_constants.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/stringio.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/formatter.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/status.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/version.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/table_help.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/whoami.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/processlist.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/alter.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/create.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/describe.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/disable.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/disable_all.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/is_disabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/drop.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/drop_all.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/enable.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/enable_all.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/is_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/exists.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/show_filters.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/alter_status.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/alter_async.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_table.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/locate_region.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_regions.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/create_namespace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/drop_namespace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/alter_namespace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/describe_namespace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_namespace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_namespace_tables.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/count.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/delete.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/deleteall.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_counter.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/incr.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/put.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/scan.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/truncate.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/truncate_preserve.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/append.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_splits.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/assign.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/balancer.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/balance_switch.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/balancer_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/normalize.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/normalizer_switch.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/normalizer_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/is_in_maintenance_mode.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/close_region.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/compact.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/flush.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/major_compact.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/split.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/merge_region.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/unassign.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/zk_dump.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/wal_roll.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/catalogjanitor_run.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/catalogjanitor_switch.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/catalogjanitor_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/cleaner_chore_run.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/cleaner_chore_switch.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/cleaner_chore_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/compact_rs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/compaction_state.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/trace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/splitormerge_switch.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/splitormerge_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/clear_compaction_queues.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_deadservers.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/clear_deadservers.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/clear_block_cache.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/add_peer.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/remove_peer.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_peers.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/enable_peer.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/disable_peer.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_replicate_all.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_namespaces.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/append_peer_namespaces.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/remove_peer_namespaces.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_exclude_namespaces.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/show_peer_tableCFs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_tableCFs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_exclude_tableCFs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_bandwidth.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_replicated_tables.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/append_peer_tableCFs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/remove_peer_tableCFs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/enable_table_replication.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/disable_table_replication.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_peer_config.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_peer_configs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/update_peer_config.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/snapshot.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/clone_snapshot.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/restore_snapshot.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/delete_snapshot.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/delete_all_snapshot.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/delete_table_snapshots.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/date/format.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/date.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/time.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_snapshots.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_table_snapshots.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/update_config.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/update_all_config.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_quota.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_quotas.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_quota_table_sizes.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_quota_snapshots.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_snapshot_sizes.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_security_capabilities.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/grant.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/revoke.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/user_permission.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/abort_procedure.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/version.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/ostruct.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/generic_object.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/common.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/ext/parser.jar
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/ext/generator.jar
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/ext.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_procedures.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_locks.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/add_labels.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_labels.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_auths.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_auths.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/clear_auths.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_visibility.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_rsgroups.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/add_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/remove_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/balance_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move_servers_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move_tables_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move_namespaces_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move_servers_tables_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move_servers_namespaces_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_server_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_table_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/remove_servers_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/e2mmap.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/init.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/workspace.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/inspector.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/src_encoding.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/magic-file.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/output-method.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/extend-command.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/notifier.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/slex.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-token.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/locale.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/version.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/irb/hirb.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/bundler_version_finder.rb
uri:classloader:/jruby/path_helper.rb
```
I still have not figured out what is going wrong yet.
One more thing is that the entrypoints of HBase shell versioned 2.0.0 and 2.4.7 are different. HBase 2.0.0 is the ruby script located at bin/hirb.rb, while in 2.4.7 all ruby scripts are packaged in hbase-shell.jar. It adapted to a standardalone jar. See https://github.com/jruby/jruby/wiki/StandaloneJarsAndClasses#standalone-executable-jar-files . To support newer HBase, the HbaseInterpreter class needs some updates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "jongyoul (via GitHub)" <gi...@apache.org>.
jongyoul commented on PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#issuecomment-2811950178
@paul8263 Could you please rebase your change? We recently fixed some flaky test issues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.1.17.0 [zeppelin]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1952177704
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
May I ask why 9.1.17.0 is chosen? I see [HBase 2.6.1](https://mvnrepository.com/artifact/org.apache.hbase/hbase-shell/2.6.1-hadoop3)(the latest version of HBase 2.x) pulls JRuby 9.3.13.0, does it work for lower HBase version?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.1.17.0 [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1952280707
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
By the way, I tested hbase interperter compiled with JRuby 9.3.13.0, connecting early version HBase i.e HBase 2.0.0. It required additional dependencies named jline. However interperter compiled with JRuby 9.1.17.0 worked like a charm. I guess higher version of jruby may require addition dependencies that lower version of HBase does not provide.
Exceptions are:
```
org.apache.zeppelin.interpreter.InterpreterException: org.jruby.embed.EvalFailedException: (LoadError) load error: irb/completion -- java.lang.NoSuchMethodError: jline.console.completer.CandidateListCompletionHandler.setPrintSpaceAfterFullCompletion(Z)V
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:76)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:861)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:769)
at org.apache.zeppelin.scheduler.Job.run(Job.java:186)
at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:135)
at org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "jongyoul (via GitHub)" <gi...@apache.org>.
jongyoul merged PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Bumps jruby to 9.2.13.0 to support hbase 2.x [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1955513663
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
Hi @pan3793
Zeppelin 0.12.0 could now work with HBase 2.0.0 after I restored hbase/pom.xml of Zeppelin 0.12.0 to 0.11.2 and upgraded jruby to 9.2.13.0.
In Zeppelin 0.12.0 HBase Interpreter, there are some dependencies short.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.1.17.0 [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1952318499
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
After solving jline dependency issue, finally I got this exception:
```
org.apache.zeppelin.interpreter.InterpreterException: org.jruby.embed.EvalFailedException: (NoMethodError) undefined method `runtime' for JRuby:Module
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:76)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:861)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:769)
at org.apache.zeppelin.scheduler.Job.run(Job.java:186)
at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:135)
at org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.jruby.embed.EvalFailedException: (NoMethodError) undefined method `runtime' for JRuby:Module
at org.jruby.embed.internal.EmbedEvalUnitImpl.run(EmbedEvalUnitImpl.java:134)
at org.jruby.embed.ScriptingContainer.runUnit(ScriptingContainer.java:1296)
at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:1327)
at org.apache.zeppelin.hbase.HbaseInterpreter.open(HbaseInterpreter.java:97)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
... 8 more
Caused by: org.jruby.exceptions.NoMethodError: (NoMethodError) undefined method `runtime' for JRuby:Module
at RUBY.`(uri:classloader:/jruby/kernel/jruby/process_manager.rb:10)
at RUBY.`(uri:classloader:/jruby/kernel/jruby/process_manager.rb:46)
at RUBY.initialize(/usr/hdp/3.0.1.0-187/hbase/lib/ruby/irb/hirb.rb:46)
at org.jruby.RubyClass.new(org/jruby/RubyClass.java:883)
at RUBY.start(hirb.rb:181)
at RUBY.<main>(hirb.rb:193)
```
It seems there are some compatibility issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.1.17.0 [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1952174620
##########
hbase/pom.xml:
##########
@@ -61,5 +61,13 @@
</plugin>
</plugins>
</build>
+ <profiles>
+ <profile>
+ <id>hbase-2.x</id>
Review Comment:
Hi @pan3793 ,
Great. I will simply upgrade it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "Reamer (via GitHub)" <gi...@apache.org>.
Reamer commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r2013500771
##########
hbase/src/main/java/org/apache/zeppelin/hbase/HbaseInterpreter.java:
##########
@@ -143,30 +145,25 @@ public Scheduler getScheduler() {
HbaseInterpreter.class.getName() + this.hashCode());
}
- @Override
- public List<InterpreterCompletion> completion(String buf, int cursor,
- InterpreterContext interpreterContext) {
- return null;
+ private String getScriptFileName(String paragraphId) {
+ return String.format("%s%s.txt", getScriptDir(), paragraphId);
}
- private static String getSystemDefault(
- String envName,
- String propertyName,
- String defaultValue) {
-
- if (envName != null && !envName.isEmpty()) {
- String envValue = System.getenv().get(envName);
- if (envValue != null) {
- return envValue;
- }
+ private String getScriptDir() {
+ String tmpProperty = System.getProperty("java.io.tmpdir");
Review Comment:
Since the HashMap<ParagraphId, process> already exists as an object variable, I would have added a second Hashmap<ParagraphId, File>.
With a composition of the path, it can potentially happen that two different HBase interpreters access the same file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "Reamer (via GitHub)" <gi...@apache.org>.
Reamer commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r2018103544
##########
hbase/src/main/java/org/apache/zeppelin/hbase/HbaseInterpreter.java:
##########
@@ -143,30 +156,26 @@ public Scheduler getScheduler() {
HbaseInterpreter.class.getName() + this.hashCode());
}
- @Override
- public List<InterpreterCompletion> completion(String buf, int cursor,
- InterpreterContext interpreterContext) {
- return null;
+ private void stopProcess(String paragraphId) {
+ if (runningProcesses.containsKey(paragraphId)) {
+ final Executor executor = runningProcesses.get(paragraphId);
+ final ExecuteWatchdog watchdog = executor.getWatchdog();
+ watchdog.destroyProcess();
+ runningProcesses.remove(paragraphId);
+ }
}
- private static String getSystemDefault(
- String envName,
- String propertyName,
- String defaultValue) {
-
- if (envName != null && !envName.isEmpty()) {
- String envValue = System.getenv().get(envName);
- if (envValue != null) {
- return envValue;
- }
- }
+ private File createTempFile(String paragraphId) throws IOException {
+ File temp = Files.createTempFile(
+ Paths.get(System.getProperty("java.io.tmpdir"), "zeppelin-hbase-scripts"), paragraphId, ".txt").toFile();
+ tempFiles.put(paragraphId, temp);
+ return temp;
+ }
- if (propertyName != null && !propertyName.isEmpty()) {
- String propValue = System.getProperty(propertyName);
- if (propValue != null) {
- return propValue;
- }
+ private void deleteTempFile(String paragraphId) {
+ if (tempFiles.containsKey(paragraphId)) {
Review Comment:
That is correct. The adaptation of the algorithm in the `stopProcess` method is still missing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "jongyoul (via GitHub)" <gi...@apache.org>.
jongyoul commented on PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#issuecomment-2791847188
I checked the code, but there are no test cases. Don't we need to test it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.1.17.0 [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1952280707
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
By the way, I tested hbase interperter compiled with JRuby 9.3.13.0, connecting early version HBase i.e HBase 2.0.0. It required additional dependencies named jline. However interperter compiled with JRuby 9.1.17.0 worked like a charm. I guess higher version of jruby may require addition dependencies that lower version of HBase does not provide.
Exceptions are:
`org.apache.zeppelin.interpreter.InterpreterException: org.jruby.embed.EvalFailedException: (LoadError) load error: irb/completion -- java.lang.NoSuchMethodError: jline.console.completer.CandidateListCompletionHandler.setPrintSpaceAfterFullCompletion(Z)V
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:76)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:861)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:769)
at org.apache.zeppelin.scheduler.Job.run(Job.java:186)
at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:135)
at org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "jongyoul (via GitHub)" <gi...@apache.org>.
jongyoul commented on PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#issuecomment-2791988454
If you need to download HBase, you can follow the way of https://github.com/apache/zeppelin/blob/3eae255c109aa224c5b5ae3f2c986510d2a26ee9/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncherTest.java#L54
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r2013278033
##########
hbase/src/main/java/org/apache/zeppelin/hbase/HbaseInterpreter.java:
##########
@@ -143,30 +145,25 @@ public Scheduler getScheduler() {
HbaseInterpreter.class.getName() + this.hashCode());
}
- @Override
- public List<InterpreterCompletion> completion(String buf, int cursor,
- InterpreterContext interpreterContext) {
- return null;
+ private String getScriptFileName(String paragraphId) {
+ return String.format("%s%s.txt", getScriptDir(), paragraphId);
}
- private static String getSystemDefault(
- String envName,
- String propertyName,
- String defaultValue) {
-
- if (envName != null && !envName.isEmpty()) {
- String envValue = System.getenv().get(envName);
- if (envValue != null) {
- return envValue;
- }
+ private String getScriptDir() {
+ String tmpProperty = System.getProperty("java.io.tmpdir");
Review Comment:
Hi @Reamer ，
Thanks for you review.
I agree with you but both interpret and cancel method need the temp file. The overhead of caching a file object might be higher than the path.
In the commit ead10d9 I simplfied the getScriptFileName method.
Correct me if I am wrong.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r2017772301
##########
hbase/src/main/java/org/apache/zeppelin/hbase/HbaseInterpreter.java:
##########
@@ -143,30 +156,26 @@ public Scheduler getScheduler() {
HbaseInterpreter.class.getName() + this.hashCode());
}
- @Override
- public List<InterpreterCompletion> completion(String buf, int cursor,
- InterpreterContext interpreterContext) {
- return null;
+ private void stopProcess(String paragraphId) {
+ if (runningProcesses.containsKey(paragraphId)) {
+ final Executor executor = runningProcesses.get(paragraphId);
+ final ExecuteWatchdog watchdog = executor.getWatchdog();
+ watchdog.destroyProcess();
+ runningProcesses.remove(paragraphId);
+ }
}
- private static String getSystemDefault(
- String envName,
- String propertyName,
- String defaultValue) {
-
- if (envName != null && !envName.isEmpty()) {
- String envValue = System.getenv().get(envName);
- if (envValue != null) {
- return envValue;
- }
- }
+ private File createTempFile(String paragraphId) throws IOException {
+ File temp = Files.createTempFile(
+ Paths.get(System.getProperty("java.io.tmpdir"), "zeppelin-hbase-scripts"), paragraphId, ".txt").toFile();
+ tempFiles.put(paragraphId, temp);
+ return temp;
+ }
- if (propertyName != null && !propertyName.isEmpty()) {
- String propValue = System.getProperty(propertyName);
- if (propValue != null) {
- return propValue;
- }
+ private void deleteTempFile(String paragraphId) {
+ if (tempFiles.containsKey(paragraphId)) {
Review Comment:
@Reamer ,
Great idea. As the return value of the map's remove method indicates the entry to be removed, we should use it instead, avoiding iterating the map again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.1.17.0 [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1952318499
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
After solving jline dependency issue, finally I got this exception:
`
org.apache.zeppelin.interpreter.InterpreterException: org.jruby.embed.EvalFailedException: (NoMethodError) undefined method `runtime' for JRuby:Module
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:76)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:861)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:769)
at org.apache.zeppelin.scheduler.Job.run(Job.java:186)
at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:135)
at org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.jruby.embed.EvalFailedException: (NoMethodError) undefined method `runtime' for JRuby:Module
at org.jruby.embed.internal.EmbedEvalUnitImpl.run(EmbedEvalUnitImpl.java:134)
at org.jruby.embed.ScriptingContainer.runUnit(ScriptingContainer.java:1296)
at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:1327)
at org.apache.zeppelin.hbase.HbaseInterpreter.open(HbaseInterpreter.java:97)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
... 8 more
Caused by: org.jruby.exceptions.NoMethodError: (NoMethodError) undefined method `runtime' for JRuby:Module
at RUBY.`(uri:classloader:/jruby/kernel/jruby/process_manager.rb:10)
at RUBY.`(uri:classloader:/jruby/kernel/jruby/process_manager.rb:46)
at RUBY.initialize(/usr/hdp/3.0.1.0-187/hbase/lib/ruby/irb/hirb.rb:46)
at org.jruby.RubyClass.new(org/jruby/RubyClass.java:883)
at RUBY.start(hirb.rb:181)
at RUBY.<main>(hirb.rb:193)
`
It seems there are some compatible issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Bumps jruby to 9.2.13.0 to support hbase 2.x [zeppelin]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1953739285
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
Does Ruby 9.2.13.0 also work for newly HBase versions, e.g. 2.6? we should choose a version that supports broad HBase versions.
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
Does JRuby 9.2.13.0 also work for newly HBase versions, e.g. 2.6? we should choose a version that supports broad HBase versions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.1.17.0 [zeppelin]
Posted by "jongyoul (via GitHub)" <gi...@apache.org>.
jongyoul commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1952075694
##########
hbase/pom.xml:
##########
@@ -61,5 +61,13 @@
</plugin>
</plugins>
</build>
+ <profiles>
+ <profile>
+ <id>hbase-2.x</id>
Review Comment:
Looks good!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Bumps jruby to 9.2.13.0 to support hbase 2.x [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1953647929
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
Hi @pan3793 ,
Thank you for your reply.
I got a breakthough. [jruby-7221](https://github.com/jruby/jruby/issues/7221) says 9.3.0.0 need to require "jruby" to avoid undefined method 'runtime' for JRuby:Module.
I tested hbase interpreter with JRuby 9.2.13.0 and it worked fine for HBase 2.0.0.
Once I upgraded JRuby to 9.3.13.0, the same exception occured. But it could be fixed by adding`require 'jruby'` at the beginning of $HBASE_HOME/lib/ruby/irb/hirb.rb (bin/hirb.rb also works) for HBase 2.0.0.
In conclusion, personally I prefer using JRuby 9.2.13.0. Correct me if I was wrong.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.1.17.0 [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1952237058
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
As [HBASE-27585](https://issues.apache.org/jira/browse/HBASE-27585) states, higher version of JRuby drops support for Ruby 2.5, which might have side effect for lower version of HBase 2.x
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "jongyoul (via GitHub)" <gi...@apache.org>.
jongyoul commented on PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#issuecomment-2820231131
merged into master and branch-0.12
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Bumps jruby to 9.2.13.0 to support hbase 2.x [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1954201913
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
Hi @pan3793 ,
I tested the latest Zeppelin 0.12.0 with HBase 2.0.0 and jruby 9.2.13.0. Got the exception like(executing 'list' and 'scan' respectively):
```
list
TABLE
ERROR: undefined method `getAdmin' for nil:NilClass
List all user tables in hbase. Optional regular expression parameter could
be used to filter the output. Examples:
hbase> list
hbase> list 'abc.*'
hbase> list 'ns:abc.*'
hbase> list 'ns:.*'
Took 0.0286 seconds
```
And:
```
scan
ERROR: undefined method `getTable' for nil:NilClass
Did you mean? get_table
Scan a table; pass table name and optionally a dictionary of scanner
specifications. Scanner specifications may include one or more of:
TIMERANGE, FILTER, LIMIT, STARTROW, STOPROW, ROWPREFIXFILTER, TIMESTAMP,
MAXLENGTH or COLUMNS, CACHE or RAW, VERSIONS, ALL_METRICS or METRICS
```
Zeppelin 0.11.2 works fine.
It might indicate that some of the methods are not reachable. I investigated further by comparing the output '$LOADED_FEATURES.each { |str| puts str }' in 0.11.2 and 0.12.0. Surprisingly they are the same.
The output is:
```
java.rb
jruby/util.rb
enumerator.rb
rational.rb
complex.rb
thread.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/unicode_normalize.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rbconfig.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/compatibility.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/defaults.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/deprecate.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/errors.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/exceptions.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/basic_specification.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/stub_specification.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/text.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/user_interaction.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification_policy.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/util/list.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/platform.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/version.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/requirement.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jars/post_install_hook.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jar_install_post_install_hook.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/defaults/jruby.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/util.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/dependency.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_gem.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/monitor.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_warn.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/defaults/operating_system.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/path_support.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/version.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/core_ext/name_error.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/levenshtein.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/jaro_winkler.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/delegate.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/name_error_checkers/class_name_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/name_error_checkers/variable_name_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/name_error_checkers.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/method_name_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/key_error_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/null_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/formatters/plain_formatter.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/readline/version.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/readline.jar
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/readline.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/completion.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/pathname.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jruby.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/admin.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/table.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/taskmonitor.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/quotas.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/security.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/visibility_labels.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/hbase.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/replication_admin.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/rsgroup_admin.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase_constants.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/stringio.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/formatter.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/status.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/version.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/table_help.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/whoami.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/processlist.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/alter.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/create.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/describe.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/disable.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/disable_all.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/is_disabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/drop.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/drop_all.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/enable.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/enable_all.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/is_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/exists.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/show_filters.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/alter_status.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/alter_async.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_table.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/locate_region.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_regions.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/create_namespace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/drop_namespace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/alter_namespace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/describe_namespace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_namespace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_namespace_tables.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/count.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/delete.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/deleteall.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_counter.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/incr.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/put.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/scan.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/truncate.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/truncate_preserve.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/append.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_splits.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/assign.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/balancer.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/balance_switch.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/balancer_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/normalize.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/normalizer_switch.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/normalizer_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/is_in_maintenance_mode.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/close_region.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/compact.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/flush.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/major_compact.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/split.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/merge_region.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/unassign.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/zk_dump.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/wal_roll.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/catalogjanitor_run.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/catalogjanitor_switch.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/catalogjanitor_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/cleaner_chore_run.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/cleaner_chore_switch.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/cleaner_chore_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/compact_rs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/compaction_state.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/trace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/splitormerge_switch.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/splitormerge_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/clear_compaction_queues.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_deadservers.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/clear_deadservers.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/clear_block_cache.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/add_peer.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/remove_peer.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_peers.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/enable_peer.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/disable_peer.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_replicate_all.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_namespaces.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/append_peer_namespaces.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/remove_peer_namespaces.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_exclude_namespaces.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/show_peer_tableCFs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_tableCFs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_exclude_tableCFs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_bandwidth.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_replicated_tables.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/append_peer_tableCFs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/remove_peer_tableCFs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/enable_table_replication.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/disable_table_replication.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_peer_config.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_peer_configs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/update_peer_config.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/snapshot.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/clone_snapshot.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/restore_snapshot.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/delete_snapshot.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/delete_all_snapshot.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/delete_table_snapshots.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/date/format.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/date.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/time.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_snapshots.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_table_snapshots.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/update_config.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/update_all_config.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_quota.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_quotas.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_quota_table_sizes.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_quota_snapshots.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_snapshot_sizes.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_security_capabilities.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/grant.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/revoke.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/user_permission.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/abort_procedure.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/version.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/ostruct.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/generic_object.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/common.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/ext/parser.jar
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/ext/generator.jar
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/ext.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_procedures.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_locks.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/add_labels.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_labels.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_auths.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_auths.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/clear_auths.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_visibility.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_rsgroups.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/add_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/remove_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/balance_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move_servers_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move_tables_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move_namespaces_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move_servers_tables_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move_servers_namespaces_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_server_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_table_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/remove_servers_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/e2mmap.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/init.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/workspace.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/inspector.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/src_encoding.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/magic-file.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/output-method.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/extend-command.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/notifier.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/slex.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-token.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/locale.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/version.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/irb/hirb.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/bundler_version_finder.rb
uri:classloader:/jruby/path_helper.rb
```
I still have not figured out what is going wrong yet.
One more thing is that the entrypoints of HBase shell versioned 2.0.0 and 2.4.7 are different. HBase 2.0.0 is the ruby script located at bin/hirb.rb, while in 2.4.7 all ruby scripts are packaged in hbase-shell.jar. It adapted to a standardalone jar. See https://github.com/jruby/jruby/wiki/StandaloneJarsAndClasses#standalone-executable-jar-files . To support newer HBase, the HbaseInterpreter class needs some updates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#issuecomment-2791938330
> I checked the code, but there are no test cases. Don't we need to test it?
Err.. I have compiled and tested the code manually. But if we need UT, a HBase environment is required. What should I do next?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#issuecomment-2795962898
> If you need to download HBase, you can follow the way of
>
> https://github.com/apache/zeppelin/blob/3eae255c109aa224c5b5ae3f2c986510d2a26ee9/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncherTest.java#L54
>
> WDYT?
Hi @jongyoul ,
I added HBase integration test. If there are some test errors I will fix it later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "Reamer (via GitHub)" <gi...@apache.org>.
Reamer commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r2012297965
##########
hbase/src/main/java/org/apache/zeppelin/hbase/HbaseInterpreter.java:
##########
@@ -143,30 +145,25 @@ public Scheduler getScheduler() {
HbaseInterpreter.class.getName() + this.hashCode());
}
- @Override
- public List<InterpreterCompletion> completion(String buf, int cursor,
- InterpreterContext interpreterContext) {
- return null;
+ private String getScriptFileName(String paragraphId) {
+ return String.format("%s%s.txt", getScriptDir(), paragraphId);
}
- private static String getSystemDefault(
- String envName,
- String propertyName,
- String defaultValue) {
-
- if (envName != null && !envName.isEmpty()) {
- String envValue = System.getenv().get(envName);
- if (envValue != null) {
- return envValue;
- }
+ private String getScriptDir() {
+ String tmpProperty = System.getProperty("java.io.tmpdir");
Review Comment:
I'm not a fan of fixed strings. We should use a method directly from Java to create a temp directory.
Files.createTempFile
or Files.createTempDirectory should be useful.
##########
docs/interpreter/hbase.md:
##########
@@ -28,19 +28,7 @@ limitations under the License.
To get start with HBase, please see [HBase Quickstart](https://hbase.apache.org/book.html#quickstart).
## HBase release supported
-By default, Zeppelin is built against HBase 1.0.x releases. To work with HBase 1.1.x releases, use the following build command:
-
-```bash
-# HBase 1.1.4
-./mvnw clean package -DskipTests -Phadoop-2.6 -Dhadoop.version=2.6.0 -P build-distr -Dhbase.hbase.version=1.1.4 -Dhbase.hadoop.version=2.6.0
-```
-
-To work with HBase 1.2.0+, use the following build command:
-
-```bash
-# HBase 1.2.0
-./mvnw clean package -DskipTests -Phadoop-2.6 -Dhadoop.version=2.6.0 -P build-distr -Dhbase.hbase.version=1.2.0 -Dhbase.hadoop.version=2.6.0
-```
+Zeppelin is built against HBase 2.x releases.
Review Comment:
If you rely on a shell script, this statement is no longer correct, is it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.1.17.0 [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1952318499
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
After solving jline dependency issue, finally I got this exception:
`
org.apache.zeppelin.interpreter.InterpreterException: org.jruby.embed.EvalFailedException: (NoMethodError) undefined method `runtime' for JRuby:Module
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:76)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:861)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:769)
at org.apache.zeppelin.scheduler.Job.run(Job.java:186)
at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:135)
at org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.jruby.embed.EvalFailedException: (NoMethodError) undefined method `runtime' for JRuby:Module
at org.jruby.embed.internal.EmbedEvalUnitImpl.run(EmbedEvalUnitImpl.java:134)
at org.jruby.embed.ScriptingContainer.runUnit(ScriptingContainer.java:1296)
at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:1327)
at org.apache.zeppelin.hbase.HbaseInterpreter.open(HbaseInterpreter.java:97)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
... 8 more
Caused by: org.jruby.exceptions.NoMethodError: (NoMethodError) undefined method `runtime' for JRuby:Module
at RUBY.`(uri:classloader:/jruby/kernel/jruby/process_manager.rb:10)
at RUBY.`(uri:classloader:/jruby/kernel/jruby/process_manager.rb:46)
at RUBY.initialize(/usr/hdp/3.0.1.0-187/hbase/lib/ruby/irb/hirb.rb:46)
at org.jruby.RubyClass.new(org/jruby/RubyClass.java:883)
at RUBY.start(hirb.rb:181)
at RUBY.<main>(hirb.rb:193)
`
It seems there are some compatibility issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Bumps jruby to 9.2.13.0 to support hbase 2.x [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1953647929
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
Hi @pan3793 ,
Thank you for your reply.
I got a breakthough. [jruby-7221](https://github.com/jruby/jruby/issues/7221) says 9.3.0.0 need to require "jruby" to avoid undefined method 'runtime' for JRuby:Module.
I tested hbase interperter with JRuby 9.2.13.0 and it worked fine for HBase 2.0.0.
Once I upgraded JRuby to 9.3.13.0, the same exception occured. But it could be fixed by adding`require 'jruby'` at the beginning of $HBASE_HOME/lib/ruby/irb/hirb.rb (bin/hirb.rb also works) for HBase 2.0.0.
In conclusion, personally I prefer using JRuby 9.2.13.0. Correct me if I was wrong.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.1.17.0 [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1953647929
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
Hi @pan3793 ,
Thank you for your reply.
I got a breakthough. [jruby-7221](https://github.com/jruby/jruby/issues/7221) says 9.3.0.0 need to require "jruby" to avoid undefined method 'runtime' for JRuby:Module.
I tested hbase interperter with JRuby 9.2.13.0 and it worked fine for HBase 2.0.0.
Once I upgraded JRuby to 9.3.13.0, the same exception occured. But it could be fixed by adding`require 'jruby'` at the beginning of $HBASE_HOME/lib/ruby/irb/hirb.rb (for HBase 2.0.0).
In conclusion, personally I prefer using JRuby 9.2.13.0. Correct me if I was wrong.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r1955727986
##########
hbase/pom.xml:
##########
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
- <jruby.version>1.6.8</jruby.version>
+ <jruby.version>9.1.17.0</jruby.version>
Review Comment:
Hi @pan3793
Finally I decided to dump jruby and use remote execution of "hbase shell" with scripts in the notebook written in a temporary txt file. Now both HBase 2.0.0 and HBase 2.4.7 works fine. Other versions of HBase should work as well.
Now the unit test requires a HBase environment. To bypass the CI I removed some of the tests. Is there any better solutions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "jongyoul (via GitHub)" <gi...@apache.org>.
jongyoul commented on PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#issuecomment-2817560559
@Reamer I'm just wondering if the CI isn't started automatically.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "Reamer (via GitHub)" <gi...@apache.org>.
Reamer commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r2015904362
##########
hbase/src/main/java/org/apache/zeppelin/hbase/HbaseInterpreter.java:
##########
@@ -143,30 +156,26 @@ public Scheduler getScheduler() {
HbaseInterpreter.class.getName() + this.hashCode());
}
- @Override
- public List<InterpreterCompletion> completion(String buf, int cursor,
- InterpreterContext interpreterContext) {
- return null;
+ private void stopProcess(String paragraphId) {
+ if (runningProcesses.containsKey(paragraphId)) {
Review Comment:
The deletion of values can also be written more elegantly here. The code currently accesses the map three times.
1) containsKey
2) get
3) remove
##########
hbase/src/main/java/org/apache/zeppelin/hbase/HbaseInterpreter.java:
##########
@@ -143,30 +156,26 @@ public Scheduler getScheduler() {
HbaseInterpreter.class.getName() + this.hashCode());
}
- @Override
- public List<InterpreterCompletion> completion(String buf, int cursor,
- InterpreterContext interpreterContext) {
- return null;
+ private void stopProcess(String paragraphId) {
+ if (runningProcesses.containsKey(paragraphId)) {
+ final Executor executor = runningProcesses.get(paragraphId);
+ final ExecuteWatchdog watchdog = executor.getWatchdog();
+ watchdog.destroyProcess();
+ runningProcesses.remove(paragraphId);
+ }
}
- private static String getSystemDefault(
- String envName,
- String propertyName,
- String defaultValue) {
-
- if (envName != null && !envName.isEmpty()) {
- String envValue = System.getenv().get(envName);
- if (envValue != null) {
- return envValue;
- }
- }
+ private File createTempFile(String paragraphId) throws IOException {
+ File temp = Files.createTempFile(
+ Paths.get(System.getProperty("java.io.tmpdir"), "zeppelin-hbase-scripts"), paragraphId, ".txt").toFile();
+ tempFiles.put(paragraphId, temp);
+ return temp;
+ }
- if (propertyName != null && !propertyName.isEmpty()) {
- String propValue = System.getProperty(propertyName);
- if (propValue != null) {
- return propValue;
- }
+ private void deleteTempFile(String paragraphId) {
+ if (tempFiles.containsKey(paragraphId)) {
Review Comment:
More elegant style for the same behavior.
```
File tmpFile = tempFiles.remove(paragraphId);
if (tmpFile != null) {
FileUtils.deleteQuietly(tmpFile);
}
```
With this code style, the map is only searched once and the program execution is faster.
I know that this map is usually very small and does not play a role on today's hardware. But maybe you have other projects where the maps are very large and so you can optimize your application accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6163] HBase interpreter supports hbase-2.x [zeppelin]
Posted by "paul8263 (via GitHub)" <gi...@apache.org>.
paul8263 commented on code in PR #4908:
URL: https://github.com/apache/zeppelin/pull/4908#discussion_r2018295224
##########
hbase/src/main/java/org/apache/zeppelin/hbase/HbaseInterpreter.java:
##########
@@ -143,30 +156,26 @@ public Scheduler getScheduler() {
HbaseInterpreter.class.getName() + this.hashCode());
}
- @Override
- public List<InterpreterCompletion> completion(String buf, int cursor,
- InterpreterContext interpreterContext) {
- return null;
+ private void stopProcess(String paragraphId) {
+ if (runningProcesses.containsKey(paragraphId)) {
Review Comment:
Hi @Reamer ,
Thanks. I will get it done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org