You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hivemall.apache.org by my...@apache.org on 2018/04/10 05:16:20 UTC
incubator-hivemall git commit: [HIVEMALL-188] Avoid KryoException:
java.lang.NullPointerException
Repository: incubator-hivemall
Updated Branches:
refs/heads/master 1e1b77ea4 -> 9580f0a8a
[HIVEMALL-188] Avoid KryoException: java.lang.NullPointerException
## What changes were proposed in this pull request?
Fix a bug in `tokenize_ja` that occasionally raises `KryoException: java.lang.NullPointerException`
## What type of PR is it?
Bug Fix
## What is the Jira issue?
https://issues.apache.org/jira/browse/HIVEMALL-188
## How was this patch tested?
Manual tests
## Checklist
(Please remove this section if not needed; check `x` for YES, blank for NO)
- [x] Did you apply source code formatter, i.e., `mvn formatter:format`, for your commit?
Author: Takuya Kitazawa <k....@gmail.com>
Closes #142 from takuti/HIVEMALL-188.
Project: http://git-wip-us.apache.org/repos/asf/incubator-hivemall/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hivemall/commit/9580f0a8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hivemall/tree/9580f0a8
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hivemall/diff/9580f0a8
Branch: refs/heads/master
Commit: 9580f0a8aba1aff381d7a75620999f61dfd8f3f5
Parents: 1e1b77e
Author: Takuya Kitazawa <k....@gmail.com>
Authored: Tue Apr 10 14:16:13 2018 +0900
Committer: Makoto Yui <my...@apache.org>
Committed: Tue Apr 10 14:16:13 2018 +0900
----------------------------------------------------------------------
nlp/src/main/java/hivemall/nlp/tokenizer/KuromojiUDF.java | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hivemall/blob/9580f0a8/nlp/src/main/java/hivemall/nlp/tokenizer/KuromojiUDF.java
----------------------------------------------------------------------
diff --git a/nlp/src/main/java/hivemall/nlp/tokenizer/KuromojiUDF.java b/nlp/src/main/java/hivemall/nlp/tokenizer/KuromojiUDF.java
index 411c89e..384c317 100644
--- a/nlp/src/main/java/hivemall/nlp/tokenizer/KuromojiUDF.java
+++ b/nlp/src/main/java/hivemall/nlp/tokenizer/KuromojiUDF.java
@@ -69,7 +69,10 @@ public final class KuromojiUDF extends GenericUDF {
private static final long MAX_INPUT_STREAM_SIZE = 32L * 1024L * 1024L; // ~32MB
private Mode _mode;
- private CharArraySet _stopWords;
+
+ // lazy instantiation to avoid org.apache.hive.com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
+ private transient CharArraySet _stopWords;
+
private Set<String> _stopTags;
private UserDictionary _userDict;