You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2017/04/10 19:47:41 UTC

[jira] [Commented] (ASTERIXDB-1877) Full-text search throw exception

    [ https://issues.apache.org/jira/browse/ASTERIXDB-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963409#comment-15963409 ] 

ASF subversion and git services commented on ASTERIXDB-1877:
------------------------------------------------------------

Commit ab01c87e5b700f1b90c67b6a58df4c225a6c88bc in asterixdb's branch refs/heads/master from [~wangsaeu]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=ab01c87 ]

ASTERIXDB-1877: Tokenizer in FullText Search fix

 - Sets the starting offset of the string correctly before
   it is used in the tokenizer of FullTextContainsEvaluator.

Change-Id: Ic949ccc0cc3e66c489f4c8435672a74c70b7c0e7
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1668
Sonar-Qube: Jenkins <je...@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <je...@fulliautomatix.ics.uci.edu>
Reviewed-by: Jianfeng Jia <ji...@gmail.com>
Tested-by: Jenkins <je...@fulliautomatix.ics.uci.edu>
BAD: Jenkins <je...@fulliautomatix.ics.uci.edu>


> Full-text search throw exception
> --------------------------------
>
>                 Key: ASTERIXDB-1877
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1877
>             Project: Apache AsterixDB
>          Issue Type: Bug
>         Environment: master: {"git.tags":"","git.build.version":"0.9.1-SNAPSHOT","git.closest.tag.commit.count":"175","git.commit.id.abbrev":"39d09a0" ... }
>            Reporter: Jianfeng Jia
>            Assignee: Taewoo Kim
>
> Use the latest AsterixDB, run the cloudberry ingestion [script|https://github.com/JavierJia/cloudberry/blob/full-text/script/ingestTwitterToLocalCluster.sh]. 
> Only the following query works 
> {code}
> for $t in dataset twitter.ds_tweet
> where ftcontains($t.'text', ['zika','virus'], {'mode':'all'})
> let $taggr := $t
> group by $g0 := get-interval-start-datetime(interval-bin($t.'create_at', datetime('1990-01-01T00:00:00.000Z'),  day-time-duration("PT1H") )) with $taggr
> return {
>    'hour' : $g0,'count' : count($taggr)
> }
> {code}
> These queries will throw exceptions
> {code}
> for $t in dataset twitter.ds_tweet
> where ftcontains($t.'text', ['zika'], {'mode':'all'})
> let $taggr := $t
> group by $g0 := get-interval-start-datetime(interval-bin($t.'create_at', datetime('1990-01-01T00:00:00.000Z'),  day-time-duration("PT1H") )) with $taggr
> return {
>    'hour' : $g0,'count' : count($taggr)
> }
> {code}
> {code}
> for $t in dataset twitter.ds_tweet
> where ftcontains($t.'text', 'zika')
> let $taggr := $t
> group by $g0 := get-interval-start-datetime(interval-bin($t.'create_at', datetime('1990-01-01T00:00:00.000Z'),  day-time-duration("PT1H") )) with $taggr
> return {
>    'hour' : $g0,'count' : count($taggr)
> }
> {code}
> {code}
> for $t in dataset twitter.ds_tweet
> where ftcontains($t.'text', ['happy','day'])
> let $taggr := $t
> group by $g0 := get-interval-start-datetime(interval-bin($t.'create_at', datetime('1990-01-01T00:00:00.000Z'),  day-time-duration("PT1H") )) with $taggr
> return {
>    'hour' : $g0,'count' : count($taggr)
> }
> {code}
> Exception :
> {code}
> Caused by: java.lang.IllegalArgumentException
> 	at org.apache.hyracks.util.string.UTF8StringUtil.charAt(UTF8StringUtil.java:60)
> 	at org.apache.hyracks.storage.am.lsm.invertedindex.tokenizers.DelimitedUTF8StringBinaryTokenizer.hasNext(DelimitedUTF8StringBinaryTokenizer.java:48)
> 	at org.apache.asterix.runtime.evaluators.common.FullTextContainsEvaluator$_EvaluatorGen.readLeftAndConductSearch(FullTextContainsEvaluator.java:358)
> 	at org.apache.asterix.runtime.evaluators.common.FullTextContainsEvaluator$_EvaluatorGen.fullTextContainsWithArg(FullTextContainsEvaluator.java:200)
> 	at org.apache.asterix.runtime.evaluators.common.FullTextContainsEvaluator$_EvaluatorGen.evaluate(FullTextContainsEvaluator.java:158)
> 	at org.apache.hyracks.algebricks.runtime.operators.std.StreamSelectRuntimeFactory$1.nextFrame(StreamSelectRuntimeFactory.java:133)
> 	at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.write(AbstractFrameAppender.java:92)
> 	at org.apache.hyracks.dataflow.common.comm.util.FrameUtils.appendProjectionToWriter(FrameUtils.java:235)
> 	at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendProjectionToFrame(AbstractOneInputOneOutputOneFramePushRuntime.java:98)
> 	at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendProjectionToFrame(AbstractOneInputOneOutputOneFramePushRuntime.java:93)
> 	at org.apache.hyracks.algebricks.runtime.operators.std.StreamProjectRuntimeFactory$1.nextFrame(StreamProjectRuntimeFactory.java:74)
> 	at org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.nextFrame(AlgebricksMetaOperatorDescriptor.java:134)
> 	at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.write(AbstractFrameAppender.java:92)
> 	at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.close(IndexSearchOperatorNodePushable.java:204)
> 	... 10 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)