You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Tarek Auel (JIRA)" <ji...@apache.org> on 2015/06/12 00:26:00 UTC
[jira] [Comment Edited] (SPARK-8301) Improve UTF8String
substring/startsWith/endsWith/contains performance
[ https://issues.apache.org/jira/browse/SPARK-8301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582609#comment-14582609 ]
Tarek Auel edited comment on SPARK-8301 at 6/11/15 10:25 PM:
-------------------------------------------------------------
Hi,
do you have concrete ideas how the implementation should be improved?
I would like to take this item. One suggestion for startWith could be:
val b = prefix.getBytes
if (b.length > bytes.length) {
return false
}
var i = 0
while (i < b.length && (bytes ( i ) == b ( i ))) {
i += 1
}
i == b.length
What do you think? Do you have already ideas in your mind how the performance should be improved?
was (Author: tarekauel):
Hi,
do you have concrete ideas how the implementation should be improved?
I would like to take this item. One suggestion for startWith could be:
val b = prefix.getBytes
if (b.length > bytes.length) {
return false
}
var i = 0
while (i < b.length && (bytes(i) == b(i))) {
i += 1
}
i == b.length
What do you think? Do you have already ideas in your mind how the performance should be improved?
> Improve UTF8String substring/startsWith/endsWith/contains performance
> ---------------------------------------------------------------------
>
> Key: SPARK-8301
> URL: https://issues.apache.org/jira/browse/SPARK-8301
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Reynold Xin
> Priority: Critical
>
> Many functions in UTF8String are unnecessarily expensive.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org