[jira] [Commented] (LUCENE-7745) Explore GPU acceleration

["Joshua Mack (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-7745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16794786#comment-16794786 ] 

Joshua Mack commented on LUCENE-7745:
-------------------------------------

Hello all,

There's a chance that I may end up being a lucky graduate student tasked with handling something similar to this a few months down the line with application to a largely textual Kafka+Logstash+Elastic pipeline

[~ichattopadhyaya], I've noticed that the Github links above/referenced in the Activate 2018 talk now lead to 404s – I understand that it's not intended to be taken as the defacto method by which functionality like this should be added to Lucene, but having a baseline to look into with some of our investigations would certainly be helpful and could save us (myself) some time. If you'd be willing to rehost somewhere or provide updated results, I would certainly appreciate it

Similarly, [~rinka], coming from a naive outsider, I think your plan sounds good, and if you have any preliminary investigations you've performed, I would appreciate being able to learn from your lessons thus far. From my perspective, I have experience with GPUs and Solr separately, but I've never dealt with them together (or with Lucene directly for that matter)

Finally, we're still in a very investigative phase and I'm certainly not the PI, but assuming this ends up being relevant to our project, I'm sure we would be happy to contribute back whatever we end up doing as a part of the project

> Explore GPU acceleration
> ------------------------
>
>                 Key: LUCENE-7745
>                 URL: https://issues.apache.org/jira/browse/LUCENE-7745
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Ishan Chattopadhyaya
>            Assignee: Ishan Chattopadhyaya
>            Priority: Major
>              Labels: gsoc2017, mentor
>         Attachments: TermDisjunctionQuery.java, gpu-benchmarks.png
>
>
> There are parts of Lucene that can potentially be speeded up if computations were to be offloaded from CPU to the GPU(s). With commodity GPUs having as high as 12GB of high bandwidth RAM, we might be able to leverage GPUs to speed parts of Lucene (indexing, search).
> First that comes to mind is spatial filtering, which is traditionally known to be a good candidate for GPU based speedup (esp. when complex polygons are involved). In the past, Mike McCandless has mentioned that "both initial indexing and merging are CPU/IO intensive, but they are very amenable to soaking up the hardware's concurrency."
> I'm opening this issue as an exploratory task, suitable for a GSoC project. I volunteer to mentor any GSoC student willing to work on this this summer.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org