Kibble and Key Phrase Extraction

[Daniel Gruno <hu...@apache.org>]

Hi folks,
before Christmas, I had some very informal talks with some other Kibble
folks in various places about Key Phrase Extraction (KPE) and how we
might be able to use that in Kibble.

KPE is the process of taking a longer text, for instance an email, and
extracting small sentences or words that are at the center of the text -
as an example, taking chapter one from a Christmas Carol would (among
other things) point to Scrooge, Marley and 'Dead as a Door-nail' as key
elements. This isn't to say that you can always grasp exactly what all
10,000 words in that chapter was about, but you get some elements that
play a key role in it. As an aside, KPE is supported by most text
analysis services out there (I've tried it with Watson, Azure and
picoAPI thus far)

My idea with KPE is so generate a list of keywords and sentences to
accompany each email (and possibly issues/tickets?) so that we can
generate a map of "hot topics" both in projects and across them, looking
for commonalities and trends. This _could_ show that most projects have
similar work-flows/activities in common, or it could show the exact
opposite - I don't know yet, but I sure would love to find out! It could
perhaps also tell us (ngram-style) something about how technology
progresses over time by following the occurrences of certain
words/sentences in projects as a time-series.

I would _love_ to get some input on this (especially as far as 'what can
we use this for, if anything' is concerned), and I will probably start
some basic KPE extraction tests in the coming days. As mentioned
elsewhere, the Kibble demo instance has virtually unlimited text
analysis credits at the moment, including KPE, so we might as well make
use of that :)

WDYT?

With regards,
Daniel.