You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Nic Crane <th...@gmail.com> on 2025/05/07 16:52:20 UTC
Re: Kapa.ai bot now live on the dev docs
A brief update on this from me - I've now added the Python, R, C++, and
Java cookbooks as a source for the kapa.ai question bot, so code examples
from those should be on there now.
At a later point, I'll take a look at potentially adding GitHub PRs and
issues, but I'd like to test what we have so far before adding more in
there.
On Mon, 28 Apr 2025 at 04:56, Nic Crane <th...@gmail.com> wrote:
> We've current had 64 questions, though many of these are from folks
> testing it out. I've personally found it useful for some of my dev work.
>
> It's hard to tell yet if it's useful for users but we'll have more data
> once it's live on the latest docs rather than the dev docs, so my plan is
> to keeping checking in on the questions and answers between the upcoming
> release and the next one so we can decide whether it's a useful tool for us
> to have.
>
> If anyone would like a login for kapa.ai so you can see the conversations
> and answers (and add in any feedback you'd like to add yourself for
> questions), then please email me and I'll add you as a user.
>
>
> On Wed, 23 Apr 2025 at 06:11, Nic Crane <th...@gmail.com> wrote:
>
>> > nit: any chance the popup can be made to respect the light/dark mode
>> toggle
>> in the docs? It seems to always use a light color scheme.
>>
>> From the looks of it, the colour can be configured via the
>> 'data-project-color' parameter[1] so I imagine this could be done via
>> updating the widget script with some custom JS on our side.
>>
>> [1] https://docs.kapa.ai/integrations/website-widget/configuration
>>
>> On Mon, 31 Mar 2025 at 17:15, Brian Hulette <bh...@apache.org> wrote:
>>
>>> Very cool, I tried a few queries and it provided good answers.
>>>
>>> nit: any chance the popup can be made to respect the light/dark mode
>>> toggle
>>> in the docs? It seems to always use a light color scheme.
>>>
>>> On Fri, Mar 28, 2025 at 5:05 PM Nic Crane <th...@gmail.com> wrote:
>>>
>>> > Cookbooks aren't part of the sources but we can certainly add them
>>> later.
>>> >
>>> > The kapa folk recommend trying with an initial sunset of sources and
>>> adding
>>> > more in gradually, so I'll check back in on this next week and think
>>> about
>>> > what we might want to add next.
>>> >
>>> > On Fri, 28 Mar 2025, 21:45 Aldrin, <oc...@pm.me.invalid> wrote:
>>> >
>>> > > Ooh, yeah it's looking fairly effective. I asked some questions and I
>>> > like
>>> > > that the answers address differences in language implementations
>>> (e.g.
>>> > > python bindings vs cpp) and that there are relatively good code
>>> > suggestions.
>>> > >
>>> > > I assume this means the tests and benchmarks are helping the RAG a
>>> lot,
>>> > so
>>> > > general well done to all contributors. Out of curiosity, are the
>>> > cookbooks
>>> > > part of the learning/retrieval sources?
>>> > >
>>> > > Sent from Proton Mail <https://proton.me/mail/home> for iOS
>>> > >
>>> > >
>>> > > On Fri, Mar 28, 2025 at 13:58, Neal Richardson <
>>> > > neal.p.richardson@gmail.com
>>> > > <On+Fri,+Mar+28,+2025+at+13:58,+Neal+Richardson+%3C%3Ca+href=>>
>>> wrote:
>>> > >
>>> > > Awesome, thanks for doing this! I tried out a few queries and was
>>> > > impressed
>>> > > with what I got back.
>>> > >
>>> > > Neal
>>> > >
>>> > > On Fri, Mar 28, 2025 at 2:01 PM Nic Crane <th...@gmail.com>
>>> wrote:
>>> > >
>>> > > > Hi folks,
>>> > > >
>>> > > > The kapa.ai bot is now live on the dev docs! We're using it to
>>> test
>>> > > > whether
>>> > > > we can enhance user experience of our documentation via retrieval
>>> > > augmented
>>> > > > generation which generates answers to user questions by combining
>>> > > > similarity searches of the docs and LLM output.
>>> > > >
>>> > > > You can use it by clicking the "Ask AI" button in the bottom right
>>> > > corner
>>> > > > of https://arrow.apache.org/docs/dev/index.html
>>> > > >
>>> > > > Currently we've just ingested the docs, but as we experiment more,
>>> we
>>> > > can
>>> > > > add other sources like Zulip and the mailing lists if we think
>>> these
>>> > > will
>>> > > > provide useful additional source material.
>>> > > >
>>> > > > Please give it a go and use the "good answer" and "bad answer"
>>> buttons
>>> > > so
>>> > > > we can assess how useful this is likely to be for our end users;
>>> bonus
>>> > > > points for adding a better answer when you're not happy with the
>>> one
>>> > > given.
>>> > > >
>>> > > > I'll be checking in on it every so often and will report back here
>>> with
>>> > > the
>>> > > > results.
>>> > > >
>>> > > > Thanks,
>>> > > >
>>> > > > Nic
>>> > > >
>>> > >
>>> > >
>>> >
>>>
>>
Re: Kapa.ai bot now live on the dev docs
Posted by Jacob Wujciak-Jens <ja...@voltrondata.com.INVALID>.
> 852 questions over the past month
Wow, that's much more than I expected!
Very cool, thanks again for your work on this Nic!
On Thu, Jun 12, 2025 at 3:29 PM Nic Crane <th...@gmail.com> wrote:
> An update for folks on this - I've extracted a few statistics below from
> the main dashboard. We said it would be worth testing this out for 1
> release to see what we thought. I haven't had time to do a deep dive, but
> overall, I do think it is providing value for users, so we should keep it
> there. Happy to discuss though.
>
> 852 questions over the past month, averaging around 191 per week.
>
> People aren't using the feedback mechanisms much, but there were 19 thumbs
> up and 3 thumbs down.
>
> In 90% of cases, the bot was able to give a confident response.
>
> Languages:
> * English - 88%
> * Chinese - 8%
> * Russian - 3%
> * Japanese - 1%
>
> Topics:
> * Unsupported features - 14%
> * Troubleshooting - 9%
> * Comparison with similar technologies - 2%
> * Off topic questions - 2%
>
> If there are any other PMC members or committers who'd like access to the
> dashboard, please do let me know.
>
> On Mon, 12 May 2025 at 15:02, Matt Topol <zo...@gmail.com> wrote:
>
> > Thanks!
> >
> > On Mon, May 12, 2025, 9:47 AM Nic Crane <th...@gmail.com> wrote:
> >
> > > Absolutely Matt - I'm adding them now and I'll add you as an admin so
> > > you'll be able to review Go questions in the UI and make any changes
> > you'd
> > > like to what we include.
> > >
> > > On Wed, 7 May 2025 at 12:59, Matt Topol <zo...@gmail.com>
> wrote:
> > >
> > > > Would we be able to feed it the arrow-go documentation too?
> > > >
> > > > On Wed, May 7, 2025, 12:52 PM Nic Crane <th...@gmail.com> wrote:
> > > >
> > > > > A brief update on this from me - I've now added the Python, R, C++,
> > and
> > > > > Java cookbooks as a source for the kapa.ai question bot, so code
> > > > examples
> > > > > from those should be on there now.
> > > > >
> > > > > At a later point, I'll take a look at potentially adding GitHub PRs
> > and
> > > > > issues, but I'd like to test what we have so far before adding more
> > in
> > > > > there.
> > > > >
> > > > >
> > > > > On Mon, 28 Apr 2025 at 04:56, Nic Crane <th...@gmail.com>
> wrote:
> > > > >
> > > > > > We've current had 64 questions, though many of these are from
> folks
> > > > > > testing it out. I've personally found it useful for some of my
> dev
> > > > work.
> > > > > >
> > > > > > It's hard to tell yet if it's useful for users but we'll have
> more
> > > data
> > > > > > once it's live on the latest docs rather than the dev docs, so my
> > > plan
> > > > is
> > > > > > to keeping checking in on the questions and answers between the
> > > > upcoming
> > > > > > release and the next one so we can decide whether it's a useful
> > tool
> > > > for
> > > > > us
> > > > > > to have.
> > > > > >
> > > > > > If anyone would like a login for kapa.ai so you can see the
> > > > > conversations
> > > > > > and answers (and add in any feedback you'd like to add yourself
> for
> > > > > > questions), then please email me and I'll add you as a user.
> > > > > >
> > > > > >
> > > > > > On Wed, 23 Apr 2025 at 06:11, Nic Crane <th...@gmail.com>
> > wrote:
> > > > > >
> > > > > >> > nit: any chance the popup can be made to respect the
> light/dark
> > > mode
> > > > > >> toggle
> > > > > >> in the docs? It seems to always use a light color scheme.
> > > > > >>
> > > > > >> From the looks of it, the colour can be configured via the
> > > > > >> 'data-project-color' parameter[1] so I imagine this could be
> done
> > > via
> > > > > >> updating the widget script with some custom JS on our side.
> > > > > >>
> > > > > >> [1]
> > https://docs.kapa.ai/integrations/website-widget/configuration
> > > > > >>
> > > > > >> On Mon, 31 Mar 2025 at 17:15, Brian Hulette <
> bhulette@apache.org>
> > > > > wrote:
> > > > > >>
> > > > > >>> Very cool, I tried a few queries and it provided good answers.
> > > > > >>>
> > > > > >>> nit: any chance the popup can be made to respect the light/dark
> > > mode
> > > > > >>> toggle
> > > > > >>> in the docs? It seems to always use a light color scheme.
> > > > > >>>
> > > > > >>> On Fri, Mar 28, 2025 at 5:05 PM Nic Crane <thisisnic@gmail.com
> >
> > > > wrote:
> > > > > >>>
> > > > > >>> > Cookbooks aren't part of the sources but we can certainly add
> > > them
> > > > > >>> later.
> > > > > >>> >
> > > > > >>> > The kapa folk recommend trying with an initial sunset of
> > sources
> > > > and
> > > > > >>> adding
> > > > > >>> > more in gradually, so I'll check back in on this next week
> and
> > > > think
> > > > > >>> about
> > > > > >>> > what we might want to add next.
> > > > > >>> >
> > > > > >>> > On Fri, 28 Mar 2025, 21:45 Aldrin,
> <octalene.dev@pm.me.invalid
> > >
> > > > > wrote:
> > > > > >>> >
> > > > > >>> > > Ooh, yeah it's looking fairly effective. I asked some
> > questions
> > > > > and I
> > > > > >>> > like
> > > > > >>> > > that the answers address differences in language
> > > implementations
> > > > > >>> (e.g.
> > > > > >>> > > python bindings vs cpp) and that there are relatively good
> > code
> > > > > >>> > suggestions.
> > > > > >>> > >
> > > > > >>> > > I assume this means the tests and benchmarks are helping
> the
> > > RAG
> > > > a
> > > > > >>> lot,
> > > > > >>> > so
> > > > > >>> > > general well done to all contributors. Out of curiosity,
> are
> > > the
> > > > > >>> > cookbooks
> > > > > >>> > > part of the learning/retrieval sources?
> > > > > >>> > >
> > > > > >>> > > Sent from Proton Mail <https://proton.me/mail/home> for
> iOS
> > > > > >>> > >
> > > > > >>> > >
> > > > > >>> > > On Fri, Mar 28, 2025 at 13:58, Neal Richardson <
> > > > > >>> > > neal.p.richardson@gmail.com
> > > > > >>> > >
> > <On+Fri,+Mar+28,+2025+at+13:58,+Neal+Richardson+%3C%3Ca+href=>>
> > > > > >>> wrote:
> > > > > >>> > >
> > > > > >>> > > Awesome, thanks for doing this! I tried out a few queries
> and
> > > was
> > > > > >>> > > impressed
> > > > > >>> > > with what I got back.
> > > > > >>> > >
> > > > > >>> > > Neal
> > > > > >>> > >
> > > > > >>> > > On Fri, Mar 28, 2025 at 2:01 PM Nic Crane <
> > thisisnic@gmail.com
> > > >
> > > > > >>> wrote:
> > > > > >>> > >
> > > > > >>> > > > Hi folks,
> > > > > >>> > > >
> > > > > >>> > > > The kapa.ai bot is now live on the dev docs! We're using
> > it
> > > to
> > > > > >>> test
> > > > > >>> > > > whether
> > > > > >>> > > > we can enhance user experience of our documentation via
> > > > retrieval
> > > > > >>> > > augmented
> > > > > >>> > > > generation which generates answers to user questions by
> > > > combining
> > > > > >>> > > > similarity searches of the docs and LLM output.
> > > > > >>> > > >
> > > > > >>> > > > You can use it by clicking the "Ask AI" button in the
> > bottom
> > > > > right
> > > > > >>> > > corner
> > > > > >>> > > > of https://arrow.apache.org/docs/dev/index.html
> > > > > >>> > > >
> > > > > >>> > > > Currently we've just ingested the docs, but as we
> > experiment
> > > > > more,
> > > > > >>> we
> > > > > >>> > > can
> > > > > >>> > > > add other sources like Zulip and the mailing lists if we
> > > think
> > > > > >>> these
> > > > > >>> > > will
> > > > > >>> > > > provide useful additional source material.
> > > > > >>> > > >
> > > > > >>> > > > Please give it a go and use the "good answer" and "bad
> > > answer"
> > > > > >>> buttons
> > > > > >>> > > so
> > > > > >>> > > > we can assess how useful this is likely to be for our end
> > > > users;
> > > > > >>> bonus
> > > > > >>> > > > points for adding a better answer when you're not happy
> > with
> > > > the
> > > > > >>> one
> > > > > >>> > > given.
> > > > > >>> > > >
> > > > > >>> > > > I'll be checking in on it every so often and will report
> > back
> > > > > here
> > > > > >>> with
> > > > > >>> > > the
> > > > > >>> > > > results.
> > > > > >>> > > >
> > > > > >>> > > > Thanks,
> > > > > >>> > > >
> > > > > >>> > > > Nic
> > > > > >>> > > >
> > > > > >>> > >
> > > > > >>> > >
> > > > > >>> >
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
>
Re: Kapa.ai bot now live on the dev docs
Posted by Nic Crane <th...@gmail.com>.
An update for folks on this - I've extracted a few statistics below from
the main dashboard. We said it would be worth testing this out for 1
release to see what we thought. I haven't had time to do a deep dive, but
overall, I do think it is providing value for users, so we should keep it
there. Happy to discuss though.
852 questions over the past month, averaging around 191 per week.
People aren't using the feedback mechanisms much, but there were 19 thumbs
up and 3 thumbs down.
In 90% of cases, the bot was able to give a confident response.
Languages:
* English - 88%
* Chinese - 8%
* Russian - 3%
* Japanese - 1%
Topics:
* Unsupported features - 14%
* Troubleshooting - 9%
* Comparison with similar technologies - 2%
* Off topic questions - 2%
If there are any other PMC members or committers who'd like access to the
dashboard, please do let me know.
On Mon, 12 May 2025 at 15:02, Matt Topol <zo...@gmail.com> wrote:
> Thanks!
>
> On Mon, May 12, 2025, 9:47 AM Nic Crane <th...@gmail.com> wrote:
>
> > Absolutely Matt - I'm adding them now and I'll add you as an admin so
> > you'll be able to review Go questions in the UI and make any changes
> you'd
> > like to what we include.
> >
> > On Wed, 7 May 2025 at 12:59, Matt Topol <zo...@gmail.com> wrote:
> >
> > > Would we be able to feed it the arrow-go documentation too?
> > >
> > > On Wed, May 7, 2025, 12:52 PM Nic Crane <th...@gmail.com> wrote:
> > >
> > > > A brief update on this from me - I've now added the Python, R, C++,
> and
> > > > Java cookbooks as a source for the kapa.ai question bot, so code
> > > examples
> > > > from those should be on there now.
> > > >
> > > > At a later point, I'll take a look at potentially adding GitHub PRs
> and
> > > > issues, but I'd like to test what we have so far before adding more
> in
> > > > there.
> > > >
> > > >
> > > > On Mon, 28 Apr 2025 at 04:56, Nic Crane <th...@gmail.com> wrote:
> > > >
> > > > > We've current had 64 questions, though many of these are from folks
> > > > > testing it out. I've personally found it useful for some of my dev
> > > work.
> > > > >
> > > > > It's hard to tell yet if it's useful for users but we'll have more
> > data
> > > > > once it's live on the latest docs rather than the dev docs, so my
> > plan
> > > is
> > > > > to keeping checking in on the questions and answers between the
> > > upcoming
> > > > > release and the next one so we can decide whether it's a useful
> tool
> > > for
> > > > us
> > > > > to have.
> > > > >
> > > > > If anyone would like a login for kapa.ai so you can see the
> > > > conversations
> > > > > and answers (and add in any feedback you'd like to add yourself for
> > > > > questions), then please email me and I'll add you as a user.
> > > > >
> > > > >
> > > > > On Wed, 23 Apr 2025 at 06:11, Nic Crane <th...@gmail.com>
> wrote:
> > > > >
> > > > >> > nit: any chance the popup can be made to respect the light/dark
> > mode
> > > > >> toggle
> > > > >> in the docs? It seems to always use a light color scheme.
> > > > >>
> > > > >> From the looks of it, the colour can be configured via the
> > > > >> 'data-project-color' parameter[1] so I imagine this could be done
> > via
> > > > >> updating the widget script with some custom JS on our side.
> > > > >>
> > > > >> [1]
> https://docs.kapa.ai/integrations/website-widget/configuration
> > > > >>
> > > > >> On Mon, 31 Mar 2025 at 17:15, Brian Hulette <bh...@apache.org>
> > > > wrote:
> > > > >>
> > > > >>> Very cool, I tried a few queries and it provided good answers.
> > > > >>>
> > > > >>> nit: any chance the popup can be made to respect the light/dark
> > mode
> > > > >>> toggle
> > > > >>> in the docs? It seems to always use a light color scheme.
> > > > >>>
> > > > >>> On Fri, Mar 28, 2025 at 5:05 PM Nic Crane <th...@gmail.com>
> > > wrote:
> > > > >>>
> > > > >>> > Cookbooks aren't part of the sources but we can certainly add
> > them
> > > > >>> later.
> > > > >>> >
> > > > >>> > The kapa folk recommend trying with an initial sunset of
> sources
> > > and
> > > > >>> adding
> > > > >>> > more in gradually, so I'll check back in on this next week and
> > > think
> > > > >>> about
> > > > >>> > what we might want to add next.
> > > > >>> >
> > > > >>> > On Fri, 28 Mar 2025, 21:45 Aldrin, <octalene.dev@pm.me.invalid
> >
> > > > wrote:
> > > > >>> >
> > > > >>> > > Ooh, yeah it's looking fairly effective. I asked some
> questions
> > > > and I
> > > > >>> > like
> > > > >>> > > that the answers address differences in language
> > implementations
> > > > >>> (e.g.
> > > > >>> > > python bindings vs cpp) and that there are relatively good
> code
> > > > >>> > suggestions.
> > > > >>> > >
> > > > >>> > > I assume this means the tests and benchmarks are helping the
> > RAG
> > > a
> > > > >>> lot,
> > > > >>> > so
> > > > >>> > > general well done to all contributors. Out of curiosity, are
> > the
> > > > >>> > cookbooks
> > > > >>> > > part of the learning/retrieval sources?
> > > > >>> > >
> > > > >>> > > Sent from Proton Mail <https://proton.me/mail/home> for iOS
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > On Fri, Mar 28, 2025 at 13:58, Neal Richardson <
> > > > >>> > > neal.p.richardson@gmail.com
> > > > >>> > >
> <On+Fri,+Mar+28,+2025+at+13:58,+Neal+Richardson+%3C%3Ca+href=>>
> > > > >>> wrote:
> > > > >>> > >
> > > > >>> > > Awesome, thanks for doing this! I tried out a few queries and
> > was
> > > > >>> > > impressed
> > > > >>> > > with what I got back.
> > > > >>> > >
> > > > >>> > > Neal
> > > > >>> > >
> > > > >>> > > On Fri, Mar 28, 2025 at 2:01 PM Nic Crane <
> thisisnic@gmail.com
> > >
> > > > >>> wrote:
> > > > >>> > >
> > > > >>> > > > Hi folks,
> > > > >>> > > >
> > > > >>> > > > The kapa.ai bot is now live on the dev docs! We're using
> it
> > to
> > > > >>> test
> > > > >>> > > > whether
> > > > >>> > > > we can enhance user experience of our documentation via
> > > retrieval
> > > > >>> > > augmented
> > > > >>> > > > generation which generates answers to user questions by
> > > combining
> > > > >>> > > > similarity searches of the docs and LLM output.
> > > > >>> > > >
> > > > >>> > > > You can use it by clicking the "Ask AI" button in the
> bottom
> > > > right
> > > > >>> > > corner
> > > > >>> > > > of https://arrow.apache.org/docs/dev/index.html
> > > > >>> > > >
> > > > >>> > > > Currently we've just ingested the docs, but as we
> experiment
> > > > more,
> > > > >>> we
> > > > >>> > > can
> > > > >>> > > > add other sources like Zulip and the mailing lists if we
> > think
> > > > >>> these
> > > > >>> > > will
> > > > >>> > > > provide useful additional source material.
> > > > >>> > > >
> > > > >>> > > > Please give it a go and use the "good answer" and "bad
> > answer"
> > > > >>> buttons
> > > > >>> > > so
> > > > >>> > > > we can assess how useful this is likely to be for our end
> > > users;
> > > > >>> bonus
> > > > >>> > > > points for adding a better answer when you're not happy
> with
> > > the
> > > > >>> one
> > > > >>> > > given.
> > > > >>> > > >
> > > > >>> > > > I'll be checking in on it every so often and will report
> back
> > > > here
> > > > >>> with
> > > > >>> > > the
> > > > >>> > > > results.
> > > > >>> > > >
> > > > >>> > > > Thanks,
> > > > >>> > > >
> > > > >>> > > > Nic
> > > > >>> > > >
> > > > >>> > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > > >>
> > > >
> > >
> >
>
Re: Kapa.ai bot now live on the dev docs
Posted by Matt Topol <zo...@gmail.com>.
Thanks!
On Mon, May 12, 2025, 9:47 AM Nic Crane <th...@gmail.com> wrote:
> Absolutely Matt - I'm adding them now and I'll add you as an admin so
> you'll be able to review Go questions in the UI and make any changes you'd
> like to what we include.
>
> On Wed, 7 May 2025 at 12:59, Matt Topol <zo...@gmail.com> wrote:
>
> > Would we be able to feed it the arrow-go documentation too?
> >
> > On Wed, May 7, 2025, 12:52 PM Nic Crane <th...@gmail.com> wrote:
> >
> > > A brief update on this from me - I've now added the Python, R, C++, and
> > > Java cookbooks as a source for the kapa.ai question bot, so code
> > examples
> > > from those should be on there now.
> > >
> > > At a later point, I'll take a look at potentially adding GitHub PRs and
> > > issues, but I'd like to test what we have so far before adding more in
> > > there.
> > >
> > >
> > > On Mon, 28 Apr 2025 at 04:56, Nic Crane <th...@gmail.com> wrote:
> > >
> > > > We've current had 64 questions, though many of these are from folks
> > > > testing it out. I've personally found it useful for some of my dev
> > work.
> > > >
> > > > It's hard to tell yet if it's useful for users but we'll have more
> data
> > > > once it's live on the latest docs rather than the dev docs, so my
> plan
> > is
> > > > to keeping checking in on the questions and answers between the
> > upcoming
> > > > release and the next one so we can decide whether it's a useful tool
> > for
> > > us
> > > > to have.
> > > >
> > > > If anyone would like a login for kapa.ai so you can see the
> > > conversations
> > > > and answers (and add in any feedback you'd like to add yourself for
> > > > questions), then please email me and I'll add you as a user.
> > > >
> > > >
> > > > On Wed, 23 Apr 2025 at 06:11, Nic Crane <th...@gmail.com> wrote:
> > > >
> > > >> > nit: any chance the popup can be made to respect the light/dark
> mode
> > > >> toggle
> > > >> in the docs? It seems to always use a light color scheme.
> > > >>
> > > >> From the looks of it, the colour can be configured via the
> > > >> 'data-project-color' parameter[1] so I imagine this could be done
> via
> > > >> updating the widget script with some custom JS on our side.
> > > >>
> > > >> [1] https://docs.kapa.ai/integrations/website-widget/configuration
> > > >>
> > > >> On Mon, 31 Mar 2025 at 17:15, Brian Hulette <bh...@apache.org>
> > > wrote:
> > > >>
> > > >>> Very cool, I tried a few queries and it provided good answers.
> > > >>>
> > > >>> nit: any chance the popup can be made to respect the light/dark
> mode
> > > >>> toggle
> > > >>> in the docs? It seems to always use a light color scheme.
> > > >>>
> > > >>> On Fri, Mar 28, 2025 at 5:05 PM Nic Crane <th...@gmail.com>
> > wrote:
> > > >>>
> > > >>> > Cookbooks aren't part of the sources but we can certainly add
> them
> > > >>> later.
> > > >>> >
> > > >>> > The kapa folk recommend trying with an initial sunset of sources
> > and
> > > >>> adding
> > > >>> > more in gradually, so I'll check back in on this next week and
> > think
> > > >>> about
> > > >>> > what we might want to add next.
> > > >>> >
> > > >>> > On Fri, 28 Mar 2025, 21:45 Aldrin, <oc...@pm.me.invalid>
> > > wrote:
> > > >>> >
> > > >>> > > Ooh, yeah it's looking fairly effective. I asked some questions
> > > and I
> > > >>> > like
> > > >>> > > that the answers address differences in language
> implementations
> > > >>> (e.g.
> > > >>> > > python bindings vs cpp) and that there are relatively good code
> > > >>> > suggestions.
> > > >>> > >
> > > >>> > > I assume this means the tests and benchmarks are helping the
> RAG
> > a
> > > >>> lot,
> > > >>> > so
> > > >>> > > general well done to all contributors. Out of curiosity, are
> the
> > > >>> > cookbooks
> > > >>> > > part of the learning/retrieval sources?
> > > >>> > >
> > > >>> > > Sent from Proton Mail <https://proton.me/mail/home> for iOS
> > > >>> > >
> > > >>> > >
> > > >>> > > On Fri, Mar 28, 2025 at 13:58, Neal Richardson <
> > > >>> > > neal.p.richardson@gmail.com
> > > >>> > > <On+Fri,+Mar+28,+2025+at+13:58,+Neal+Richardson+%3C%3Ca+href=>>
> > > >>> wrote:
> > > >>> > >
> > > >>> > > Awesome, thanks for doing this! I tried out a few queries and
> was
> > > >>> > > impressed
> > > >>> > > with what I got back.
> > > >>> > >
> > > >>> > > Neal
> > > >>> > >
> > > >>> > > On Fri, Mar 28, 2025 at 2:01 PM Nic Crane <thisisnic@gmail.com
> >
> > > >>> wrote:
> > > >>> > >
> > > >>> > > > Hi folks,
> > > >>> > > >
> > > >>> > > > The kapa.ai bot is now live on the dev docs! We're using it
> to
> > > >>> test
> > > >>> > > > whether
> > > >>> > > > we can enhance user experience of our documentation via
> > retrieval
> > > >>> > > augmented
> > > >>> > > > generation which generates answers to user questions by
> > combining
> > > >>> > > > similarity searches of the docs and LLM output.
> > > >>> > > >
> > > >>> > > > You can use it by clicking the "Ask AI" button in the bottom
> > > right
> > > >>> > > corner
> > > >>> > > > of https://arrow.apache.org/docs/dev/index.html
> > > >>> > > >
> > > >>> > > > Currently we've just ingested the docs, but as we experiment
> > > more,
> > > >>> we
> > > >>> > > can
> > > >>> > > > add other sources like Zulip and the mailing lists if we
> think
> > > >>> these
> > > >>> > > will
> > > >>> > > > provide useful additional source material.
> > > >>> > > >
> > > >>> > > > Please give it a go and use the "good answer" and "bad
> answer"
> > > >>> buttons
> > > >>> > > so
> > > >>> > > > we can assess how useful this is likely to be for our end
> > users;
> > > >>> bonus
> > > >>> > > > points for adding a better answer when you're not happy with
> > the
> > > >>> one
> > > >>> > > given.
> > > >>> > > >
> > > >>> > > > I'll be checking in on it every so often and will report back
> > > here
> > > >>> with
> > > >>> > > the
> > > >>> > > > results.
> > > >>> > > >
> > > >>> > > > Thanks,
> > > >>> > > >
> > > >>> > > > Nic
> > > >>> > > >
> > > >>> > >
> > > >>> > >
> > > >>> >
> > > >>>
> > > >>
> > >
> >
>
Re: Kapa.ai bot now live on the dev docs
Posted by Nic Crane <th...@gmail.com>.
Absolutely Matt - I'm adding them now and I'll add you as an admin so
you'll be able to review Go questions in the UI and make any changes you'd
like to what we include.
On Wed, 7 May 2025 at 12:59, Matt Topol <zo...@gmail.com> wrote:
> Would we be able to feed it the arrow-go documentation too?
>
> On Wed, May 7, 2025, 12:52 PM Nic Crane <th...@gmail.com> wrote:
>
> > A brief update on this from me - I've now added the Python, R, C++, and
> > Java cookbooks as a source for the kapa.ai question bot, so code
> examples
> > from those should be on there now.
> >
> > At a later point, I'll take a look at potentially adding GitHub PRs and
> > issues, but I'd like to test what we have so far before adding more in
> > there.
> >
> >
> > On Mon, 28 Apr 2025 at 04:56, Nic Crane <th...@gmail.com> wrote:
> >
> > > We've current had 64 questions, though many of these are from folks
> > > testing it out. I've personally found it useful for some of my dev
> work.
> > >
> > > It's hard to tell yet if it's useful for users but we'll have more data
> > > once it's live on the latest docs rather than the dev docs, so my plan
> is
> > > to keeping checking in on the questions and answers between the
> upcoming
> > > release and the next one so we can decide whether it's a useful tool
> for
> > us
> > > to have.
> > >
> > > If anyone would like a login for kapa.ai so you can see the
> > conversations
> > > and answers (and add in any feedback you'd like to add yourself for
> > > questions), then please email me and I'll add you as a user.
> > >
> > >
> > > On Wed, 23 Apr 2025 at 06:11, Nic Crane <th...@gmail.com> wrote:
> > >
> > >> > nit: any chance the popup can be made to respect the light/dark mode
> > >> toggle
> > >> in the docs? It seems to always use a light color scheme.
> > >>
> > >> From the looks of it, the colour can be configured via the
> > >> 'data-project-color' parameter[1] so I imagine this could be done via
> > >> updating the widget script with some custom JS on our side.
> > >>
> > >> [1] https://docs.kapa.ai/integrations/website-widget/configuration
> > >>
> > >> On Mon, 31 Mar 2025 at 17:15, Brian Hulette <bh...@apache.org>
> > wrote:
> > >>
> > >>> Very cool, I tried a few queries and it provided good answers.
> > >>>
> > >>> nit: any chance the popup can be made to respect the light/dark mode
> > >>> toggle
> > >>> in the docs? It seems to always use a light color scheme.
> > >>>
> > >>> On Fri, Mar 28, 2025 at 5:05 PM Nic Crane <th...@gmail.com>
> wrote:
> > >>>
> > >>> > Cookbooks aren't part of the sources but we can certainly add them
> > >>> later.
> > >>> >
> > >>> > The kapa folk recommend trying with an initial sunset of sources
> and
> > >>> adding
> > >>> > more in gradually, so I'll check back in on this next week and
> think
> > >>> about
> > >>> > what we might want to add next.
> > >>> >
> > >>> > On Fri, 28 Mar 2025, 21:45 Aldrin, <oc...@pm.me.invalid>
> > wrote:
> > >>> >
> > >>> > > Ooh, yeah it's looking fairly effective. I asked some questions
> > and I
> > >>> > like
> > >>> > > that the answers address differences in language implementations
> > >>> (e.g.
> > >>> > > python bindings vs cpp) and that there are relatively good code
> > >>> > suggestions.
> > >>> > >
> > >>> > > I assume this means the tests and benchmarks are helping the RAG
> a
> > >>> lot,
> > >>> > so
> > >>> > > general well done to all contributors. Out of curiosity, are the
> > >>> > cookbooks
> > >>> > > part of the learning/retrieval sources?
> > >>> > >
> > >>> > > Sent from Proton Mail <https://proton.me/mail/home> for iOS
> > >>> > >
> > >>> > >
> > >>> > > On Fri, Mar 28, 2025 at 13:58, Neal Richardson <
> > >>> > > neal.p.richardson@gmail.com
> > >>> > > <On+Fri,+Mar+28,+2025+at+13:58,+Neal+Richardson+%3C%3Ca+href=>>
> > >>> wrote:
> > >>> > >
> > >>> > > Awesome, thanks for doing this! I tried out a few queries and was
> > >>> > > impressed
> > >>> > > with what I got back.
> > >>> > >
> > >>> > > Neal
> > >>> > >
> > >>> > > On Fri, Mar 28, 2025 at 2:01 PM Nic Crane <th...@gmail.com>
> > >>> wrote:
> > >>> > >
> > >>> > > > Hi folks,
> > >>> > > >
> > >>> > > > The kapa.ai bot is now live on the dev docs! We're using it to
> > >>> test
> > >>> > > > whether
> > >>> > > > we can enhance user experience of our documentation via
> retrieval
> > >>> > > augmented
> > >>> > > > generation which generates answers to user questions by
> combining
> > >>> > > > similarity searches of the docs and LLM output.
> > >>> > > >
> > >>> > > > You can use it by clicking the "Ask AI" button in the bottom
> > right
> > >>> > > corner
> > >>> > > > of https://arrow.apache.org/docs/dev/index.html
> > >>> > > >
> > >>> > > > Currently we've just ingested the docs, but as we experiment
> > more,
> > >>> we
> > >>> > > can
> > >>> > > > add other sources like Zulip and the mailing lists if we think
> > >>> these
> > >>> > > will
> > >>> > > > provide useful additional source material.
> > >>> > > >
> > >>> > > > Please give it a go and use the "good answer" and "bad answer"
> > >>> buttons
> > >>> > > so
> > >>> > > > we can assess how useful this is likely to be for our end
> users;
> > >>> bonus
> > >>> > > > points for adding a better answer when you're not happy with
> the
> > >>> one
> > >>> > > given.
> > >>> > > >
> > >>> > > > I'll be checking in on it every so often and will report back
> > here
> > >>> with
> > >>> > > the
> > >>> > > > results.
> > >>> > > >
> > >>> > > > Thanks,
> > >>> > > >
> > >>> > > > Nic
> > >>> > > >
> > >>> > >
> > >>> > >
> > >>> >
> > >>>
> > >>
> >
>
Re: Kapa.ai bot now live on the dev docs
Posted by Matt Topol <zo...@gmail.com>.
Would we be able to feed it the arrow-go documentation too?
On Wed, May 7, 2025, 12:52 PM Nic Crane <th...@gmail.com> wrote:
> A brief update on this from me - I've now added the Python, R, C++, and
> Java cookbooks as a source for the kapa.ai question bot, so code examples
> from those should be on there now.
>
> At a later point, I'll take a look at potentially adding GitHub PRs and
> issues, but I'd like to test what we have so far before adding more in
> there.
>
>
> On Mon, 28 Apr 2025 at 04:56, Nic Crane <th...@gmail.com> wrote:
>
> > We've current had 64 questions, though many of these are from folks
> > testing it out. I've personally found it useful for some of my dev work.
> >
> > It's hard to tell yet if it's useful for users but we'll have more data
> > once it's live on the latest docs rather than the dev docs, so my plan is
> > to keeping checking in on the questions and answers between the upcoming
> > release and the next one so we can decide whether it's a useful tool for
> us
> > to have.
> >
> > If anyone would like a login for kapa.ai so you can see the
> conversations
> > and answers (and add in any feedback you'd like to add yourself for
> > questions), then please email me and I'll add you as a user.
> >
> >
> > On Wed, 23 Apr 2025 at 06:11, Nic Crane <th...@gmail.com> wrote:
> >
> >> > nit: any chance the popup can be made to respect the light/dark mode
> >> toggle
> >> in the docs? It seems to always use a light color scheme.
> >>
> >> From the looks of it, the colour can be configured via the
> >> 'data-project-color' parameter[1] so I imagine this could be done via
> >> updating the widget script with some custom JS on our side.
> >>
> >> [1] https://docs.kapa.ai/integrations/website-widget/configuration
> >>
> >> On Mon, 31 Mar 2025 at 17:15, Brian Hulette <bh...@apache.org>
> wrote:
> >>
> >>> Very cool, I tried a few queries and it provided good answers.
> >>>
> >>> nit: any chance the popup can be made to respect the light/dark mode
> >>> toggle
> >>> in the docs? It seems to always use a light color scheme.
> >>>
> >>> On Fri, Mar 28, 2025 at 5:05 PM Nic Crane <th...@gmail.com> wrote:
> >>>
> >>> > Cookbooks aren't part of the sources but we can certainly add them
> >>> later.
> >>> >
> >>> > The kapa folk recommend trying with an initial sunset of sources and
> >>> adding
> >>> > more in gradually, so I'll check back in on this next week and think
> >>> about
> >>> > what we might want to add next.
> >>> >
> >>> > On Fri, 28 Mar 2025, 21:45 Aldrin, <oc...@pm.me.invalid>
> wrote:
> >>> >
> >>> > > Ooh, yeah it's looking fairly effective. I asked some questions
> and I
> >>> > like
> >>> > > that the answers address differences in language implementations
> >>> (e.g.
> >>> > > python bindings vs cpp) and that there are relatively good code
> >>> > suggestions.
> >>> > >
> >>> > > I assume this means the tests and benchmarks are helping the RAG a
> >>> lot,
> >>> > so
> >>> > > general well done to all contributors. Out of curiosity, are the
> >>> > cookbooks
> >>> > > part of the learning/retrieval sources?
> >>> > >
> >>> > > Sent from Proton Mail <https://proton.me/mail/home> for iOS
> >>> > >
> >>> > >
> >>> > > On Fri, Mar 28, 2025 at 13:58, Neal Richardson <
> >>> > > neal.p.richardson@gmail.com
> >>> > > <On+Fri,+Mar+28,+2025+at+13:58,+Neal+Richardson+%3C%3Ca+href=>>
> >>> wrote:
> >>> > >
> >>> > > Awesome, thanks for doing this! I tried out a few queries and was
> >>> > > impressed
> >>> > > with what I got back.
> >>> > >
> >>> > > Neal
> >>> > >
> >>> > > On Fri, Mar 28, 2025 at 2:01 PM Nic Crane <th...@gmail.com>
> >>> wrote:
> >>> > >
> >>> > > > Hi folks,
> >>> > > >
> >>> > > > The kapa.ai bot is now live on the dev docs! We're using it to
> >>> test
> >>> > > > whether
> >>> > > > we can enhance user experience of our documentation via retrieval
> >>> > > augmented
> >>> > > > generation which generates answers to user questions by combining
> >>> > > > similarity searches of the docs and LLM output.
> >>> > > >
> >>> > > > You can use it by clicking the "Ask AI" button in the bottom
> right
> >>> > > corner
> >>> > > > of https://arrow.apache.org/docs/dev/index.html
> >>> > > >
> >>> > > > Currently we've just ingested the docs, but as we experiment
> more,
> >>> we
> >>> > > can
> >>> > > > add other sources like Zulip and the mailing lists if we think
> >>> these
> >>> > > will
> >>> > > > provide useful additional source material.
> >>> > > >
> >>> > > > Please give it a go and use the "good answer" and "bad answer"
> >>> buttons
> >>> > > so
> >>> > > > we can assess how useful this is likely to be for our end users;
> >>> bonus
> >>> > > > points for adding a better answer when you're not happy with the
> >>> one
> >>> > > given.
> >>> > > >
> >>> > > > I'll be checking in on it every so often and will report back
> here
> >>> with
> >>> > > the
> >>> > > > results.
> >>> > > >
> >>> > > > Thanks,
> >>> > > >
> >>> > > > Nic
> >>> > > >
> >>> > >
> >>> > >
> >>> >
> >>>
> >>
>