You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@netbeans.apache.org by Will Hartung <wi...@gmail.com> on 2024/11/16 23:35:23 UTC
Can I use Maven Indexer CLI for the NB Maven index?
Can I use the Maven Indexer CLI to maintain the maven index that NB manages
internally?
Specifically, I'd like to just have it run "routinely" (like, say,
overnight) to keep the local index up to date.
More specifically, I never want to see "unpacking index..." progress bar in
my IDE ever again.
It is a never ending pain point of frustration when it occurs.
I tend to leave my machine running for weeks on end, NB constantly open,
and, inevitably if I have to restart NB, it's likely not because I "want
to", but because "something happened" that forced a restart. And,
inevitably, it triggers a reload or whatever of the index and shuts me down
for far too long while it does whatever it does. Or simply having to be
conscious of the concept of "oh I better launch NB so it can go through
that update while I do something else".
Just now, I was doing something in NB. I started it days ago, and just let
it sit. And, of course, now, for my 10 line "hello world" project I was
messing with, off it goes. I was simply trying to update the pom file to a
later plugin version. "Oh, so sorry, updating the Index, so go suck an egg
meat sack, I'm busy". As if "oh, they started me, but I'm going to do this
lazily when they NEED it, instead of right away when I launch." Thanks a
lot.
So, if I can externalize the process to run on something elses time (i.e.
not mine), all the better.
And my brief research led me to the discovery that the Maven Indexer has a
CLI (and that's all I know, that there is something called "Maven Indexer",
and that it has a CLI, and that the NB maven.index module in NB has a
reference to it). So I was curious if that could be used to handle this
task.
Or, of course, is there some other way to never get blocked by the indexer.
Thanks!
Regards,
Will Hartung
Re: Can I use Maven Indexer CLI for the NB Maven index?
Posted by Will Hartung <wi...@gmail.com>.
On Mon, Nov 18, 2024 at 11:09 AM Peter Hull <pe...@gmail.com> wrote:
> Hi,
> Are you using a recent NB? Because the maven indexing got a lot better
> recently.
> Pete
>
>
Yes, I'm using 23.
Regards,
Will Hartung
Re: Can I use Maven Indexer CLI for the NB Maven index?
Posted by Peter Hull <pe...@gmail.com>.
Hi,
Are you using a recent NB? Because the maven indexing got a lot better
recently.
Pete
On Mon, 18 Nov 2024 at 19:03, Will Hartung <wi...@gmail.com> wrote:
>
>
> On Sun, Nov 17, 2024 at 10:47 AM Michael Bien <mb...@gmail.com> wrote:
>
>> I personally barely notice index updates since they don't happen all
>> that often. The one I just tested took about 150s. I don't think it ever
>> ran while I used completion and I needed something that wasn't in the
>> local index results already.
>>
>
> For whatever reason, they're intrusive enough that I take notice. I've
> even had one recently that took over an hour to update (that's abnormal, no
> idea why that took so long). It's impactful enough in my environment that I
> dread it and strive to avoid it (as I mentioned). If nothing else, it
> lights the fans up on my machine. It's just one of those triggering "oh no"
> things.
>
> If I could easily eliminate it entirely, I'd probably do it (sort of what
> I was trying for with the external index updater -- let it run at 3am and
> take as long as it wants).
>
> I do a lot of "green field" fiddling. PoCs and what not, so I'm constantly
> digging into the depths of the Maven repositories.
>
> Regards,
>
> Will Hartung
>
>
Re: Can I use Maven Indexer CLI for the NB Maven index?
Posted by Michael Bien <mb...@gmail.com>.
On 18.11.24 20:02, Will Hartung wrote:
>
>
> On Sun, Nov 17, 2024 at 10:47 AM Michael Bien <mb...@gmail.com> wrote:
>
> I personally barely notice index updates since they don't happen all
> that often. The one I just tested took about 150s. I don't think
> it ever
> ran while I used completion and I needed something that wasn't in the
> local index results already.
>
>
> For whatever reason, they're intrusive enough that I take notice. I've
> even had one recently that took over an hour to update (that's
> abnormal, no idea why that took so long).
forgot to reply to that one. What you saw there most likely is some kind
of server side connection throttling. The update segments are relatively
small - so both download and import don't take long under normal
conditions.
(I have seen something like that before during tests when dl dropped
down to a few kb/s, but this doesn't seem to happen very often)
I heard the compute and bandwidth cost for the index upkeep seem to be
high for maven central too and it doesn't scale well in its current
form. So the future for this feature is probably not certain.
I am planning to maintain the indexer in NetBeans as long as remote
indices are still available but probably won't put a lot of energy into
optimizations.
NB 25 might be (finally) able to migrate the index from NB 24 cache though:
https://github.com/apache/netbeans/pull/7976
-mbien
> It's impactful enough in my environment that I dread it and strive to
> avoid it (as I mentioned). If nothing else, it lights the fans up on
> my machine. It's just one of those triggering "oh no" things.
>
> If I could easily eliminate it entirely, I'd probably do it (sort of
> what I was trying for with the external index updater -- let it run at
> 3am and take as long as it wants).
>
> I do a lot of "green field" fiddling. PoCs and what not, so I'm
> constantly digging into the depths of the Maven repositories.
>
> Regards,
>
> Will Hartung
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@netbeans.apache.org
For additional commands, e-mail: users-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Can I use Maven Indexer CLI for the NB Maven index?
Posted by Will Hartung <wi...@gmail.com>.
On Sun, Nov 17, 2024 at 10:47 AM Michael Bien <mb...@gmail.com> wrote:
> I personally barely notice index updates since they don't happen all
> that often. The one I just tested took about 150s. I don't think it ever
> ran while I used completion and I needed something that wasn't in the
> local index results already.
>
For whatever reason, they're intrusive enough that I take notice. I've even
had one recently that took over an hour to update (that's abnormal, no idea
why that took so long). It's impactful enough in my environment that I
dread it and strive to avoid it (as I mentioned). If nothing else, it
lights the fans up on my machine. It's just one of those triggering "oh no"
things.
If I could easily eliminate it entirely, I'd probably do it (sort of what I
was trying for with the external index updater -- let it run at 3am and
take as long as it wants).
I do a lot of "green field" fiddling. PoCs and what not, so I'm constantly
digging into the depths of the Maven repositories.
Regards,
Will Hartung
Re: Can I use Maven Indexer CLI for the NB Maven index?
Posted by Michael Bien <mb...@gmail.com>.
On 17.11.24 17:40, Will Hartung wrote:
>
>
> On Sat, Nov 16, 2024 at 6:17 PM Michael Bien <mb...@gmail.com> wrote:
>
> Hi,
>
> the index updates happen in the background, can you clarify what you
> mean by being "blocked by the indexer"? It shouldn't block anything.
>
>
> A specific example is in the pom file, you can go to a dependency,
> blank out the version field, CMD-SPACE for the autocomplete, and it
> will show you "all" of the versions available.
>
> There are times when I do that and I get an "Incomplete result, still
> processing indices" or something like that message. At that point, I'm
> effectively stuck.
ah I see. In that case you will still get results from the local index,
while the remote index is updating. I tested this scenario and cancel
did make the remote index immediately available again. This will also
pause further download attempts for ~2h as previously mentioned.
I personally barely notice index updates since they don't happen all
that often. The one I just tested took about 150s. I don't think it ever
ran while I used completion and I needed something that wasn't in the
local index results already.
>
> I use NB extensively and maven exclusively, so for me I don't consider
> this optional.
nice to hear that someone is using this feature.
-mbien
>
> Thanks for the info.
>
> Regards,
>
> Will Hartung
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@netbeans.apache.org
For additional commands, e-mail: users-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Can I use Maven Indexer CLI for the NB Maven index?
Posted by Will Hartung <wi...@gmail.com>.
On Sat, Nov 16, 2024 at 6:17 PM Michael Bien <mb...@gmail.com> wrote:
> Hi,
>
> the index updates happen in the background, can you clarify what you
> mean by being "blocked by the indexer"? It shouldn't block anything.
>
A specific example is in the pom file, you can go to a dependency, blank
out the version field, CMD-SPACE for the autocomplete, and it will show you
"all" of the versions available.
There are times when I do that and I get an "Incomplete result, still
processing indices" or something like that message. At that point, I'm
effectively stuck.
> If it happens at a time you don't want it to run and it already started
> (e.g low battery), you can simply cancel the task and it won't try again
> for a few hours or toggle remote index downloads in the options.
>
If I run into it again, I can try to cancel and see if those messages go
away.
> One way to keep it up to date would be to let NB check on startup or
> daily for updates (also in the indexer options). Note, that this doesn't
> mean that every check automatically results in an update, since it will
> only update if the remote index actually published a new segment.
>
> There is no supported way to manage the index externally atm. The index
> import is a bit customized and drops a few fields. It might still work
> but the footprint might be larger and this is not tested.
>
>
> Also keep in mind that remote index downloads are completely optional.
> In case you don't use maven or NetBeans that often: NetBeans does also
> scan the local maven cache, so most features will be still functional
> even though information about the newest artifacts won't be available
> until they are in the cache.
>
I use NB extensively and maven exclusively, so for me I don't consider this
optional.
Thanks for the info.
Regards,
Will Hartung
Re: Can I use Maven Indexer CLI for the NB Maven index?
Posted by Michael Bien <mb...@gmail.com>.
Hi,
the index updates happen in the background, can you clarify what you
mean by being "blocked by the indexer"? It shouldn't block anything.
If it happens at a time you don't want it to run and it already started
(e.g low battery), you can simply cancel the task and it won't try again
for a few hours or toggle remote index downloads in the options.
One way to keep it up to date would be to let NB check on startup or
daily for updates (also in the indexer options). Note, that this doesn't
mean that every check automatically results in an update, since it will
only update if the remote index actually published a new segment.
There is no supported way to manage the index externally atm. The index
import is a bit customized and drops a few fields. It might still work
but the footprint might be larger and this is not tested.
Also keep in mind that remote index downloads are completely optional.
In case you don't use maven or NetBeans that often: NetBeans does also
scan the local maven cache, so most features will be still functional
even though information about the newest artifacts won't be available
until they are in the cache.
regards,
-mbien
On 17.11.24 00:35, Will Hartung wrote:
> Can I use the Maven Indexer CLI to maintain the maven index that NB
> manages internally?
>
> Specifically, I'd like to just have it run "routinely" (like, say,
> overnight) to keep the local index up to date.
>
> More specifically, I never want to see "unpacking index..." progress
> bar in my IDE ever again.
>
> It is a never ending pain point of frustration when it occurs.
>
> I tend to leave my machine running for weeks on end, NB constantly
> open, and, inevitably if I have to restart NB, it's likely not because
> I "want to", but because "something happened" that forced a restart.
> And, inevitably, it triggers a reload or whatever of the index and
> shuts me down for far too long while it does whatever it does. Or
> simply having to be conscious of the concept of "oh I better launch NB
> so it can go through that update while I do something else".
>
> Just now, I was doing something in NB. I started it days ago, and just
> let it sit. And, of course, now, for my 10 line "hello world" project
> I was messing with, off it goes. I was simply trying to update the pom
> file to a later plugin version. "Oh, so sorry, updating the Index, so
> go suck an egg meat sack, I'm busy". As if "oh, they started me, but
> I'm going to do this lazily when they NEED it, instead of right away
> when I launch." Thanks a lot.
>
> So, if I can externalize the process to run on something elses time
> (i.e. not mine), all the better.
>
> And my brief research led me to the discovery that the Maven Indexer
> has a CLI (and that's all I know, that there is something called
> "Maven Indexer", and that it has a CLI, and that the NB maven.index
> module in NB has a reference to it). So I was curious if that could be
> used to handle this task.
>
> Or, of course, is there some other way to never get blocked by the
> indexer.
>
> Thanks!
>
> Regards,
>
> Will Hartung
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@netbeans.apache.org
For additional commands, e-mail: users-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists