You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@netbeans.apache.org by GitBox <gi...@apache.org> on 2020/10/09 19:16:33 UTC
[GitHub] [netbeans] Chris2011 opened a new pull request #2438: NETBEANS-4893 - Added node_modules and bower_components to the option
Chris2011 opened a new pull request #2438:
URL: https://github.com/apache/netbeans/pull/2438
ignore files pattern in the misc tab.
It will prevent indexing those folders while using NetBeans for the
first time and opening a web project with a package.json and/or a
bower.json.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] matthiasblaesing commented on pull request #2438: NETBEANS-4893 - Added node_modules and bower_components to the option
Posted by GitBox <gi...@apache.org>.
matthiasblaesing commented on pull request #2438:
URL: https://github.com/apache/netbeans/pull/2438#issuecomment-706499873
If I understand correctly, this will remove code completion for javascript, if the corresponding library is in node_modules? Or what I'm missing?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] Chris2011 commented on pull request #2438: NETBEANS-4893 - Added node_modules and bower_components to the option
Posted by GitBox <gi...@apache.org>.
Chris2011 commented on pull request #2438:
URL: https://github.com/apache/netbeans/pull/2438#issuecomment-706745636
So everyne who uses npm and a major cli to create a project is just a "normal" JS user BTW.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] lkishalmi commented on pull request #2438: NETBEANS-4893 - Added node_modules and bower_components to the option
Posted by GitBox <gi...@apache.org>.
lkishalmi commented on pull request #2438:
URL: https://github.com/apache/netbeans/pull/2438#issuecomment-706467166
Seems to be trivial enough. Merging.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] matthiasblaesing commented on pull request #2438: NETBEANS-4893 - Added node_modules and bower_components to the option
Posted by GitBox <gi...@apache.org>.
matthiasblaesing commented on pull request #2438:
URL: https://github.com/apache/netbeans/pull/2438#issuecomment-706744123
@Chris2011 so this is a big hammer that helps you, but might be a problem for "normal" JS users.
I can explain why indexing in netbeans is slow: Because the JS indexer tries to deal with the mess, that is the JS ecosystem. The JS files are parsed in context and that is a dramatic slowdown.
By your description your "fix" basicly breaks all index based support for every JS project, that uses npm to manage its dependencies.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] Chris2011 commented on pull request #2438: NETBEANS-4893 - Added node_modules and bower_components to the option
Posted by GitBox <gi...@apache.org>.
Chris2011 commented on pull request #2438:
URL: https://github.com/apache/netbeans/pull/2438#issuecomment-706742979
@matthiasblaesing unfortunately you are right. But what speaks against it is the following case:
I have 5 - 6 Web projects open (next to a lot of others, web independent projects).
One project is the biggest, that I'm working on, it is from my company. The node_modules folder has 115.605 files and 18.964 folders. The node_modules folder is 963 mb big. It is a mono repo with nrwl-nx - Angular application. After I removed the node_modules from the ignored files pattern, NetBeans started indexing. This project, needed round about 1h to finish and the web projects needed round about 1 more hour. So all in all it ~ 2h for indexing/background scanning the node_modules folder. Which seems waaaaaay to long.
My environment is a Surface Laptop 3, Windows 10, latest version with an i7, 16GB RAM and a M2 SSD. Afaik, not even Windows need that much time to indexing 100k files. I know this is another case but there is also another application called everything: https://en.wikipedia.org/wiki/Everything_(software) It took round about 30s - 1m to index the whole SSD with 3.638.195 files on it. It is written in C/C++ and assembler. Just to let you know, that I don't understand, why NetBeans took so long so as you see it gets faster. Btw Windows Defender is disabled, so this can't be the problem.
So long story short, whether there is possible to be as fast as the everything tool, which would be pretty awesome in any case and second, if not possible my solution for indexing the node_modules would be to read out all packages from the package.json and bower.json, that should be fast enough I guess. Also, indexing all modules, that are used by require() and the ES 6 modules with import etc. So that would be a good improvement to still have code completion for all imported modules and maybe sub modules like incrementialy index.
Is one of those solutions possible? IMHO, it is not needed to index all files in node_modules. Or someone knows how VS Code and WebStorm does it. WebStorm is also fast enough for my point of view. It just needed ~2min to index the company project which is quite good.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] Chris2011 commented on pull request #2438: NETBEANS-4893 - Added node_modules and bower_components to the option
Posted by GitBox <gi...@apache.org>.
Chris2011 commented on pull request #2438:
URL: https://github.com/apache/netbeans/pull/2438#issuecomment-706745496
what is a "normal" js user? If you use npm you install your dep and it installs more deps, etc. The other 4 projects are not that big but still have > 20k files inside of it and those are normal JS projects created with Vue CLI or Angular CLI etc. So some other people complained about that in Stack Overflow too "How can I prevent node_modules from indexing in NetBeans". And as I said, why does WebStorm not has this problem with JS parsing?
> I can explain why indexing in netbeans is slow: Because the JS indexer tries to deal with the mess, that is the JS ecosystem. The JS files are parsed in context and that is a dramatic slowdown.
Just in NetBeans, not in any other major IDEs/Editor. I tried it couple of minutes ago with WebStorm, the result is significant and this is what I expect for NetBeans too. WebStorm also indexes the node_modules folder as I can see, when I open a web project, what my company project was. So as I understand it right, just the JS parser in NetBeans is slow and not a JS parser in general.
Anyway, thx for the hint with the indexing we can revert it, but we need a solution for that. Here again:
* Make the JS paraser as fast as possible to make it comparable with other major IDEs. 2h is to long and it is even possible.
* Just index the packages from the package.json and bower.json because those are the packages that are used via require() and import
* Make the indexing in NetBeans as fast as possible and take a look into the everything tool.
Yes I can understand that not just indexing, but also parsing is done here. So everything could be just a hint for indexing and not for parsing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] matthiasblaesing edited a comment on pull request #2438: NETBEANS-4893 - Added node_modules and bower_components to the option
Posted by GitBox <gi...@apache.org>.
matthiasblaesing edited a comment on pull request #2438:
URL: https://github.com/apache/netbeans/pull/2438#issuecomment-706744123
@Chris2011 so this is a big hammer that helps you, but might be a problem for "normal" JS users.
I can explain why indexing in netbeans is slow: Because the JS indexer tries to deal with the mess, that is the JS ecosystem. The JS files are parsed in context and that is a dramatic slowdown.
By your description your "fix" basicly breaks all index based support for every JS project, that uses npm to manage its dependencies. The only support that is retained is typescript completion, as that is handled by the typescript language server.
Given, that the side effect described above this is a -1 for me.
Edit: added personal judgement.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] matthiasblaesing commented on pull request #2438: NETBEANS-4893 - Added node_modules and bower_components to the option
Posted by GitBox <gi...@apache.org>.
matthiasblaesing commented on pull request #2438:
URL: https://github.com/apache/netbeans/pull/2438#issuecomment-706746666
@Chris2011 you are barking at the wrong tree:
a) I'm not the author of javascript and/or javascript2 implementation in JS and even if I were, they were conceived in a different time
b) Yes I noticed the problem too
c) Breaking index based JS features is not a solution, but just another bug
You could run a profiler an see where the time is burned. If I remember correctly the biggest problem was the support for one special framework.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] lkishalmi merged pull request #2438: NETBEANS-4893 - Added node_modules and bower_components to the option
Posted by GitBox <gi...@apache.org>.
lkishalmi merged pull request #2438:
URL: https://github.com/apache/netbeans/pull/2438
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] Chris2011 edited a comment on pull request #2438: NETBEANS-4893 - Added node_modules and bower_components to the option
Posted by GitBox <gi...@apache.org>.
Chris2011 edited a comment on pull request #2438:
URL: https://github.com/apache/netbeans/pull/2438#issuecomment-706745636
So everyne who uses npm and a major cli to create a project is just a "normal" JS user BTW. Also we have business cases and not business cases. Or maybe enterprise applications and not so enterprise.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] Chris2011 commented on pull request #2438: NETBEANS-4893 - Added node_modules and bower_components to the option
Posted by GitBox <gi...@apache.org>.
Chris2011 commented on pull request #2438:
URL: https://github.com/apache/netbeans/pull/2438#issuecomment-706748377
@matthiasblaesing you are right, so let us revert the change :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] emilianbold commented on pull request #2438: NETBEANS-4893 - Added node_modules and bower_components to the option
Posted by GitBox <gi...@apache.org>.
emilianbold commented on pull request #2438:
URL: https://github.com/apache/netbeans/pull/2438#issuecomment-706746126
I wonder how parallel the indexer is. I know we have multithreading there
but I forgot how much: an i7 means 8 threads at least.
It would be very cool to have pre-computed indexes which are just
downloaded for a node module. Then there's no CPU waste, just bandwidth.
In an alternate universe it would be very cool to look into the indexing
infra and decouple it a bit, maybe throw it to a separate process, etc.
Lots of indexing problems with other languages too.
--emi
On Sun, Oct 11, 2020 at 9:01 PM Chrizzly <no...@github.com> wrote:
> @matthiasblaesing <https://github.com/matthiasblaesing> unfortunately you
> are right. But what speaks against it is the following case:
>
> I have 5 - 6 Web projects open (next to a lot of others, web independent
> projects).
> One project is the biggest, that I'm working on, it is from my company.
> The node_modules folder has 115.605 files and 18.964 folders. The
> node_modules folder is 963 mb big. It is a mono repo with nrwl-nx - Angular
> application. After I removed the node_modules from the ignored files
> pattern, NetBeans started indexing. This project, needed round about 1h to
> finish and the web projects needed round about 1 more hour. So all in all
> it ~ 2h for indexing/background scanning the node_modules folder. Which
> seems waaaaaay to long.
>
> My environment is a Surface Laptop 3, Windows 10, latest version with an
> i7, 16GB RAM and a M2 SSD. Afaik, not even Windows need that much time to
> indexing 100k files. I know this is another case but there is also another
> application called everything:
> https://en.wikipedia.org/wiki/Everything_(software) It took round about
> 30s - 1m to index the whole SSD with 3.638.195 files on it. It is written
> in C/C++ and assembler. Just to let you know, that I don't understand, why
> NetBeans took so long so as you see it gets faster. Btw Windows Defender is
> disabled, so this can't be the problem.
>
> So long story short, whether there is possible to be as fast as the
> everything tool, which would be pretty awesome in any case and second, if
> not possible my solution for indexing the node_modules would be to read out
> all packages from the package.json and bower.json, that should be fast
> enough I guess. Also, indexing all modules, that are used by require() and
> the ES 6 modules with import etc. So that would be a good improvement to
> still have code completion for all imported modules and maybe sub modules
> like incrementialy index.
>
> Is one of those solutions possible? IMHO, it is not needed to index all
> files in node_modules. Or someone knows how VS Code and WebStorm does it.
> WebStorm is also fast enough for my point of view. It just needed ~2min to
> index the company project which is quite good.
>
> —
> You are receiving this because your review was requested.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/netbeans/pull/2438#issuecomment-706742979>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHSCQVC7DYATC3XWLBO2GTSKHXHHANCNFSM4SKOCMLA>
> .
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists