You are viewing a plain text version of this content. The canonical link for it is here.
Posted to devnull@infra.apache.org by "kirillgroshkov (GitHub)" <gi...@apache.org> on 2018/08/24 21:06:38 UTC
[GitHub] [cordova-cli] kirillgroshkov opened issue #303: Please
restore `--nofetch` flag to support Yarn
We're using Yarn and without `--nofetch` our `node_modules` folder gets screwed by npm (invoked by `cordova-fetch`). We would really really like `--nofetch` back to be able to work.
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] raphinesse commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "raphinesse (GitHub)" <gi...@apache.org>.
@kirillgroshkov Thanks for testing! Could you please re-run your experiment using `--verbose` (i.e. `cordova platform add android --verbose`) and share the output of that and running `cordova info` in your project root? Always using the nightly. That would be great!
*Edit:* Oh, and please be sure to do a `yarn install` before, so yarn actually installs the cordova dependencies from your package.json.
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] paulsouche commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "paulsouche (GitHub)" <gi...@apache.org>.
Hi,
Can't wait for this too ! But what about yarn workspaces and plug and play ? Dependencies won't always be local.
Thanks anyway for the great job
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] cmckni3 commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "cmckni3 (GitHub)" <gi...@apache.org>.
Would the work need to be done in [cordova-lib](https://github.com/apache/cordova-lib)? cordova-cli appears to be a wrapper to me.
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] kirillgroshkov commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "kirillgroshkov (GitHub)" <gi...@apache.org>.
@leandro-manifesto totally agree, cordova should do it's job and not try to do npm's (or yarn's or pnpm's) job.
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] raphinesse commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "raphinesse (GitHub)" <gi...@apache.org>.
@kirillgroshkov Thanks for testing! Could you please re-run your experiment using `--verbose` (i.e. `cordova platform add android --verbose`) and share the output of that and running `cordova info` in your project root? Always using the nightly. That would be great!
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] kirillgroshkov commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "kirillgroshkov (GitHub)" <gi...@apache.org>.
Thanks for detailed comments!
Currently we use `cordova@7.1` because it has `--nofetch` working. It fetches plugins into `plugins` directory (not in `node_modules`).
What we would like to have:
- To be on the latest version of Cordova
- To keep using Yarn (and `yarn.lock`) as we do (and not use `npm` / `package-lock.json`)
Cordova 8 doesn't have `--nofetch`, so when we run e.g `cordova platform add android` it "screws" our `node_modules` folder, cause Yarn and npm are incompatible in a way they store files in `node_modules`.
Since I see the effort to help from your side - I'll do my best to test it with the nightly build as you suggested!
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] janpio commented on issue #303: Please restore
`--nofetch` flag to support Yarn
Posted by "janpio (GitHub)" <gi...@apache.org>.
@kirillgroshkov Cordova is Open Source - feel free to invest the time and submit a Pull Request if you really need this feature.
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] raphinesse commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "raphinesse (GitHub)" <gi...@apache.org>.
> Looking at the PR and description I don't believe it work (cause e.g we don't have plugins in node_modules at all, since we don't use `cordova-fetch`).
Well, then where does `yarn` install your dependencies if not in `node_modules`?
You should probably clarify your use case. Because I don't think `--no-fetch` will be coming back in its previous form. But maybe your use case can still be handled. What do you want to do and what do you want to happen? Step-by-step instructions help tremendously.
> Also, I don't feel comfortable to test nightlies, cause something else can be broken.
It won't set your machine on fire, I promise :wink:. But seriously, I am only asking you to test it, not to use it in production.
> But I'm ready to test when the next release is shipped! Will it be release of cordova (cli) package?
The change will be part of the next Cordova CLI major version. Testing it only after that has been released might be a little late. The earlier we know what you really need, the earlier we *can* start working on it :wink:
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] dpogue commented on issue #303: Please restore
`--nofetch` flag to support Yarn
Posted by "dpogue (GitHub)" <gi...@apache.org>.
I would love to do that, but people need to be able to run commands like `cordova platform add` and `cordova plugin add`, and that needs to add the platform/plugin to package.json and install it into node_modules. We need to use npm to do that.
What we can do (and I'm working on this) is to not run any package management commands as part of `cordova prepare`. We should tell people to run npm/yarn/pnpm themselves and then look for what we need in node_modules, and warn if we can't find it.
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] cmckni3 commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "cmckni3 (GitHub)" <gi...@apache.org>.
Would the work need to be done in [cordova-lib](https://github.com/apache/cordova-lib). cordova-cli appears to be a wrapper to me.
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] kirillgroshkov commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "kirillgroshkov (GitHub)" <gi...@apache.org>.
Crazy that such complex solutions as spypkg are developed to address this issue:) Should indicate the need for native yarn support!
If Capacitor (https://capacitor.ionicframework.com/) will reach GA sooner - we'll probably switch to it instead of waiting for yarn support here...
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] raphinesse commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "raphinesse (GitHub)" <gi...@apache.org>.
Actually, this *could* have been fixed by https://github.com/apache/cordova-fetch/pull/24. Could you please check if your problem persists when using a recent nightly build? `cordova@8.0.1-nightly.2018.8.26.9a2be3df` should work.
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] dpogue commented on issue #303: Please restore
`--nofetch` flag to support Yarn
Posted by "dpogue (GitHub)" <gi...@apache.org>.
To provide a bit of background here, there are several reasons that `--no-fetch` will not be coming back, but I *believe* what we want to accomplish should be compatible with Yarn.
The old installation code was unfortunately tied to npm2, which is several years old and relies on versions of dependencies with known security vulnerabilities. Because npm2 was installed as a dependency, running npm scripts within projects using Cordova would run with npm2 rather than the system npm. This was a frequent source of problems for users of Cordova, because npm2 and npm3 have different behaviour around peerDependencies.
The previous code was also doing custom behaviour that bypassed npm's caching helpers, didn't perform all the same verifications on the downloaded packages (which meant packages could get corrupt, and stick around), and was essentially us trying to poorly replicate a bunch of npm's behaviour. The code was unmaintainable and a source of continued bugs.
The way forward for Cordova is to have platforms and plugins listed in package.json and installed into node_modules. When you ask Cordova to install something, it will use npm to install it **if it is not already installed**.
>From what I understand, this means it should be possible to make this work with Yarn, but you'll need to add the platforms to your package.json manually, and install them with Yarn.
For instance, with the following in your package.json:
```json
{
"dependencies": {
"cordova-android": "^7.1.0"
},
"cordova": {
"platforms": [
"android"
]
}
}
```
After you run `yarn install`, you should be able to run `cordova prepare` (on the nightly build) to create the `platforms/android` folder without npm being invoked.
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] leandro-manifesto commented on issue #303:
Please restore `--nofetch` flag to support Yarn
Posted by "leandro-manifesto (GitHub)" <gi...@apache.org>.
Hello guys, here at my company we are using Cordova (and Ionic) to make some mobile apps.
We have a Jenkins agent doing Android builds with the following steps:
- `npm install`
- `rm -rf platforms/ plugins/`
- `cordova prepare android`
- `cordova compile android --release`
We have a mix of registry, git and local plugins and currently the prepare step takes a **long** time. From what I could understand cordova uses npm to install everything again, even though the install step already placed everything inside `node_modules`.
I tried using Yarn and pinning `cordova-fetch` to the patched version (apache/cordova-fetch@10870103c0d2a58ce0d8b91cf14053dd0c315b20) but npm is still being called for git and local dependencies.
I can't help but wonder if it wouldn't be better if cordova ditched the dependency manager role and only used the node resolution algorithm to find the referenced plugins (and platforms), allowing the user to pick whatever package manager he prefers.
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] janpio commented on issue #303: Please restore
`--nofetch` flag to support Yarn
Posted by "janpio (GitHub)" <gi...@apache.org>.
@kirillgroshkov Cordova is open source. Feel free to submit a Pull Request.
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] kirillgroshkov commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "kirillgroshkov (GitHub)" <gi...@apache.org>.
I tried nightly, previously cleaned `platforms` and `plugins` folders, but kept `node_modules` and `yarn.lock` intact. Here's the result.
`cordova platform add android`
```
Using cordova-fetch for cordova-android@7.1.0
Adding android project...
Creating Cordova project for the Android platform:
Path: platforms/android
Package: com.naturalcycles.cordova
Name: Natural_Cycles
Activity: MainActivity
Android target: android-27
Subproject Path: CordovaLib
Subproject Path: app
Android project created with cordova-android@7.1.0
Android Studio project detected
Android Studio project detected
*** /Users/kirill/Idea/NCApp3/cordova-hooks/before_prepare.js started ***
Command failed: yarn ts-node -r tsconfig-paths/register ./src/scripts/addSourceMappingUrl.ts --dir /Users/kirill/Idea/NCApp3/www/build
module.js:549
throw err;
^
Error: Cannot find module 'minimist'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/Users/kirill/Idea/NCApp3/node_modules/ts-node/dist/bin.js:9:16)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
error Command failed with exit code 1.
```
It broke on one of our cordova hooks, because previously it "screwed" `node_modules`, so package `minimist` disappeared (it's not even our dependency, it's some sub-dependency of something). It generated `package-lock.json` as a proof that it used `npm` underneath, and it's known that you cannot run `npm` when you have `node_modules` folder installed by Yarn.
So, problem is there.
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] raphinesse commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "raphinesse (GitHub)" <gi...@apache.org>.
@kirillgroshkov Thanks for testing! Could you please re-run your experiment using `--verbose` (i.e. `cordova platform add android --verbose`) and also share the output of running `cordova info` in your project root (always using the nightly). That would be great!
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] kirillgroshkov commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "kirillgroshkov (GitHub)" <gi...@apache.org>.
Looking at the PR and description I don't believe it work (cause e.g we don't have plugins in `node_modules` at all, since we don't use `cordova-fetch`).
Also, I don't feel comfortable to test nightlies, cause something else can be broken.
But I'm ready to test when the next release is shipped! Will it be release of `cordova` (cli) package?
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] kirillgroshkov commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "kirillgroshkov (GitHub)" <gi...@apache.org>.
@dpogue
1. Just an idea, maybe `cordova platform/plugin add` should just add lines to configuration file (`package.json`) and then user can just use his package manager (npm/yarn/pnpm) to get dependencies installed (running `npm i` / `yarn` / etc)?
> What we can do (and I'm working on this) is to not run any package management commands as part of cordova prepare. We should tell people to run npm/yarn/pnpm themselves and then look for what we need in node_modules, and warn if we can't find it.
Oh, that would be awesome, it'll solve the problem for so many people!
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] kirillgroshkov commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "kirillgroshkov (GitHub)" <gi...@apache.org>.
@dpogue
Just an idea, maybe `cordova platform/plugin add` should just add lines to configuration file (`package.json`) and then user can just use his package manager (npm/yarn/pnpm) to get dependencies installed (running `npm i` / `yarn` / etc)?
> What we can do (and I'm working on this) is to not run any package management commands as part of cordova prepare. We should tell people to run npm/yarn/pnpm themselves and then look for what we need in node_modules, and warn if we can't find it.
Oh, that would be awesome, it'll solve the problem for so many people!
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] kirillgroshkov commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "kirillgroshkov (GitHub)" <gi...@apache.org>.
@dpogue Great explanation! And I totally agree that instead of reinventing different package management methods it's easier for everyone to just stick to the same one that is also battle-tested (NPM), to have everything inside `node_modules`. Totally makes sense. The only missing part is then to make it work with Yarn (second-popular package management system). There's also 3rd most popular (PNPM) which I personally don't care about, but some people probably do.
Looks like the issue is:
- To run Yarn instead of NPM to install missing packages (detect it e.g by having `yarn.lock` present, instead of `package-lock.json`).
- To not reinstall plugins when they are already installed. (But that is less of a problem, cause even if you call `yarn` again one more time and nothing is changed - yarn will detect it itself and it'll take it no more than 1 second to return)
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] raphinesse commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "raphinesse (GitHub)" <gi...@apache.org>.
I tried nightly, previously cleaned `platforms` and `plugins` folders, but kept `node_modules` and `yarn.lock` intact. Here's the result of running `cordova platform add android`:
<details>
<summary>Click to view output</summary>
```
Using cordova-fetch for cordova-android@7.1.0
Adding android project...
Creating Cordova project for the Android platform:
Path: platforms/android
Package: com.naturalcycles.cordova
Name: Natural_Cycles
Activity: MainActivity
Android target: android-27
Subproject Path: CordovaLib
Subproject Path: app
Android project created with cordova-android@7.1.0
Android Studio project detected
Android Studio project detected
*** /Users/kirill/Idea/NCApp3/cordova-hooks/before_prepare.js started ***
Command failed: yarn ts-node -r tsconfig-paths/register ./src/scripts/addSourceMappingUrl.ts --dir /Users/kirill/Idea/NCApp3/www/build
module.js:549
throw err;
^
Error: Cannot find module 'minimist'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/Users/kirill/Idea/NCApp3/node_modules/ts-node/dist/bin.js:9:16)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
error Command failed with exit code 1.
```
</details>
It broke on one of our cordova hooks, because previously it "screwed" `node_modules`, so package `minimist` disappeared (it's not even our dependency, it's some sub-dependency of something). It generated `package-lock.json` as a proof that it used `npm` underneath, and it's known that you cannot run `npm` when you have `node_modules` folder installed by Yarn.
So, problem is there.
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org
[GitHub] [cordova-cli] marckassay commented on issue #303: Please
restore `--nofetch` flag to support Yarn
Posted by "marckassay (GitHub)" <gi...@apache.org>.
An alternative to @dpogue recommendation, is installing and configuring spypkg so that npm will be mapped to yarn. See configuration [here](https://www.npmjs.com/package/spypkg#yarn-in-lieu-of-npm-configuration-example).
[ Full content available at: https://github.com/apache/cordova-cli/issues/303 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org