Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

shim npm package for IntelliJ/WebStorm (#100) #129

Closed
wants to merge 1 commit into from

Conversation

segrey
Copy link

@segrey segrey commented Jan 8, 2019

closes #100

Copy link
Member

@jasonkarns jasonkarns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. It has helped clarify a few questions. However, we won't be shipping this shim as part of nodenv since it's a workaround that is only necessary for Webstorm.

We may make it available as a plugin or something, but for now we've just documented the workaround in our FAQ wiki.

// Bundled npm package is detected as "/path/to/node/../lib/node_modules/npm/".
// For "$(nodenv root)/shims/node", it's "$(nodenv root)/lib/node_modules/npm/".

// To run a npm command, IntelliJ/WebStorm runs the main entry JavaScript file directly (npm/bin/npm-cli.js). Benefits:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

runs the main entry JavaScript file directly (npm/bin/npm-cli.js)

The main entry for npm is not npm-cli.js but rather lib/npm.js https://github.com/npm/cli/blob/ab0f0260e5b6333f98062fb2d9f4f9954d3ee6cd/package.json#L30 so I assume you mean that webstorm follows the bin setting, not main? But the package.json bin setting is what sets up the symlink into bin. So webstorm is just ignoring the symlink and hitting the configured file directly. Of course, this means you must be specially hardcoding the knowledge about which bin to follow, since npm has two (npx being the other)

Do you know if Webstorm would respect main if a bin isn't present?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, bin, not main. Yes, bin/npm-cli.js is hard-coded for npm currently.
WebStorm can respect bin property in future versions if needed.

// not found in $PATH due to configuration issues.
// * It allows to pass node options (--debug/v8-options) if needed. Though none are currently passed.

const nodenvRoot = require('path').resolve(__dirname, '../../../..')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't use const because it's not fully supported until node 6. (Though this simple usage would probably work fine, at least to 0.10.)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jasonkarns Do you need a fix for this? AFAIU, it will be available as a plugin, so the change probably goes to a different repo? Feel free to close this PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, I left this comment before I found out that all this was hardcoded. If I could drop a package in that had this shim, I would think about it as a plugin. But hardcoding the location? And having the shim get added to the user's PATH? I don't think so.

So this will likely stay as a manual fix users of webstorm will have to do themselves.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But hardcoding the location?

@jasonkarns Agree, the hardcoded location is not nice as it requires to maintain magic filepath, bin/npm-cli.js in this case. Well, there hasn't been cases like this before when a bundled npm package couldn't be found by path to Node.js interpreter. So it worked well so far. In this case a clearer way to integrate would be to rely on bin property. Will be fixed in 2019.1 (https://youtrack.jetbrains.com/issue/WEB-36645).

And having the shim get added to the user's PATH?

Do you mean lib/node_modules/npm/bin should be added to user's PATH? It worked for me locally without adding new locations to the PATH.

So this will likely stay as a manual fix users of webstorm will have to do themselves.

Will it work for you if a package.json with bin property pointing to bin/npm-cli.js is added along with bin/npm-cli.js. Once IntelliJ/WebStorm supports bin property from package.json, JavaScript file location could be changed if there is a need for this.
Just not sure nodenv&IntelliJ/WebStorm users should suffer, because of hardcoded location which will be changed unlikely.

Regarding a package with that shim, IIUC, such a plugin was intended to be installed additionally to nodenv installation, so it won't work out-of-the-box?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean lib/node_modules/npm/bin should be added to user's PATH?

No! If we create a package named jetbrains-npm-shim that has bin/npm-cli.js, and we specify it's bin via: "bin": "./bin/npm-cli.js", then when the user installs this package via npm or yarn, then they will have jetbrains-npm-shim in their PATH. That's how the bin property works. It tells npm which files to symlink into PATH!

Since webstorm doesn't use PATH to find the npm script, then there's no need for it to be in PATH at all. But having this shim get added to PATH would be a negative side effect for webstorm user's who need to use the shim package.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The suggested managers list looks like:
image

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jasonkarns Sorry for the delay. Text Project in "Package manager" field of the Npm run/debug configuration means that the package manager specified in "Settings | Languages & Frameworks | Node.js and NPM" should be used. Please check how "Package manager" is specified in "Settings | ... | Node.js and NPM". Looks like it's empty there.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. I thought I read somewhere that ../lib/node_modules/npm/bin/npm-cli.js would be picked up automatically if it existed relative to the node interpreter. It was definitely found by webstorm automatically (I can see it resolved the path and version number in the drop-down list), but I still had to explicitly select it when attempting to run a script.

I was thinking that it would be used automatically if found, without need to explicitly select it from the list.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jasonkarns According to your screenshots, the "npm" reference was resolved correctly to /usr/local/var/nodenv/lib/node_modules/npm. But the "Project" reference is unresolved unexpectedly. Could you please attach a screenshot of "Settings | Languages & Frameworks | Node.js and NPM" with expanded "Package manager" field?
Looks like a bug. Your file structure is still like described in #129 (comment), right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@segrey I cleared it out manually hoping to simulate a "fresh install". apparently I just needed to restart webstorm and it selected that one automatically. all good 👍

@@ -0,0 1,4 @@
{
"name": "npm",
"description": "a package manager for JavaScript; this package.json file helps IntelliJ/WebStorm to recognize the npm directory as package"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've confirmed that webstorm doesn't actually need any of this (strictly) and works as long as bin/npm-cli.js exists in the folder that's configured as the "package manager". I'm not against including this file, but if so, we should actually use it by declaring the main/bin file appropriately.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

package.json file was added as WebStorm checks its presence. It can be empty. Otherwise, lib/node_modules/npm won't be considered as a package.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dropped bin/npm-cli.js in a random folder and it works, without any package.json. Whatever folder you point webstorm to for npm, as long as it has bin/npm-cli.js it works. (Which is nice for a workaround, but a dreadful sign for what this means per the implementation.)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strange, it doesn't work for me without package.json (possibly empty).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workaround as documented works for me with webstorm 2018.3. https://github.com/nodenv/nodenv/wiki/FAQ#workaround

The workaround just adds bin/npm-cli.js to the nodenv root directory, and requires user to point webstorm to the nodenv root directory as the package manager. That was all that was necessary for it to work for me. No lib/node_modules, no package.json, etc. Just bin/npm-cli.js in the directory configured as the package manager.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, right, it works, because $(nodenv root) was specified as npm package directory manually in UI, and there is $(nodenv root)/package.json file related to nodenv itself.
The workaround with lib/node_modules/npm/{package.json, bin/npm-cli.js} won't require users to specify "Package manager" field manually, it will be inferred automatically if "Node interpreter" points to $(nodenv root)/shims/node.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a custom location for $NODENV_ROOT, so there is no package.json in mine:

$ cd `nodenv root`
$ pwd
/usr/local/var/nodenv
$ ls
bin/              default-packages  nodenv.d/         shims/            versions/
cache/            etc/              plugins/          sources/
$ ls bin/
npm-cli.js

The workaround with lib/node_modules/npm/{package.json, bin/npm-cli.js} won't require users to specify "Package manager" field manually, it will be inferred automatically if "Node interpreter" points to $(nodenv root)/shims/node.

That's good to know but it would be a "weird" install location for nodenv users. nodenv plugins live in $(nodenv root)/plugins. My intention for the shim plugin is to be installable as a global npm package or as a nodenv plugin. But both installations would require manually specifying its location via webstorm package manager, since neither approach would end up in $(nodenv root)/lib/node_modules/...

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I forget that an absolute path to npm package is specified in UI. Existence of package.json is checked only when inferring npm package automatically by node interpreter location or npm executable. Manually specified npm packages are not checked for package.json.

@jasonkarns
Copy link
Member

I tried making a plugin (essentially an npm package) that would have this shim. Evidently webstorm doesn't actually read the main property, nor the bin property because the shim file is only "found" successfully if actually placed at bin/npm-cli.js. So the path is completely hardcoded in webstorm instead of actually using the package metadata provided by node modules. ☹️

@segrey
Copy link
Author

segrey commented Jan 8, 2019

it's a workaround that is only necessary for Webstorm

Maybe it can be useful for other tools running npm commands via specifying an explicit path to a JavaScript file.

@jasonkarns
Copy link
Member

Maybe it can be useful for other tools running npm commands via specifying an explicit path to a JavaScript file.

But there would be no need for it to have a particular filename. Or for it to be in bin/. Any other use case that needs to invoke a particular js file can do so via standard means of execution. This workaround is only necessary because webstorm doesn't invoke npm from PATH and doesn't allow selecting an arbitrary executable as the package manager.

@jasonkarns jasonkarns closed this Jan 9, 2019
@segrey
Copy link
Author

segrey commented Jan 9, 2019

But there would be no need for it to have a particular filename. Or for it to be in bin/. Any other use case that needs to invoke a particular js file can do so via standard means of execution. This workaround is only necessary because webstorm doesn't invoke npm from PATH and doesn't allow selecting an arbitrary executable as the package manager.

Regarding supporting bin property, see the thread above (it will be supported in the next WebStorm version, 2019.1).
I just wanted to say that some tools might also want to run a JavaScript file directly (via reading bin property if that matter) to be able to pass node parameters like --debug/v8-options/etc. or to make sure that a particular node interpreter is used to run a npm command. In this case, imitating structure of a standard node installation with npm might help.

@segrey
Copy link
Author

segrey commented Jan 9, 2019

@jasonkarns Please let me know if nodenv can help WebStorm to resolve npm package directory.

@jasonkarns
Copy link
Member

some tools might also want to run a JavaScript file directly (via reading bin property if that matter) to be able to pass node parameters like --debug/v8-options/etc

This is not a problem that's unique to Webstorm and it already has existing and well known solutions.. Webstorm's attempt to "solve" this problem simply violates how tools are expected to be found and run (via CLI, using PATH and executables).

Firstly, nearly every configuration for node can be provided via environment variables: https://nodejs.org/api/all.html#cli_environment_variables. So they could be set by the user's actual environment, or even inline in the npm script itself: "foo": "NODE_DEBUG=x node foo"

Secondly, npm itself has a mechanism to set node flags: --node-options https://docs.npmjs.com/misc/config#node-options. Which could be added using Webstorm's runner configuration.

Thirdly, an npm script can invoke node any way it wants. It could pass flags to node directly: "foo": "node --debug my-script.js"

Lastly, the npm script itself could provide a direct path to the node executable desired. Or the script being executed could have a shebang that points to a specific path.

In short, I think this "problem" is already solved via existing means that are well known and standard across tools and environments. Webstorm's attempt to solve it violates expectations for how tools are normally run and invoked, and in so doing, causes problems for tooling that relies on those standard means of invocation.

@jasonkarns
Copy link
Member

jasonkarns commented Jan 9, 2019

Please let me know if nodenv can help WebStorm to resolve npm package directory

@segrey thanks! I have a working prototype of a plugin and npm package that will solve this for nodenv users. I'll ping you when I publish. (and thanks for digging into this with me)

@segrey
Copy link
Author

segrey commented Jan 9, 2019

Webstorm's attempt to "solve" this problem simply violates how tools are expected to be found and run (via CLI, using PATH and executables).
In short, I think this "problem" is already solved via existing means that are well known and standard across tools and environments. Webstorm's attempt to solve it violates expectations for how tools are normally run and invoked, and in so doing, causes problems for tooling that relies on those standard means of invocation.

@jasonkarns I'm not sure that the current approach is a violation. IIUC, you suggest to run npm executable directly, not /path/to/node /path/to/npm-cli.js, right? It will work, but has downsides: it won't be possible to change node interpreter and if node is not in PATH due to misconfiguration, it won't work. This is a different approach and will likely lead to a bunch of issues, because WebStorm doesn't have the same environment as a terminal. Although, making it optional is possible.

Regarding passing node options: yes, NODE_OPTIONS is a valid way to debug/pass v8-options, indeed, though it targets node >= 8. Other ways won't work (NODE_DEBUG controls util.debuglog()) or are inconvenient unfortunately (specifying manually "foo": "node --debug my-script.js" or modifying a shebang).

@jasonkarns
Copy link
Member

They might be inconvenient but they are well established and don't require a bunch of hoop-jumping just for a single IDE. nodenv works out of the box vim and vscode, and atom and sublime text precisely because they all invoke npm from PATH as normal executables. and all the other editors seem fine with the standard mechanisms for passing node options and flags.

@jasonkarns
Copy link
Member

jasonkarns commented Jan 9, 2019

@segrey Here's the planned shim plugin package: https://github.com/nodenv/jetbrains-npm

Still working on the readme.

If installed via npm or yarn as a global package, its installation folder must be specified as the package manager in webstorm. It will also require one to be using the default nodenv root location (~/.nodenv) OR NODENV_ROOT env var must set and available to webstorm (ie, in .profile).

If installed as a nodenv plugin (cloned to $(nodenv root)/plugins) it will still require webstorm's package manager be pointing to its installation folder. But this installation method will work without NODENV_ROOT being set in the environment.

Lastly, it can be installed by cloning as $(nodenv root)/lib/node_modules/npm. This method still requires the package manager setting be set explicitly. However, at least webstorm finds the path automatically. So one just needs to pick "npm" from the prepopulated package manager list.

The latter two methods of installation are preferred.

@segrey
Copy link
Author

segrey commented Jan 9, 2019

Awesome, thanks a lot!

@jasonkarns
Copy link
Member

@segrey 🚀 any further discussion regarding the shim can be made via an issue in that repo!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for using nodenv in Intellij Idea/Webstorm
2 participants