Project

General

Profile

Bug #39556

npm problem causes "make-dist" to fail when directory contains colon character

Added by Nathan Cutler about 2 years ago. Updated 21 days ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
pacific, octopus, nautilus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Environment: openSUSE Leap 15.1, Tumbleweed, Python 2+3 and Python 3 only

tl;dr - when the directory path contains a colon (':') character, make-dist fails in the npm phase with the error messages shown below. When the directory does not contain a colon, the bug does not manifest and the npm phase succeeds.

When I run make-dist on tip of master branch, I get:

Collecting nodeenv
Installing collected packages: nodeenv
Successfully installed nodeenv-1.3.3
 * Install prebuilt node (10.13.0) ..... done.
 * Appending data to /tmp/tmp.PRGQz8NIRr/bin/activate
 * Appending data to /tmp/tmp.PRGQz8NIRr/bin/activate.fish

> fsevents@1.2.4 install /home/smithfarm/obs/filesystems:ceph:octopus:upstream/ceph/tmp.gs6pfXbgP2/src/pybind/mgr/dashboard/frontend/node_modules/fsevents
> node install

> node-sass@4.11.0 install /home/smithfarm/obs/filesystems:ceph:octopus:upstream/ceph/tmp.gs6pfXbgP2/src/pybind/mgr/dashboard/frontend/node_modules/node-sass
> node scripts/install.js

Cached binary found at /home/smithfarm/.npm/node-sass/4.11.0/linux-x64-64_binding.node

> node-sass@4.11.0 postinstall /home/smithfarm/obs/filesystems:ceph:octopus:upstream/ceph/tmp.gs6pfXbgP2/src/pybind/mgr/dashboard/frontend/node_modules/node-sass
> node scripts/build.js

Binary found at /home/smithfarm/obs/filesystems:ceph:octopus:upstream/ceph/tmp.gs6pfXbgP2/src/pybind/mgr/dashboard/frontend/node_modules/node-sass/vendor/linux-x64-64/binding.node
Testing binary
Binary is fine

> fsevents@1.2.7 install /home/smithfarm/obs/filesystems:ceph:octopus:upstream/ceph/tmp.gs6pfXbgP2/src/pybind/mgr/dashboard/frontend/node_modules/@compodoc/compodoc/node_modules/fsevents
> node install

> @compodoc/compodoc@1.1.9 postinstall /home/smithfarm/obs/filesystems:ceph:octopus:upstream/ceph/tmp.gs6pfXbgP2/src/pybind/mgr/dashboard/frontend/node_modules/@compodoc/compodoc
> opencollective postinstall

sh: opencollective: command not found
npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn
npm ERR! @compodoc/compodoc@1.1.9 postinstall: `opencollective postinstall`
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the @compodoc/compodoc@1.1.9 postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/smithfarm/.npm/_logs/2019-05-01T11_31_10_328Z-debug.log

This only happens if the directory path contains a colon (':') character.


Related issues

Copied to Ceph - Backport #50356: nautilus: npm problem causes "make-dist" to fail when directory contains colon character In Progress
Copied to Ceph - Backport #50357: octopus: npm problem causes "make-dist" to fail when directory contains colon character In Progress
Copied to Ceph - Backport #50358: pacific: npm problem causes "make-dist" to fail when directory contains colon character In Progress

History

#1 Updated by Nathan Cutler about 2 years ago

  • Category set to dashboard/qa

#2 Updated by Nathan Cutler about 2 years ago

  • Description updated (diff)

#3 Updated by Nathan Cutler about 2 years ago

  • Status changed from New to Can't reproduce

#4 Updated by Nathan Cutler about 2 years ago

  • Subject changed from mgr/dashboard: npm problem causes "make check" to fail to mgr/dashboard: npm problem causes "make-dist" to fail
  • Status changed from Can't reproduce to New

#5 Updated by Nathan Cutler about 2 years ago

  • Description updated (diff)

#6 Updated by Nathan Cutler about 2 years ago

  • Description updated (diff)

#7 Updated by Nathan Cutler about 2 years ago

#8 Updated by Nathan Cutler about 2 years ago

When I revert the compodoc commit, "make-dist" throws a different error:

Collecting nodeenv
Installing collected packages: nodeenv
Successfully installed nodeenv-1.3.3
 * Install prebuilt node (10.13.0) ..... done.
 * Appending data to /tmp/tmp.Q9zQYPkTHK/bin/activate
 * Appending data to /tmp/tmp.Q9zQYPkTHK/bin/activate.fish

> fsevents@1.2.4 install /home/smithfarm/filesystems:ceph:octopus:upstream/ceph/tmp.tMwmwqMkNd/src/pybind/mgr/dashboard/frontend/node_modules/fsevents
> node install

> node-sass@4.11.0 install /home/smithfarm/filesystems:ceph:octopus:upstream/ceph/tmp.tMwmwqMkNd/src/pybind/mgr/dashboard/frontend/node_modules/node-sass
> node scripts/install.js

Cached binary found at /home/smithfarm/.npm/node-sass/4.11.0/linux-x64-64_binding.node

> node-sass@4.11.0 postinstall /home/smithfarm/filesystems:ceph:octopus:upstream/ceph/tmp.tMwmwqMkNd/src/pybind/mgr/dashboard/frontend/node_modules/node-sass
> node scripts/build.js

Binary found at /home/smithfarm/filesystems:ceph:octopus:upstream/ceph/tmp.tMwmwqMkNd/src/pybind/mgr/dashboard/frontend/node_modules/node-sass/vendor/linux-x64-64/binding.node
Testing binary
Binary is fine

> fsevents@1.2.7 install /home/smithfarm/filesystems:ceph:octopus:upstream/ceph/tmp.tMwmwqMkNd/src/pybind/mgr/dashboard/frontend/node_modules/@angular/compiler-cli/node_modules/fsevents
> node install

added 1556 packages in 14.823s

> ceph-dashboard@0.0.0 build /home/smithfarm/filesystems:ceph:octopus:upstream/ceph/tmp.tMwmwqMkNd/src/pybind/mgr/dashboard/frontend
> npm run env_build && ng build "--prod" "--progress=false" 

> ceph-dashboard@0.0.0 env_build /home/smithfarm/filesystems:ceph:octopus:upstream/ceph/tmp.tMwmwqMkNd/src/pybind/mgr/dashboard/frontend
> cp src/environments/environment.tpl.ts src/environments/environment.prod.ts && cp src/environments/environment.tpl.ts src/environments/environment.ts  && node ./environment.build.js

Environment variables have been set
sh: ng: command not found
npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn
npm ERR! ceph-dashboard@0.0.0 build: `npm run env_build && ng build "--prod" "--progress=false"`
npm ERR! spawn ENOENT
npm ERR! 
npm ERR! Failed at the ceph-dashboard@0.0.0 build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/smithfarm/.npm/_logs/2019-05-01T15_14_42_038Z-debug.log

#9 Updated by Tiago Melo about 2 years ago

I just created a new docker container using openSUSE Leap 15.1 and ceph master, and was not able to reproduce the problem.
I think similar problems have happened before and was related to cached files.

Before I compiled ceph I remove the following folders:

rm -rf /ceph/build/
rm -rf /ceph/src/pybind/mgr/dashboard/frontend/node_modules/
rm -rf /ceph/src/pybind/mgr/dashboard/frontend/dist/

I used this dockerfile: https://github.com/ricardoasmarques/ceph-dev-docker/blob/master/Dockerfile
and only changed the distro and installed bzip2 and python-setuptools.

#10 Updated by Nathan Cutler about 2 years ago

  • Status changed from New to Can't reproduce

Thanks, @Tiago - I also tried with a fresh VM and could not reproduce, sigh.

(BTW I always run "make-dist" in a fresh clone, so in my case the issue is being caused by something out-of-tree. Probably some interaction between python modules installed system-wide via pip and others installed via RPM.)

#11 Updated by Nathan Cutler almost 2 years ago

  • Status changed from Can't reproduce to New

Reopening. I can reproduce this in an openSUSE 15.1 docker container. The procedure for doing so is:

git clone https://github.com/smithfarm/heebeejeebee.git
cd heebeejeebee
./build.sh --rebuild-base
./build.sh

@Tiago - I noticed your docker container uses Tumbleweed. Could you try it with 15.1 instead?

#12 Updated by Nathan Cutler almost 2 years ago

  • Subject changed from mgr/dashboard: npm problem causes "make-dist" to fail to mgr/dashboard: npm problem causes "make-dist" to fail on openSUSE Leap 15.1

#13 Updated by Nathan Cutler almost 2 years ago

@Tiago: I see now you wrote you changed the distro in your Dockerfile. There must be a new dependency that your Dockerfile is installing but mine is not. I will try to figure out which one it is.

#14 Updated by Nathan Cutler almost 2 years ago

  • Subject changed from mgr/dashboard: npm problem causes "make-dist" to fail on openSUSE Leap 15.1 to mgr/dashboard: npm problem causes "make-dist" to fail when directory contains colon character
  • Description updated (diff)
  • Status changed from New to 12

Updated title and description to reflect Tiago's analysis - i.e., the problem only happens when the directory contains a colon (':') character.

#15 Updated by Lenz Grimmer almost 2 years ago

Is this something worthwhile reporting to the npm maintainers? Or maybe it's a known limitation?

#16 Updated by Patrick Donnelly over 1 year ago

  • Status changed from 12 to New

#17 Updated by Tiago Melo 10 months ago

  • Category changed from dashboard/qa to dashboard/build

#18 Updated by Ernesto Puerta 2 months ago

  • Status changed from New to Rejected

Closing as, per discussion in the corresponding NPM CLI issue, it seems that the <colon> is not a valid char in POSIX paths (in fact it's used for separating paths, like in the PATH environment variable).

#19 Updated by Nathan Cutler 29 days ago

  • Project changed from mgr to Ceph
  • Category deleted (dashboard/build)
  • Status changed from Rejected to In Progress
  • Assignee set to Nathan Cutler

Ernesto Puerta wrote:

Closing as, per discussion in the corresponding NPM CLI issue, it seems that the <colon> is not a valid char in POSIX paths (in fact it's used for separating paths, like in the PATH environment variable).

Thanks. The "fix", then, is to ensure that the full path to the directory where make-dist is run does not contain any ':' characters. I will open a PR to add a sanity check for this to the script.

#20 Updated by Nathan Cutler 29 days ago

  • Subject changed from mgr/dashboard: npm problem causes "make-dist" to fail when directory contains colon character to npm problem causes "make-dist" to fail when directory contains colon character

#21 Updated by Nathan Cutler 29 days ago

  • Status changed from In Progress to Fix Under Review
  • Pull request ID set to 40614

#22 Updated by Nathan Cutler 29 days ago

  • Backport set to pacific, octopus, nautilus

#23 Updated by Nathan Cutler 29 days ago

#24 Updated by Kefu Chai 21 days ago

  • Status changed from Fix Under Review to Pending Backport

#25 Updated by Backport Bot 21 days ago

  • Copied to Backport #50356: nautilus: npm problem causes "make-dist" to fail when directory contains colon character added

#26 Updated by Backport Bot 21 days ago

  • Copied to Backport #50357: octopus: npm problem causes "make-dist" to fail when directory contains colon character added

#27 Updated by Backport Bot 21 days ago

  • Copied to Backport #50358: pacific: npm problem causes "make-dist" to fail when directory contains colon character added

Also available in: Atom PDF