Bug #24466
closed
build process runs "pip install" and "npm install" while building the mgr frontend
Added by Patrick McLean almost 6 years ago.
Updated about 3 years ago.
Description
The current build process runs "pip install" and "npm install" while building the mgr frontend. This is a violation of most distros' packaging policies, but more importantly this makes the frontend code someone is running be non-deterministic. The code that someone is running will depend on when they built ceph, not just the version they are running. This will quite likely complicate triaging and debugging of any issues that crop up.
I would suggest just listing python dependencies in the global dependency list, and bundling node dependencies in the distributed tarball, that way you can be sure that everyone who builds from the source tarball is actually running the same code.
I would also suggest separating out test dependencies from build dependencies in the npm packages.json, since the test dependencies download precompiled binaries for phantomjs (which is abandoned and vulnerable according to it's website) and other things.
Files
It also fails for various reasons on arm64 such as binaries not existing.
The node dependency issues are fixable by adding package-lock.json to git (which is the whole purpose of that file that was added in npm 5).
The package lock might help with the dependency drift, but it does not change the fact that the build system is fetching random files off the internet halfway through the build. Everything that is needed for the build should be included in the tarball, that is generally the purpose of providing tarballs rather than just telling people to get it from git.
- Project changed from Ceph to mgr
- Category changed from build to 132
Patrick McLean wrote:
The package lock might help with the dependency drift, but it does not change the fact that the build system is fetching random files off the internet halfway through the build. Everything that is needed for the build should be included in the tarball, that is generally the purpose of providing tarballs rather than just telling people to get it from git.
make-dist already builds the dashboard frontend completely and includes the resulting npm output in the tarball.
The build logic for the frontend seems to be duplicated in src/pybind/mgr/dashboard/CMakeLists.txt, I haven't checked the details but it should not run pip/npm install when building the tarball. If it does, it seems easy to fix.
Patrick: can you confirm whether your tarball build is fetching files (with a log, if so)? As Paul says, the tarball is supposed to include a pre-built dashboard frontend.
The tarball is definitely fetching files, I have attached a log of the relevant part (I can't attach the full log due to file size restrictions). I am running the build in an environment without network access.
If I patch out the pip install calls, it still calls npm install, which also uses internet.
Patrick McLean wrote:
The tarball is definitely fetching files, I have attached a log of the relevant part (I can't attach the full log due to file size restrictions). I am running the
build in an environment without network access.
If I patch out the pip install calls, it still calls npm install, which also uses internet.
Please try building from tarball by passing
-DWITH_MGR_DASHBOARD_FRONTEND=OFF
to CMake - this should prevent the re-building of the WebUI.
Some updates regarding npm:
- We have added "package-lock.json" to git, this way each version of ceph will always install the same version of npm packages
- We have removed phantomJS
- Since we removed phantomJS, I don't think its a priority to separate dev packages from test packages
Tiago Melo wrote:
Some updates regarding npm:
- We have added "package-lock.json" to git, this way each version of ceph will always install the same version of npm packages
- We have removed phantomJS
- Since we removed phantomJS, I don't think its a priority to separate dev packages from test packages
Can we close this issue then?
- Status changed from New to Won't Fix
Resolving as "Won't fix" - when using an officially released tarball, the dashboard assets are shipped in a pre-built form that does not require downloading any additional packages (e.g. when building RPMs)
- Project changed from mgr to Dashboard
- Category changed from 132 to General
Also available in: Atom
PDF