Bug #24466
closedbuild process runs "pip install" and "npm install" while building the mgr frontend
0%
Description
The current build process runs "pip install" and "npm install" while building the mgr frontend. This is a violation of most distros' packaging policies, but more importantly this makes the frontend code someone is running be non-deterministic. The code that someone is running will depend on when they built ceph, not just the version they are running. This will quite likely complicate triaging and debugging of any issues that crop up.
I would suggest just listing python dependencies in the global dependency list, and bundling node dependencies in the distributed tarball, that way you can be sure that everyone who builds from the source tarball is actually running the same code.
I would also suggest separating out test dependencies from build dependencies in the npm packages.json, since the test dependencies download precompiled binaries for phantomjs (which is abandoned and vulnerable according to it's website) and other things.
Files
Updated by Paul Emmerich almost 6 years ago
It also fails for various reasons on arm64 such as binaries not existing.
The node dependency issues are fixable by adding package-lock.json to git (which is the whole purpose of that file that was added in npm 5).
Updated by Patrick McLean almost 6 years ago
The package lock might help with the dependency drift, but it does not change the fact that the build system is fetching random files off the internet halfway through the build. Everything that is needed for the build should be included in the tarball, that is generally the purpose of providing tarballs rather than just telling people to get it from git.
Updated by Nathan Cutler almost 6 years ago
- Project changed from Ceph to mgr
- Category changed from build to 132
Updated by Paul Emmerich almost 6 years ago
Patrick McLean wrote:
The package lock might help with the dependency drift, but it does not change the fact that the build system is fetching random files off the internet halfway through the build. Everything that is needed for the build should be included in the tarball, that is generally the purpose of providing tarballs rather than just telling people to get it from git.
make-dist already builds the dashboard frontend completely and includes the resulting npm output in the tarball.
The build logic for the frontend seems to be duplicated in src/pybind/mgr/dashboard/CMakeLists.txt, I haven't checked the details but it should not run pip/npm install when building the tarball. If it does, it seems easy to fix.
Updated by John Spray almost 6 years ago
Patrick: can you confirm whether your tarball build is fetching files (with a log, if so)? As Paul says, the tarball is supposed to include a pre-built dashboard frontend.
Updated by Patrick McLean almost 6 years ago
- File ceph-pip.log ceph-pip.log added
The tarball is definitely fetching files, I have attached a log of the relevant part (I can't attach the full log due to file size restrictions). I am running the build in an environment without network access.
If I patch out the pip install calls, it still calls npm install, which also uses internet.
Updated by Lenz Grimmer almost 6 years ago
Patrick McLean wrote:
The tarball is definitely fetching files, I have attached a log of the relevant part (I can't attach the full log due to file size restrictions). I am running the
build in an environment without network access.If I patch out the pip install calls, it still calls npm install, which also uses internet.
Please try building from tarball by passing
-DWITH_MGR_DASHBOARD_FRONTEND=OFFto CMake - this should prevent the re-building of the WebUI.
Updated by Tiago Melo about 5 years ago
- We have added "package-lock.json" to git, this way each version of ceph will always install the same version of npm packages
- We have removed phantomJS
- Since we removed phantomJS, I don't think its a priority to separate dev packages from test packages
Updated by Sebastian Wagner almost 5 years ago
Tiago Melo wrote:
Some updates regarding npm:
- We have added "package-lock.json" to git, this way each version of ceph will always install the same version of npm packages
- We have removed phantomJS
- Since we removed phantomJS, I don't think its a priority to separate dev packages from test packages
Can we close this issue then?
Updated by Lenz Grimmer about 4 years ago
- Status changed from New to Won't Fix
Resolving as "Won't fix" - when using an officially released tarball, the dashboard assets are shipped in a pre-built form that does not require downloading any additional packages (e.g. when building RPMs)
Updated by Ernesto Puerta about 3 years ago
- Project changed from mgr to Dashboard
- Category changed from 132 to General