Ceph

so we have two issues here:

1) we should check if we really populate all required wheels in install-deps.sh instead of checking for the wheelhouse directory
2) find a pypi mirror in mainland china. and see if our script can be configured to use it.

I agree. The current script kind of assumes it will exit on error if one of the dependencies fails to be downloaded. But it appears to behave differently.

i think the script did exit on error, but it leaved us with a "wheelhouse" directory not fully populated with required packages. but we took the "wheelhouse" directory as a green light for tox. in Xinze's case, the "wheelhouse" directory was empty: he failed to download any packages.

(install-deps-python2.7)[root@ceph-code-server ceph-detect-init]# sh run-tox.sh
New python executable in make-check/bin/python
Installing Setuptools..............................................................................................................................................................................................................................done.
Installing Pip.....................................................................................................................................................................................................................................................................................................................................done.
Downloading/unpacking pip>=6.1
  Downloading pip-7.0.3.tar.gz (1.1MB): 736kB downloaded
  Hash of the package https://pypi.python.org/packages/source/p/pip/pip-7.0.3.tar.gz#md5=54cbf5ae000fb3af3367345f5d299d1c (from https://pypi.python.org/simple/pip/) (45f57faaad44593bf30effd2915ac3bd) doesn't match the expected hash 54cbf5ae000fb3af3367345f5d299d1c!
Cleaning up...
Bad md5 hash for package https://pypi.python.org/packages/source/p/pip/pip-7.0.3.tar.gz#md5=54cbf5ae000fb3af3367345f5d299d1c (from https://pypi.python.org/simple/pip/)
Storing complete log in /root/.pip/pip.log
Traceback (most recent call last):
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/bin/pip", line 9, in <module>
    load_entry_point('pip==1.4.1', 'console_scripts', 'pip')()
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/__init__.py", line 148, in main
    return command.main(args[1:], options)
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/basecommand.py", line 176, in main
    logger.fatal('Storing complete log in %s' % log_fn)
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/log.py", line 48, in fatal
    self.log(self.FATAL, msg, *args, **kw)
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/log.py", line 75, in log
    backwardcompat.fwrite(consumer, rendered)
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/backwardcompat/__init__.py", line 90, in fwrite
    f.write(s)
ValueError: I/O operation on closed file
Ignoring indexes: https://pypi.python.org/simple/
Downloading/unpacking distribute
Unpacking ./wheelhouse/setuptools-17.0-py2.py3-none-any.whl
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/commands/install.py", line 241, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/req.py", line 1256, in install
    if req.name == 'distribute' and req.installed_version in distribute_req:
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/req.py", line 390, in installed_version
    return self.pkg_info()['version']
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/req.py", line 357, in pkg_info
    data = self.egg_info_data('PKG-INFO')
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/req.py", line 293, in egg_info_data
    filename = self.egg_info_path(filename)
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/req.py", line 307, in egg_info_path
    filenames = os.listdir(base)
OSError: [Errno 2] No such file or directory: '/home/chixinze/git/ceph/src/ceph-detect-init/make-check/build/distribute/pip-egg-info'

Storing complete log in /root/.pip/pip.log
Traceback (most recent call last):
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/bin/pip", line 9, in <module>
    load_entry_point('pip==1.4.1', 'console_scripts', 'pip')()
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/__init__.py", line 148, in main
    return command.main(args[1:], options)
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/basecommand.py", line 176, in main
    logger.fatal('Storing complete log in %s' % log_fn)
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/log.py", line 48, in fatal
    self.log(self.FATAL, msg, *args, **kw)
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/log.py", line 75, in log
    backwardcompat.fwrite(consumer, rendered)
  File "/home/chixinze/git/ceph/src/ceph-detect-init/make-check/lib/python2.7/site-packages/pip/backwardcompat/__init__.py", line 90, in fwrite
    f.write(s)
ValueError: I/O operation on closed file
Ignoring indexes: https://pypi.python.org/simple/
Downloading/unpacking tox>=1.9
Downloading/unpacking pluggy>=0.3.0,<0.4.0 (from tox>=1.9)
Downloading/unpacking virtualenv>=1.11.2 (from tox>=1.9)
Downloading/unpacking py>=1.4.17 (from tox>=1.9)
Installing collected packages: tox, pluggy, virtualenv, py
Successfully installed tox pluggy virtualenv py
Cleaning up...
run-tox.sh: line 33: tox: command not found

@Kefu Chai as you said

find a pypi mirror in mainland china. and see if our script can be configured to use it"

I think if we can find some deps mirrors in mainland china inclduing pypi mirror, it would be more nice for our chinese developers.

Xinze Chi wrote:

[...]

@Kefu Chai as you said

find a pypi mirror in mainland china. and see if our script can be configured to use it"

I think if we can find some deps mirrors in mainland china inclduing pypi mirror, it would be more nice for our chinese developers.

for mirrors for pypi in china, please see http://segmentfault.com/a/1190000000375848#articleHeader2

to configure your pip to use one of them, see https://pip.pypa.io/en/latest/user_guide.html#configuration

for example.

[global]

index-url = https://pypi.mirrors.ustc.edu.cn/simple