Project

General

Profile

Bug #4890

ceph-deploy: install fails on RHEL 6.3

Added by Tamilarasi muthamizhan almost 11 years ago. Updated almost 11 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
-
Category:
ceph-deploy
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

this is the error seen, when trying to install ceph on RHEL machines

tamil@tamil-VirtualBox:~/rhel/ceph-deploy$ ./ceph-deploy install --dev=next burnupi27 burnupi28
http://gitbuilder.ceph.com/ceph-rpm-centos6-x86_64-basic/ref/next/x86_64/ceph-0.60-851.gfdbab85.el6.x86_64.rpm: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404" ##################################################
Traceback (most recent call last):
File "./ceph-deploy", line 9, in <module>
load_entry_point('ceph-deploy==0.1', 'console_scripts', 'ceph-deploy')()
File "/home/tamil/rhel/ceph-deploy/ceph_deploy/cli.py", line 112, in main
return args.func(args)
File "/home/tamil/rhel/ceph-deploy/ceph_deploy/install.py", line 215, in install
version=version,
File "/home/tamil/rhel/ceph-deploy/virtualenv/local/lib/python2.7/site-packages/pushy-0.5.1-py2.7.egg/pushy/protocol/proxy.py", line 255, in <lambda>
(conn.operator(type_, self, args, kwargs))
File "/home/tamil/rhel/ceph-deploy/virtualenv/local/lib/python2.7/site-packages/pushy-0.5.1-py2.7.egg/pushy/protocol/connection.py", line 66, in operator
return self.send_request(type_, (object, args, kwargs))
File "/home/tamil/rhel/ceph-deploy/virtualenv/local/lib/python2.7/site-packages/pushy-0.5.1-py2.7.egg/pushy/protocol/baseconnection.py", line 323, in send_request
return self.__handle(m)
File "/home/tamil/rhel/ceph-deploy/virtualenv/local/lib/python2.7/site-packages/pushy-0.5.1-py2.7.egg/pushy/protocol/baseconnection.py", line 639, in __handle
raise e
pushy.protocol.proxy.ExceptionProxyTrying other mirror.
http://gitbuilder.ceph.com/ceph-rpm-centos6-x86_64-basic/ref/next/x86_64/libcephfs1-0.60-851.gfdbab85.el6.x86_64.rpm: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404"
Trying other mirror.
http://gitbuilder.ceph.com/ceph-rpm-centos6-x86_64-basic/ref/next/x86_64/librados2-0.60-851.gfdbab85.el6.x86_64.rpm: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404"
Trying other mirror.
http://gitbuilder.ceph.com/ceph-rpm-centos6-x86_64-basic/ref/next/x86_64/librbd1-0.60-851.gfdbab85.el6.x86_64.rpm: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404"
Trying other mirror.

Error Downloading Packages:
librbd1-0.60-851.gfdbab85.el6.x86_64: failure: librbd1-0.60-851.gfdbab85.el6.x86_64.rpm from Ceph: [Errno 256] No more mirrors to try.
librados2-0.60-851.gfdbab85.el6.x86_64: failure: librados2-0.60-851.gfdbab85.el6.x86_64.rpm from Ceph: [Errno 256] No more mirrors to try.
ceph-0.60-851.gfdbab85.el6.x86_64: failure: ceph-0.60-851.gfdbab85.el6.x86_64.rpm from Ceph: [Errno 256] No more mirrors to try.
libcephfs1-0.60-851.gfdbab85.el6.x86_64: failure: libcephfs1-0.60-851.gfdbab85.el6.x86_64.rpm from Ceph: [Errno 256] No more mirrors to try.

##################################################
: Command '['yum', '-y', '-q', 'install', 'ceph', 'ceph-common', 'ceph-fs-common']' returned non-zero exit status 1

History

#1 Updated by Anonymous almost 11 years ago

This error was due to cached metadata on the target systems. ceph-deploy is attempting to install version 0.60-851, while what is out on the gitbuilder server at this moment in next is version 0.60-784, so yum doesn't find the packages it's looking for.

Some previous install on that system caused the metadata for version 0.60-851 to be cached on the system, probably an install of master since it is currently at that release. Yum is attempting to reuse that metadata since it doesn't know that it is no longer valid for the repo. The can occur both for a switch of the installation sources between branches, and occasionally when gitbuilder pushes out a new release.

The solution is to clear the yum metatdata when cleaning up. The yum clean command only works for currently enabled repos, so it needs to be done before removing the ceph-release file:

sudo yum clean all
sudo rpm -e ceph-release-1-0.el6.noarch

If a problem like this occurs and the ceph-release package is not installed (or a ceph repo is not manually configured), then remove /var/cache/yum/x86_64/6Server/{Ceph,Ceph-noarch,ceph-source}. That path may be slightly different on different systems.

After clean up installation of next branch worked ok. I didn't test more than just the installation. Transcript below:

[ubuntu@gary-centos-01 ceph-deploy]$ ./ceph-deploy new burnupi27 burnupi28
[ubuntu@gary-centos-01 ceph-deploy]$ ./ceph-deploy install --dev=next burnupi27 burnupi28 ##################################################
Warning: RPMDB altered outside of yum. ##################################################
[ubuntu@gary-centos-01 ceph-deploy]$

[ubuntu@gary-centos-01 ceph-deploy]$ ssh burnupi27
Last login: Thu May 2 01:18:30 2013 from gary-centos-01.front.sepia.ceph.com
[ubuntu@burnupi27 ~]$ rpm -qa | egrep "ceph|rbd|rad"
librados2-0.60-784.gc194151.el6.x86_64
ceph-release-1-0.el6.noarch
librbd1-0.60-784.gc194151.el6.x86_64
ceph-0.60-784.gc194151.el6.x86_64
libcephfs1-0.60-784.gc194151.el6.x86_64
[ubuntu@burnupi27 ~]$
[ubuntu@burnupi27 ~]$ exit
logout
Connection to burnupi27 closed.

[ubuntu@gary-centos-01 ceph-deploy]$ ssh burnupi28
Last login: Thu May 2 01:18:40 2013 from gary-centos-01.front.sepia.ceph.com
[ubuntu@burnupi28 ~]$ rpm -qa | egrep "ceph|rbd|rad"
librbd1-0.60-784.gc194151.el6.x86_64
librados2-0.60-784.gc194151.el6.x86_64
libcephfs1-0.60-784.gc194151.el6.x86_64
ceph-release-1-0.el6.noarch
ceph-0.60-784.gc194151.el6.x86_64
[ubuntu@burnupi28 ~]$ exit
logout
Connection to burnupi28 closed.

#2 Updated by Anonymous almost 11 years ago

  • Status changed from New to Resolved

BTW, I've left burnupi27/28 with ceph installed. There is a ~ubuntu/remove.sh script on both systems that does the clean up I mentioned above.

#3 Updated by Sage Weil almost 11 years ago

  • Status changed from Resolved to In Progress
  • Assignee changed from Anonymous to Anonymous

Warren, sounds like we should do 'yum clean all' in three places:

- right after we install the yum source rpm (in install.py)
- right before we remove it (in install.py cleanup)
- right before we remove extraneous src packages in nuke.py

If we have to do something manually on the machine we have failed. Any kick in the pants we might deliver should be done by nuke, and (if possible) done preemptively in the install or cleanup to avoid nuke in the first place... we should only need nuke if a job was interrupted at an inopportune time (like a machine reboot)

#4 Updated by Sage Weil almost 11 years ago

  • Assignee deleted (Anonymous)

whoops, i thought this was the teuthology side.

Gary, what do you think ceph-deploy should do? Install the release package, 'yum clean all', and then 'yum update'?

#5 Updated by Sage Weil almost 11 years ago

  • Status changed from In Progress to Resolved

after some discussion we think we should not do anything drastic. customers are unlikely to see this, and the root issue was teuthology's install.py.

#6 Updated by Anonymous almost 11 years ago

It looks like it possible to clean a specific repo cache by:

yum clean all --enablerepo=<repo> --disablerepo="*"

I'll verify this and see if I can add it to the package remove script so that removing the ceph-release rpm will also remove it's metadata.

Also available in: Atom PDF