Bug #39387
ceph-debuginfo on CentOS cannot be installed
0%
Description
It appears that the repodata for CentOS 7 builds of ceph-debuginfo are returning large negative size metadata for the 'ceph-debuginfo' package now that its size is over 2GiB. This is resulting in all RBD 'memcheck' tests failing since it fails to install 'ceph-debuginfo'. I manually downloaded the contents of the repo and ran "createrepo" against it and was able to install 'ceph-debuginfo' from my local repo. I wonder if the shaman/chacra nodes are using an older version that has a 32 signed number issue like [1].
See https://3.chacra.ceph.com/r/ceph/master/be4d819013565e74d354199bdb7bc23e1aaa4399/centos/7/flavors/notcmalloc/x86_64/repodata/
See http://qa-proxy.ceph.com/teuthology/teuthology-2019-04-13_02:01:03-rbd-master-distro-basic-smithi/3840046/teuthology.log
History
#1 Updated by Alfredo Deza almost 5 years ago
Hey Jason, I went through the BZ details and couldn't tell what needs to change in createrepo. You mentioned that a newer version might work fine, this is the version we have in our builders:
$ createrepo --version createrepo 0.10.3
More information about the package (these are Ubuntu boxes):
Package: createrepo Priority: optional Section: universe/admin Installed-Size: 207 Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com> Original-Maintainer: Mike Miller <mtmiller@debian.org> Architecture: all Version: 0.10.3-1 Depends: python (>= 2.7), python (<< 2.8), python:any (>= 2.7.1-0ubuntu2), deltarpm, python-deltarpm, python-libxml2, python-lzma, python-rpm, python-sqlitecachec, rpm (>= 4.1.1), yum (>= 3.2.29) Filename: pool/universe/c/createrepo/createrepo_0.10.3-1_all.deb Size: 34978 MD5sum: aba92ac685e2b8a7381f42d94aea36b7 SHA1: 5db58348c6d1e0ef83fb393c9b610b9c7cc83412 SHA256: fdb8f04f6f11f6a4a2f215c7261351f5d261b8e416be425a2151c016a4d8e876 Description-en: tool to generate the metadata for a yum repository The createrepo tool generates the repodata directory and XML metadata that makes up a repository of RPM packages. This repository format is supported by apt-rpm, red-carpet(zen), smartpm, up2date, yast, and yum. . This package is similar to the apt-ftparchive or reprepro commands, but for working with RPM repositories. Description-md5: 662c2a56f1fd73a334d887f76205f2f0 Homepage: http://createrepo.baseurl.org/ Bugs: https://bugs.launchpad.net/ubuntu/+filebug Origin: Ubuntu
#2 Updated by Jason Dillaman almost 5 years ago
@Alfredo: apparently the fix (or at least one of them) was in yum-metadata-parser, but it doesn't look like the Ubuntu version depends on it. What release of Ubuntu (and architecture) are you using for this environment? I will see if I can re-create it in a test. Just using Python to manipulate the "corrupt" sqlite DB, I can properly set the size to a long integer so it might be a bug w/ the sqlite Python packages in that OS?
#3 Updated by Alfredo Deza almost 5 years ago
Ubuntu Xenial.
Looking through the BZ it seems that this is code in yum-metadata-parser that does the logic wrong, I doubt this is Python (python has builtin support for sqlite, not a library)
#4 Updated by Jason Dillaman almost 5 years ago
@Alfredo: looks like 'yum-metadata-parser' is actually 'python-sqlitecachec' on Ubuntu and it's the source of the corruption. The XML files are properly generated, and then 'python-sqlitecachec' reads the XMLs and generates the corrupt sqlite DB entries since it's using C bindings of sqlite under Python. Since it looks like python-sqlitecachec-1.1.4-1 is the latest available under Ubuntu, I guess we will need a custom-built version that actually pulls in the latest fixes [1].
[1] https://github.com/rpm-software-management/yum-metadata-parser/commits/master
#5 Updated by Jason Dillaman almost 5 years ago
... could also try running "createrepo" w/ the "--no-database" option to only generate the XML files and not the sqlite DB cache of the XML.
#6 Updated by Alfredo Deza almost 5 years ago
can you confirm if that flags makes it work? alternatively, if you have a bionic box vm to try out and verify it works we might just be forced to update and take the hit
#7 Updated by Jason Dillaman almost 5 years ago
@Alfredo: Adding "--no-database" made it work for my local "file://" repo. I don't know how it would scale for the lab, but perhaps someone can just delete the sqlite repodata files out of one build and we can kick off a small suite.
#8 Updated by Alfredo Deza over 4 years ago
- Status changed from New to In Progress
This was deployed and everything went great, but no actual PR went out to fix it. I'll follow up on that
#9 Updated by Alfredo Deza over 4 years ago
- Status changed from In Progress to Fix Under Review
Master PR https://github.com/ceph/chacra/pull/266
#10 Updated by Alfredo Deza over 4 years ago
- Status changed from Fix Under Review to Closed