CI

Hey Jason, I went through the BZ details and couldn't tell what needs to change in createrepo. You mentioned that a newer version might work fine, this is the version we have in our builders:

$ createrepo --version
createrepo 0.10.3

More information about the package (these are Ubuntu boxes):

Package: createrepo
Priority: optional
Section: universe/admin
Installed-Size: 207
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Mike Miller <mtmiller@debian.org>
Architecture: all
Version: 0.10.3-1
Depends: python (>= 2.7), python (<< 2.8), python:any (>= 2.7.1-0ubuntu2), deltarpm, python-deltarpm, python-libxml2, python-lzma, python-rpm, python-sqlitecachec, rpm (>= 4.1.1), yum (>= 3.2.29)
Filename: pool/universe/c/createrepo/createrepo_0.10.3-1_all.deb
Size: 34978
MD5sum: aba92ac685e2b8a7381f42d94aea36b7
SHA1: 5db58348c6d1e0ef83fb393c9b610b9c7cc83412
SHA256: fdb8f04f6f11f6a4a2f215c7261351f5d261b8e416be425a2151c016a4d8e876
Description-en: tool to generate the metadata for a yum repository
 The createrepo tool generates the repodata directory and XML metadata that
 makes up a repository of RPM packages. This repository format is supported
 by apt-rpm, red-carpet(zen), smartpm, up2date, yast, and yum.
 .
 This package is similar to the apt-ftparchive or reprepro commands, but for
 working with RPM repositories.
Description-md5: 662c2a56f1fd73a334d887f76205f2f0
Homepage: http://createrepo.baseurl.org/
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu

@Alfredo: apparently the fix (or at least one of them) was in yum-metadata-parser, but it doesn't look like the Ubuntu version depends on it. What release of Ubuntu (and architecture) are you using for this environment? I will see if I can re-create it in a test. Just using Python to manipulate the "corrupt" sqlite DB, I can properly set the size to a long integer so it might be a bug w/ the sqlite Python packages in that OS?

Ubuntu Xenial.

Looking through the BZ it seems that this is code in yum-metadata-parser that does the logic wrong, I doubt this is Python (python has builtin support for sqlite, not a library)

@Alfredo: looks like 'yum-metadata-parser' is actually 'python-sqlitecachec' on Ubuntu and it's the source of the corruption. The XML files are properly generated, and then 'python-sqlitecachec' reads the XMLs and generates the corrupt sqlite DB entries since it's using C bindings of sqlite under Python. Since it looks like python-sqlitecachec-1.1.4-1 is the latest available under Ubuntu, I guess we will need a custom-built version that actually pulls in the latest fixes [1].

[1] https://github.com/rpm-software-management/yum-metadata-parser/commits/master

... could also try running "createrepo" w/ the "--no-database" option to only generate the XML files and not the sqlite DB cache of the XML.

can you confirm if that flags makes it work? alternatively, if you have a bionic box vm to try out and verify it works we might just be forced to update and take the hit

@Alfredo: Adding "--no-database" made it work for my local "file://" repo. I don't know how it would scale for the lab, but perhaps someone can just delete the sqlite repodata files out of one build and we can kick off a small suite.