Ceph

Upstream snappy has rejected my PR to re-enable RTTI (see https://github.com/google/snappy/pull/144#issuecomment-968371042)

This means we're stuck with either:

1) Somehow figuring out how to build at least the compression plugin bits of ceph without RTTI? Maybe? I've no idea if this is possible/viable/sensible.

2) Getting patches into the various linux distros to re-enable RTTI for snappy versions >= 1.1.9.

It turns out we had the exact same issue with leveldb - upstream disabled RTTI in leveldb 1.23 (see https://github.com/google/leveldb/releases/tag/1.23), and openSUSE, Fedora, Arch (and I assume others, I only checked those three quickly) have since added patches to the downstream leveldb packages to re-enable RTTI. On the assumption this was going to be the way to go, I've got the appropriate change into openSUSE for snappy: https://build.opensuse.org/request/show/929633. I don't know what the procedure is for other distros offhand, but hopefully this information is helpful for whoever's able to take on that work.

In the past, the snappy's folks had changed a type in their API without even bumping a major. Now, they refuse to merge a mini change to the building code that simply allows us, and others, to dlopen() the library. It looks like they are open source for advertising reasons only but still code like they work for MS.

Best move: stop using snappy...

Yeah, but Ubuntu (one of the most popular distros) just ignores my ticket:

https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/1998636

We now have Ceph that's completely broken with default packages for over half a year on Ubuntu :(
It's crazy. They advertise Ceph on Ubuntu (https://ubuntu.com/ceph) and then they leave it broken for months and nobody cares.
I know the Ceph folks can't do much about it but overall this is really a lowlight.
I wonder how I can get out of this situation because it gets particularly ugly to pin the APT package for so long.
Is there a way to easily switch off Bluestore compression? Is there any disadvatange of doing so?

I hadn't even thought to look up Ubuntu, I just thought they'd pull in Debian's change.

(We do our testing against Ubuntu LTS and it looks like Jammy's version of the package is before the RTTI change.)

I wonder if you could file a bug against Snappy directly.

It looks like there are bluestore options for setting whether to use compression and which algorithm to use, but they can all be overridden by per-pool settings.

I'm on the RGW team, so I don't feel qualified to say too much, but in the upstream call when we discussed this, someone on the bluestore team mentioned they had a way to migrate between compression algorithm. So you might want to ask in irc/slack or on the mailing list for more on how.

It looks like lunar lobster has the fixed version of snappy, at least its changelog mentions re-enabling RTTI.

Did you have a chance to see if Lunar Lobster works? From what I can tell it seems that Jammy and Lunar should have a working libnspapy and only Kinetic has the version without RTTI.

@Adam DC949 Emerson: Thanks you so much! Really helpful. I can confirm, Lunar Lobster works. In case anyone is wondering: You don't need to make it work before upgrading to Lunar. If you upgraded to Kinetic and you see that error message, just do another "do-release-upgrade" to upgrade to Lunar and things will magically work again.

Nevertheless, Canonical didn't give a good impression on this at all. No reaction whatsoever; a whole supported(!) distro version for 6(!) months with completely broken Ceph support; leaving users after an upgrade without any explanation why suddenly their Ceph installation doesn't work; finally fixing it silently without any announcement and without any reaction to my bug ticket there => Not very professional, not funny :-(