Ceph » Linux kernel client

[Edit] https://pulpito.ceph.com/vshankar-2024-02-27_04:05:06-fs-wip-vshankar-testing-20240226.124304-testing-default-smithi/7574246/

[22559.788266] libceph: osd5 (2)172.21.15.123:6816 protocol error, bad epilogue^M
[22559.796022] libceph: mon0 (2)172.21.15.3:3300 session established^M
[22559.797179] libceph: prepare_sparse_read_cont: ret 0x1000 total_resid 0x0 resid 0x0^M
[22559.809937] libceph: osd5 (2)172.21.15.123:6816 read processing error^M
[22559.817265] libceph: mon0 (2)172.21.15.3:3300 session established^M
[22559.824822] libceph: bad late_status 0x1^M
[22559.828856] libceph: osd5 (2)172.21.15.123:6816 protocol error, bad epilogue^M
[22559.836819] libceph: mon0 (2)172.21.15.3:3300 session established^M
[22559.844404] libceph: prepare_sparse_read_cont: ret 0x1000 total_resid 0x0 resid 0x0^M
[22559.852212] libceph: osd5 (2)172.21.15.123:6816 read processing error^M
[22559.859695] libceph: mon0 (2)172.21.15.3:3300 session established^M
[22559.860766] libceph: bad late_status 0x1^M
[22559.869820] libceph: osd5 (2)172.21.15.123:6816 protocol error, bad epilogue^M
[22559.877715] libceph: mon0 (2)172.21.15.3:3300 session established^M
[22559.878964] libceph: prepare_sparse_read_cont: ret 0x1000 total_resid 0x0 resid 0x0^M
[22559.891578] libceph: osd5 (2)172.21.15.123:6816 read processing error^M
[22559.898896] libceph: mon0 (2)172.21.15.3:3300 session established^M
[22559.900353] libceph: bad late_status 0x1^M
[22559.909087] libceph: osd5 (2)172.21.15.123:6816 protocol error, bad epilogue^M
[22559.917071] libceph: mon0 (2)172.21.15.3:3300 session established^M
[22559.924877] libceph: prepare_sparse_read_cont: ret 0x1000 total_resid 0x0 resid 0x0^M
[22559.932645] libceph: osd5 (2)172.21.15.123:6816 read processing error^M
[22559.939957] libceph: mon0 (2)172.21.15.3:3300 session established^M
[22559.941506] libceph: bad late_status 0x1^M
[22559.950120] libceph: osd5 (2)172.21.15.123:6816 protocol error, bad epilogue^M
[22559.958116] libceph: mon0 (2)172.21.15.3:3300 session established^M
[22559.959112] libceph: prepare_sparse_read_cont: ret 0x1000 total_resid 0x0 resid 0x0^M
[22559.971974] libceph: osd5 (2)172.21.15.123:6816 read processing error^M
[22559.979145] libceph: mon0 (2)172.21.15.3:3300 session established^M
[22559.980394] libceph: bad late_status 0x1^M

For the messager V2 test for fscrypt we need to backport https://tracker.ceph.com/issues/59195.

I want to answer a question from Ilya about whether we should be catching this in upstream QA. The answer is, we should except there are build failures right now with xfstests caused by the switch to centos9. So we are unlucky. Here is one job where we would have likely caught this bug:

https://pulpito.ceph.com/vshankar-2024-02-27_04:05:06-fs-wip-vshankar-testing-20240226.124304-testing-default-smithi/7574294/

which uses the kclient (stock kernel) and ms_mode/crc

Patrick Donnelly wrote:

Here is one job where we would have likely caught this bug:

https://pulpito.ceph.com/vshankar-2024-02-27_04:05:06-fs-wip-vshankar-testing-20240226.124304-testing-default-smithi/7574294/

which uses the kclient (stock kernel) and ms_mode/crc

This one wouldn't have caught it because it just wanted to run fsx -- 5-workunit/suites/fsx boils down to:

OPTIONS="-z"  # don't use zero range calls; not supported by cephfs

./fsx $OPTIONS  1MB -N 50000 -p 10000 -l 1048576
./fsx $OPTIONS  10MB -N 50000 -p 10000 -l 10485760
./fsx $OPTIONS 100MB -N 50000 -p 10000 -l 104857600

For completeness, it would be nice to link a job that wanted to run xfstests themselves here, and double check that the workunit in question wouldn't have excluded generic/580 for whatever reason.