Project

General

Profile

Pacific » History » Version 131

Kotresh Hiremath Ravishankar, 12/29/2023 03:26 PM

1 1 Patrick Donnelly
h1. Pacific
2
3 4 Patrick Donnelly
h2. On-call Schedule
4
5 98 Venky Shankar
* Jul: Venky
6
* Aug: Patrick
7 118 Jos Collin
* Sep: Jos Collin
8 98 Venky Shankar
* Oct: Xiubo
9
* Nov: Rishabh
10
* Dec: Kotresh
11
* Jan: Milind
12 81 Venky Shankar
13 1 Patrick Donnelly
h2. Reviews
14 85 Rishabh Dave
15 119 Rishabh Dave
h3. ADD NEW ENTRY BELOW
16
17 131 Kotresh Hiremath Ravishankar
h3. 29 Dec 2023
18
19
https://pulpito.ceph.com/yuriw-2023-12-28_15:45:00-fs-wip-yuri7-testing-2023-12-27-1008-pacific-distro-default-smithi/
20
21
* https://tracker.ceph.com/issues/63212
22
  qa: failed to download ior.tbz2
23
* https://tracker.ceph.com/issues/52624
24
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
25
* https://tracker.ceph.com/issues/62501
26
  pacific(?): qa: mgr-osd-full causes OSD aborts due to ENOSPC (fs/full/subvolume_snapshot_rm.sh)
27
* https://tracker.ceph.com/issues/50224
28
  test_mirroring_init_failure_with_recovery - (tasks.cephfs.test_mirroring.TestMirroring) 
29
* https://tracker.ceph.com/issues/58992
30
  test_acls (tasks.cephfs.test_acls.TestACLs) 
31
* https://tracker.ceph.com/issues/50223
32
  qa: "client.4737 isn't responding to mclientcaps(revoke)"
33
* https://tracker.ceph.com/issues/63907
34
  cephfs-mirror: Mirror::update_fs_mirrors crashes while taking lock (coredump)
35
36 128 Venky Shankar
h3. 21 Dec 2023
37
38
https://pulpito.ceph.com/?branch=wip-yuri5-testing-2023-12-14-1107-pacific
39
40
* https://tracker.ceph.com/issues/52624
41
    cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
42
* https://tracker.ceph.com/issues/63539
43
    fs/full/subvolume_clone.sh: Health check failed: 1 full osd(s) (OSD_FULL)
44
* test_acls failed because known distro wasn't detected
45 129 Venky Shankar
* https://tracker.ceph.com/issues/50223
46
    qa: "client.4737 isn't responding to mclientcaps(revoke)"
47 128 Venky Shankar
48 127 Rishabh Dave
h3. 4 Dec 2023 2
49 126 Rishabh Dave
50
https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/
51
52
* ior/mdtest failures because packages were missing from download.ceph.com
53
* test_acls failed because known distro wasn't detected
54
* https://tracker.ceph.com/issues/63539
55
  fs/full/subvolume_clone.sh: Health check failed: 1 full osd(s) (OSD_FULL)
56
* https://tracker.ceph.com/issues/52624
57
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
58
* https://tracker.ceph.com/issues/37808
59
  osd: osdmap cache weak_refs assert during shutdown
60 125 Kotresh Hiremath Ravishankar
61
h3. 04 Dec 2023
62
63
https://pulpito.ceph.com/yuriw-2023-11-30_15:46:13-fs-wip-yuri8-testing-2023-11-29-0706-pacific-distro-default-smithi/
64
65
* ior package was missing because it was deleted from download.ceph.com
66
* Error re-imaging machines:
67
  https://pulpito.ceph.com/yuriw-2023-11-30_15:46:13-fs-wip-yuri8-testing-2023-11-29-0706-pacific-distro-default-smithi/7472083
68
  https://pulpito.ceph.com/yuriw-2023-11-30_15:46:13-fs-wip-yuri8-testing-2023-11-29-0706-pacific-distro-default-smithi/7472084
69
  https://pulpito.ceph.com/yuriw-2023-11-30_15:46:13-fs-wip-yuri8-testing-2023-11-29-0706-pacific-distro-default-smithi/7472104
70
* https://tracker.ceph.com/issues/52624
71
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
72
* https://tracker.ceph.com/issues/51282
73
  cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log
74
* https://tracker.ceph.com/issues/62501
75
  pacific(?): qa: mgr-osd-full causes OSD aborts due to ENOSPC (fs/full/subvolume_snapshot_rm.sh)
76
* https://tracker.ceph.com/issues/58992
77
  test_acls (tasks.cephfs.test_acls.TestACLs)
78
* https://tracker.ceph.com/issues/54462
79
  Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi055 with status 128
80
81
82 123 Rishabh Dave
h3. 22 Nov 2023
83
84
https://pulpito.ceph.com/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/
85 124 Rishabh Dave
https://pulpito.ceph.com/yuriw-2023-11-21_15:54:56-smoke-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/
86 123 Rishabh Dave
87
* https://tracker.ceph.com/issues/62501
88
  pacific(?): qa: mgr-osd-full causes OSD aborts due to ENOSPC (fs/full/subvolume_snapshot_rm.sh)
89
* https://tracker.ceph.com/issues/50223
90
  qa: "client.4737 isn't responding to mclientcaps(revoke)"
91
* ior/mdtest failures because packages were missing from download.ceph.com
92
* test_acls failed because known distro wasn't detected
93
* https://tracker.ceph.com/issues/52624
94
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
95
96 122 Rishabh Dave
h3. 15 Nov 2023
97
98
https://pulpito.ceph.com/yuriw-2023-10-24_00:02:48-fs-wip-yuri7-testing-2023-10-23-1230-pacific-distro-default-smithi/
99
100
* https://tracker.ceph.com/issues/63539
101
  fs/full/subvolume_clone.sh: Health check failed: 1 full osd(s) (OSD_FULL)
102
* test_acls: distro name for RHEL 8.4 wasn't recognized one by xfstests_dev.py
103
* ior package was missing because it was deleted from download.ceph.com
104
* https://tracker.ceph.com/issues/52624
105
  qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
106
107 120 Rishabh Dave
h3. 8 Nov 2023
108 121 Rishabh Dave
109 120 Rishabh Dave
fs: https://pulpito.ceph.com/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/
110 121 Rishabh Dave
smoke: https://pulpito.ceph.com/vshankar-2023-11-06_07:53:57-smoke-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/
111 120 Rishabh Dave
112
* https://tracker.ceph.com/issues/51964
113
  qa: test_cephfs_mirror_restart_sync_on_blocklist failure
114
* https://tracker.ceph.com/issues/62501
115
  pacific(?): qa: mgr-osd-full causes OSD aborts due to ENOSPC (fs/full/subvolume_snapshot_rm.sh)
116
* https://tracker.ceph.com/issues/52624
117
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
118
119 114 Jos Collin
h3. 2023 September 12
120
121 117 Jos Collin
https://pulpito.ceph.com/yuriw-2023-08-21_21:30:15-fs-wip-yuri2-testing-2023-08-21-0910-pacific-distro-default-smithi
122 115 Jos Collin
123 117 Jos Collin
  * https://tracker.ceph.com/issues/58992
124
    test_acls (tasks.cephfs.test_acls.TestACLs)
125
  * https://tracker.ceph.com/issues/62580
126
    testing: Test failure: test_snapshot_remove (tasks.cephfs.test_strays.TestStrays)
127 114 Jos Collin
128 111 Patrick Donnelly
h3. 2023 August 31
129
130 113 Patrick Donnelly
https://github.com/ceph/ceph/pull/53189
131
https://github.com/ceph/ceph/pull/53243
132
https://github.com/ceph/ceph/pull/53185
133
https://github.com/ceph/ceph/pull/52744
134
https://github.com/ceph/ceph/pull/51045
135
136 111 Patrick Donnelly
https://pulpito.ceph.com/pdonnell-2023-08-31_15:31:51-fs-wip-batrick-testing-20230831.124848-pacific-distro-default-smithi/
137
138 112 Patrick Donnelly
* https://tracker.ceph.com/issues/62501
139
    pacific(?): qa: mgr-osd-full causes OSD aborts due to ENOSPC (fs/full/subvolume_snapshot_rm.sh)
140
* https://tracker.ceph.com/issues/52624
141
    cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
142
* https://tracker.ceph.com/issues/54462
143
    Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi055 with status 128
144
* https://tracker.ceph.com/issues/50222
145
    osd: 5.2s0 deep-scrub : stat mismatch
146
* https://tracker.ceph.com/issues/50250
147
    mds: "log [WRN] : Scrub error on inode 0x10000004506 (/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.a log and `damage ls` output for details" ("freshly-calculated rstats don't match existing ones")
148
149
Some spurious infrastructure / valgrind noise during cleanup.
150 111 Patrick Donnelly
151 107 Patrick Donnelly
h3. 2023 August 22
152
153
Pacific v16.2.14 QA
154
155
https://pulpito.ceph.com/yuriw-2023-08-22_14:48:56-fs-pacific-release-distro-default-smithi/
156
https://pulpito.ceph.com/yuriw-2023-08-22_23:49:19-fs-pacific-release-distro-default-smithi/
157
158
* https://tracker.ceph.com/issues/62578
159
    mon: osd pg-upmap-items command causes PG_DEGRADED warnings
160 109 Patrick Donnelly
* https://tracker.ceph.com/issues/52624
161
    cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
162 108 Patrick Donnelly
* https://tracker.ceph.com/issues/62501
163
    pacific(?): qa: mgr-osd-full causes OSD aborts due to ENOSPC (fs/full/subvolume_snapshot_rm.sh)
164 107 Patrick Donnelly
* https://tracker.ceph.com/issues/58992
165
    test_acls (tasks.cephfs.test_acls.TestACLs)
166 108 Patrick Donnelly
* https://tracker.ceph.com/issues/62579
167
    client: evicted warning because client completes unmount before thrashed MDS comes back
168 110 Patrick Donnelly
* https://tracker.ceph.com/issues/62580
169
    testing: Test failure: test_snapshot_remove (tasks.cephfs.test_strays.TestStrays)
170 107 Patrick Donnelly
171 106 Patrick Donnelly
h3. 2023 August 16-2
172
173
https://trello.com/c/qRJogcXu/1827-wip-yuri2-testing-2023-08-16-1142-pacific
174
https://pulpito.ceph.com/?branch=wip-yuri2-testing-2023-08-16-1142-pacific
175
176
* https://tracker.ceph.com/issues/52624
177
    cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
178
* https://tracker.ceph.com/issues/58992
179
    test_acls (tasks.cephfs.test_acls.TestACLs)
180
* https://tracker.ceph.com/issues/62501
181
    pacific(?): qa: mgr-osd-full causes OSD aborts due to ENOSPC
182
183 105 Patrick Donnelly
h3. 2023 August 16
184
185
https://trello.com/c/4TiiTR0k/1821-wip-yuri7-testing-2023-08-16-1309-pacific-old-wip-yuri7-testing-2023-08-16-0933-pacific-old-wip-yuri7-testing-2023-08-15-0741-pa
186
https://pulpito.ceph.com/?branch=wip-yuri7-testing-2023-08-16-1309-pacific
187
188
* https://tracker.ceph.com/issues/62499
189
    testing (?): deadlock ffsb task
190
* https://tracker.ceph.com/issues/62501
191
    pacific(?): qa: mgr-osd-full causes OSD aborts due to ENOSPC
192
193 103 Patrick Donnelly
h3. 2023 August 11
194
195
https://trello.com/c/ONHeA3yz/1823-wip-yuri8-testing-2023-08-11-0834-pacific
196
https://pulpito.ceph.com/yuriw-2023-08-15_01:22:18-fs-wip-yuri8-testing-2023-08-11-0834-pacific-distro-default-smithi/
197
198
Some infra noise causes dead job.
199
200
* https://tracker.ceph.com/issues/52624
201
    cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
202
* https://tracker.ceph.com/issues/58992
203
    test_acls (tasks.cephfs.test_acls.TestACLs)
204 1 Patrick Donnelly
* https://tracker.ceph.com/issues/58340
205 107 Patrick Donnelly
    fsstress.sh failed with errno 124 
206 103 Patrick Donnelly
* https://tracker.ceph.com/issues/48773
207 107 Patrick Donnelly
    Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete
208 103 Patrick Donnelly
* https://tracker.ceph.com/issues/50527
209
    pacific: qa: untar_snap_rm timeout during osd thrashing (ceph-fuse)
210
211 100 Venky Shankar
h3. 2023 August 8
212
213
https://pulpito.ceph.com/?branch=wip-yuri10-testing-2023-08-01-0753-pacific
214
215
* https://tracker.ceph.com/issues/52624
216
    cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
217 1 Patrick Donnelly
* https://tracker.ceph.com/issues/50223
218
    qa: "client.4737 isn't responding to mclientcaps(revoke)"
219 101 Patrick Donnelly
* https://tracker.ceph.com/issues/62164
220
    qa: "cluster [ERR] MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13..."
221
* https://tracker.ceph.com/issues/51964
222
    qa: test_cephfs_mirror_restart_sync_on_blocklist failure
223
* https://tracker.ceph.com/issues/58992
224
    test_acls (tasks.cephfs.test_acls.TestACLs)
225
* https://tracker.ceph.com/issues/62465
226
    pacific (?): LibCephFS.ShutdownRace segmentation fault
227 102 Patrick Donnelly
* "fs/full/subvolume_snapshot_rm.sh" failure caused by bluestore crash.
228 100 Venky Shankar
229 104 Patrick Donnelly
h3. 2023 August 03
230
231
https://trello.com/c/8HALLv9T/1813-wip-yuri6-testing-2023-08-03-0807-pacific-old-wip-yuri6-testing-2023-07-24-0819-pacific2-old-wip-yuri6-testing-2023-07-24-0819-p
232
https://pulpito.ceph.com/?branch=wip-yuri6-testing-2023-08-03-0807-pacific
233
234
* https://tracker.ceph.com/issues/52624
235
    cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
236
* https://tracker.ceph.com/issues/58992
237
    test_acls (tasks.cephfs.test_acls.TestACLs)
238
239 99 Venky Shankar
h3. 2023 July 25
240
241
https://pulpito.ceph.com/?branch=wip-yuri-testing-2023-07-19-1340-pacific
242
243
* https://tracker.ceph.com/issues/52624
244
    cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
245
* https://tracker.ceph.com/issues/58992
246
    test_acls (tasks.cephfs.test_acls.TestACLs)
247
* https://tracker.ceph.com/issues/50223
248
    qa: "client.4737 isn't responding to mclientcaps(revoke)"
249
* https://tracker.ceph.com/issues/62160
250
    mds: MDS abort because newly corrupt dentry to be committed
251
* https://tracker.ceph.com/issues/61201
252
    qa: test_rebuild_moved_file (tasks/data-scan) fails because mds crashes in pacific (recovery thread crash)
253
254
255 94 Kotresh Hiremath Ravishankar
h3. 2023 May 17
256
257
https://pulpito.ceph.com/yuriw-2023-05-15_21:56:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/
258 97 Kotresh Hiremath Ravishankar
https://pulpito.ceph.com/yuriw-2023-05-17_14:20:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/ (fresh pkg and re-run because of package/installation issues)
259 94 Kotresh Hiremath Ravishankar
260
* https://tracker.ceph.com/issues/52624
261
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
262 96 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/61201 (NEW)
263 94 Kotresh Hiremath Ravishankar
  Test failure: test_rebuild_moved_file (tasks.cephfs.test_data_scan.TestDataScan)
264
* https://tracker.ceph.com/issues/58340
265
  fsstress.sh failed with errno 124 
266 1 Patrick Donnelly
* https://tracker.ceph.com/issues/52624
267 94 Kotresh Hiremath Ravishankar
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
268
* https://tracker.ceph.com/issues/58992
269
  test_acls (tasks.cephfs.test_acls.TestACLs)
270
* https://tracker.ceph.com/issues/54462
271 95 Kotresh Hiremath Ravishankar
  Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi055 with status 128 
272 94 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/58674
273 95 Kotresh Hiremath Ravishankar
  teuthology.exceptions.MaxWhileTries: reached maximum tries (180) after waiting for 180 seconds 
274 94 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/55446
275 95 Kotresh Hiremath Ravishankar
  fs/upgrade/mds_upgrade_sequence - hit max job timeout
276
 
277 94 Kotresh Hiremath Ravishankar
278
279 91 Kotresh Hiremath Ravishankar
h3. 2023 May 11
280
281
https://pulpito.ceph.com/yuriw-2023-05-09_19:39:46-fs-wip-yuri4-testing-2023-05-08-0846-pacific-distro-default-smithi
282
283
* https://tracker.ceph.com/issues/52624
284
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
285
* https://tracker.ceph.com/issues/51964
286
  qa: test_cephfs_mirror_restart_sync_on_blocklist failure
287
* https://tracker.ceph.com/issues/48773
288
  Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete
289
* https://tracker.ceph.com/issues/50223
290
  qa: "client.4737 isn't responding to mclientcaps(revoke)"
291 1 Patrick Donnelly
* https://tracker.ceph.com/issues/58992
292
  test_acls (tasks.cephfs.test_acls.TestACLs)
293 91 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/58340
294
  fsstress.sh failed with errno 124
295 93 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/51964
296
  Test failure: test_cephfs_mirror_restart_sync_on_blocklist (tasks.cephfs.test_mirroring.TestMirroring)
297
* https://tracker.ceph.com/issues/55446
298
  fs/upgrade/mds_upgrade_sequence - hit max job timeout
299 91 Kotresh Hiremath Ravishankar
300 90 Rishabh Dave
h3. 2023 May 4
301
302
https://pulpito.ceph.com/yuriw-2023-04-25_19:03:49-fs-wip-yuri5-testing-2023-04-25-0837-pacific-distro-default-smithi/
303
304
* https://tracker.ceph.com/issues/59560
305
  qa: RuntimeError: more than one file system available
306
* https://tracker.ceph.com/issues/59626
307
  FSMissing: File system xxxx does not exist in the map
308
* https://tracker.ceph.com/issues/58340
309
  fsstress.sh failed with errno 124
310
* https://tracker.ceph.com/issues/58992
311
  test_acls
312
* https://tracker.ceph.com/issues/48773
313
  Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete
314
* https://tracker.ceph.com/issues/57676
315
  qa: error during scrub thrashing: rank damage found: {'backtrace'}
316
317
318 86 Rishabh Dave
h3. 2023 Apr 13
319
320
https://pulpito.ceph.com/yuriw-2023-04-04_15:06:57-fs-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/
321
322
https://tracker.ceph.com/issues/52624
323
cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
324
* https://tracker.ceph.com/issues/57594
325
  Test failure: test_rebuild_moved_dir (tasks.cephfs.test_data_scan.TestDataScan)
326
* https://tracker.ceph.com/issues/54108
327
  qa: iogen workunit: "The following counters failed to be set on mds daemons: {'mds.exported', 'mds.imported'}" 
328
* https://tracker.ceph.com/issues/58340
329
  fsstress.sh failed with errno 125
330
* https://tracker.ceph.com/issues/54462
331
  Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi055 with status 128
332
* https://tracker.ceph.com/issues/49287     
333
  cephadm: podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found
334
* https://tracker.ceph.com/issues/58726
335
  test_acls: expected a yum based or a apt based system
336 40 Ramana Raja
337 84 Venky Shankar
h3. 2022 Dec 07
338
339
https://pulpito.ceph.com/yuriw-2022-12-07_15:45:30-fs-wip-yuri4-testing-2022-12-05-0921-pacific-distro-default-smithi/
340
341
many transient git.ceph.com related timeouts
342
343
* https://tracker.ceph.com/issues/52624
344
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
345
* https://tracker.ceph.com/issues/50224
346
    test_mirroring_init_failure_with_recovery - (tasks.cephfs.test_mirroring.TestMirroring)
347
* https://tracker.ceph.com/issues/56644
348
    qa: test_rapid_creation fails with "No space left on device"
349
* https://tracker.ceph.com/issues/58221
350
    pacific: Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan)
351
352 83 Venky Shankar
h3. 2022 Dec 02
353
354
many transient git.ceph.com related timeouts
355
many transient 'Failed to connect to the host via ssh' failures
356
357
* https://tracker.ceph.com/issues/57723
358
  pacific: qa: test_subvolume_snapshot_info_if_orphan_clone fails
359
* https://tracker.ceph.com/issues/52624
360
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
361
362 82 Venky Shankar
h3. 2022 Dec 01
363
364
many transient git.ceph.com related timeouts
365
366
* https://tracker.ceph.com/issues/57723
367
  pacific: qa: test_subvolume_snapshot_info_if_orphan_clone fails
368
* https://tracker.ceph.com/issues/52624
369
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
370
371 80 Patrick Donnelly
h3. 2022 Nov 18
372
373
https://trello.com/c/ecysAxl6/1656-wip-yuri-testing-2022-11-18-1500-pacific-old-wip-yuri-testing-2022-10-23-0729-pacific-old-wip-yuri-testing-2022-10-21-0927-pacif
374
https://pulpito.ceph.com/yuriw-2022-11-28_16:28:56-fs-wip-yuri-testing-2022-11-18-1500-pacific-distro-default-smithi/
375
376
2 ansible dead failures.
377
12 transient git.ceph.com related timeouts
378
379
* https://tracker.ceph.com/issues/57723
380
  pacific: qa: test_subvolume_snapshot_info_if_orphan_clone fails
381
* https://tracker.ceph.com/issues/52624
382
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
383
384 78 Milind Changire
h3. 2022 Oct 19
385
386
http://pulpito.front.sepia.ceph.com/yuriw-2022-10-10_17:18:40-fs-wip-yuri5-testing-2022-10-10-0837-pacific-distro-default-smithi/
387
388
* https://tracker.ceph.com/issues/57723
389
  pacific: qa: test_subvolume_snapshot_info_if_orphan_clone fails
390
* https://tracker.ceph.com/issues/52624
391
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
392
* https://tracker.ceph.com/issues/56644
393
  qa: test_rapid_creation fails with "No space left on device"
394
* https://tracker.ceph.com/issues/54460
395
  snaptest-multiple-capsnaps.sh test failure
396
* https://tracker.ceph.com/issues/57892
397
  sudo dd of=/home/ubuntu/cephtest/valgrind.supp failure during node setup phase
398
399
400
401 76 Venky Shankar
h3. 2022 Oct 06
402
403
http://pulpito.front.sepia.ceph.com/yuriw-2022-09-21_13:07:25-rados-wip-yuri5-testing-2022-09-20-1347-pacific-distro-default-smithi/
404
405
* https://tracker.ceph.com/issues/52624
406
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
407
* https://tracker.ceph.com/issues/50223
408
	qa: "client.4737 isn't responding to mclientcaps(revoke)"
409 77 Milind Changire
* https://tracker.ceph.com/issues/56507
410
  Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
411 76 Venky Shankar
412 75 Venky Shankar
h3. 2022 Sep 27
413
414
http://pulpito.front.sepia.ceph.com/yuriw-2022-09-22_22:35:04-fs-wip-yuri2-testing-2022-09-22-1400-pacific-distro-default-smithi/
415
416
* https://tracker.ceph.com/issues/52624
417
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
418
* https://tracker.ceph.com/issues/48773
419
    Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete
420
* https://tracker.ceph.com/issues/50224
421
    test_mirroring_init_failure_with_recovery - (tasks.cephfs.test_mirroring.TestMirroring)
422
* https://tracker.ceph.com/issues/56507
423
    Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
424
* https://tracker.ceph.com/issues/50223
425
	qa: "client.4737 isn't responding to mclientcaps(revoke)"
426
427 74 Venky Shankar
h3. 2022 Sep 22
428
429
http://pulpito.front.sepia.ceph.com/?branch=wip-yuri2-testing-2022-09-06-1007-pacific
430
431
* https://tracker.ceph.com/issues/52624
432
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
433
* https://tracker.ceph.com/issues/51282
434
    cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log
435
* https://tracker.ceph.com/issues/53360
436
    pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
437
438 73 Venky Shankar
h3. 2022 Sep 19
439
440
http://pulpito.front.sepia.ceph.com/yuriw-2022-09-16_19:05:25-fs-wip-yuri11-testing-2022-09-16-0958-pacific-distro-default-smithi/
441
442
* https://tracker.ceph.com/issues/52624
443
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
444
* https://tracker.ceph.com/issues/57594
445
    pacific: Test failure: test_rebuild_moved_dir (tasks.cephfs.test_data_scan.TestDataScan)
446
447 71 Venky Shankar
h3. 2022 Sep 15
448
449
http://pulpito.front.sepia.ceph.com/yuriw-2022-09-13_14:22:44-fs-wip-yuri5-testing-2022-09-09-1109-pacific-distro-default-smithi/
450
451
* https://tracker.ceph.com/issues/51282
452
    cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log
453
* https://tracker.ceph.com/issues/52624
454
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
455
* https://tracker.ceph.com/issues/53360
456
    pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
457 72 Venky Shankar
* https://tracker.ceph.com/issues/48773
458
    Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete
459 71 Venky Shankar
460 70 Kotresh Hiremath Ravishankar
h3. 2022 AUG 18
461
462
https://pulpito.ceph.com/yuriw-2022-08-18_23:16:33-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-default-smithi/
463
* https://tracker.ceph.com/issues/52624 - cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
464
* https://tracker.ceph.com/issues/51267 - tasks/workunit/snaps failure - Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status
465
* https://tracker.ceph.com/issues/48773 - Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete
466
* https://tracker.ceph.com/issues/51964 - qa: test_cephfs_mirror_restart_sync_on_blocklist failure
467
* https://tracker.ceph.com/issues/57218 - qa: tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} fails
468
* https://tracker.ceph.com/issues/56507 - Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
469
470
471
Re-run1: https://pulpito.ceph.com/yuriw-2022-08-19_21:01:11-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-default-smithi/
472
* https://tracker.ceph.com/issues/52624 - cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
473
* https://tracker.ceph.com/issues/57219 - mds crashed while running workunit test fs/misc/dirfrag.sh
474
475
476 69 Kotresh Hiremath Ravishankar
h3. 2022 AUG 11
477
478
https://pulpito.ceph.com/yuriw-2022-08-11_16:57:01-fs-wip-yuri3-testing-2022-08-11-0809-pacific-distro-default-smithi/
479
* Most of the failures are passed in re-run. Please check rerun failures below.
480
  - https://tracker.ceph.com/issues/57147 - test_full_fsync (tasks.cephfs.test_full.TestClusterFull) - mds stuck in up:creating state - osd crash
481
  - https://tracker.ceph.com/issues/52624 - cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
482
  - https://tracker.ceph.com/issues/51282 - cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log
483
  - https://tracker.ceph.com/issues/50224 - test_mirroring_init_failure_with_recovery - (tasks.cephfs.test_mirroring.TestMirroring)
484
  - https://tracker.ceph.com/issues/57083 - https://tracker.ceph.com/issues/53360 - tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} 
485
  - https://tracker.ceph.com/issues/51183 - Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
486
  - https://tracker.ceph.com/issues/56507 - Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)   
487
488
Re-run-1: https://pulpito.ceph.com/yuriw-2022-08-15_13:45:54-fs-wip-yuri3-testing-2022-08-11-0809-pacific-distro-default-smithi/
489
* https://tracker.ceph.com/issues/52624
490
  cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
491
* tasks/mirror - Command failed on smithi078 with status 123: "sudo find /var/log/ceph -name '*.log' -print0 ..." - Asked for re-run.
492
* https://tracker.ceph.com/issues/57083
493
* https://tracker.ceph.com/issues/53360
494
  tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} - tasks.cephfs.fuse_mount:mount command failed
495
  client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
496
* https://tracker.ceph.com/issues/51183
497
  Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
498
* https://tracker.ceph.com/issues/56507
499
  Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
500
501 68 Kotresh Hiremath Ravishankar
502
h3. 2022 AUG 04
503
504
https://pulpito.ceph.com/yuriw-2022-08-04_20:54:08-fs-wip-yuri6-testing-2022-08-04-0617-pacific-distro-default-smithi/
505
506
* https://tracker.ceph.com/issues/57087
507
  test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan)
508
* https://tracker.ceph.com/issues/52624
509
  cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
510
* https://tracker.ceph.com/issues/51267
511
  tasks/workunit/snaps failure - Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status      
512
* https://tracker.ceph.com/issues/53360
513
* https://tracker.ceph.com/issues/57083
514
  qa/import-legacy tasks.cephfs.fuse_mount:mount command failed - :ModuleNotFoundError: No module named 'ceph_volume_client'
515
* https://tracker.ceph.com/issues/56507
516
  Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
517
518
519 67 Kotresh Hiremath Ravishankar
h3. 2022 July 15
520
521
https://pulpito.ceph.com/yuriw-2022-07-21_22:57:30-fs-wip-yuri2-testing-2022-07-15-0755-pacific-distro-default-smithi
522
Re-run: https://pulpito.ceph.com/yuriw-2022-07-24_15:34:38-fs-wip-yuri2-testing-2022-07-15-0755-pacific-distro-default-smithi
523
524
* https://tracker.ceph.com/issues/52624
525
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
526
* https://tracker.ceph.com/issues/57083
527
* https://tracker.ceph.com/issues/53360
528
        tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} - tasks.cephfs.fuse_mount:mount command failed
529
        client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" 
530
* https://tracker.ceph.com/issues/51183
531
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
532
* https://tracker.ceph.com/issues/56507
533
        pacific: Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
534
535
536 66 Venky Shankar
h3. 2022 July 08
537
538
* https://tracker.ceph.com/issues/52624
539
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
540
* https://tracker.ceph.com/issues/53360
541
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
542
* https://tracker.ceph.com/issues/51183
543
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
544
* https://tracker.ceph.com/issues/56506
545
        pacific: Test failure: test_rebuild_backtraceless (tasks.cephfs.test_data_scan.TestDataScan)
546
* https://tracker.ceph.com/issues/56507
547
        pacific: Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
548
549 65 Venky Shankar
h3. 2022 Jun 28
550
551
https://pulpito.ceph.com/?branch=wip-yuri3-testing-2022-06-22-1121-pacific
552
553
* https://tracker.ceph.com/issues/52624
554
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
555
* https://tracker.ceph.com/issues/53360
556
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
557
* https://tracker.ceph.com/issues/51183
558
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
559
560 64 Venky Shankar
h3. 2022 Jun 22
561
562
https://pulpito.ceph.com/?branch=wip-yuri4-testing-2022-06-21-0704-pacific
563
564
* https://tracker.ceph.com/issues/52624
565
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
566
* https://tracker.ceph.com/issues/53360
567
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
568
* https://tracker.ceph.com/issues/51183
569
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
570
571 63 Venky Shankar
h3. 2022 Jun 17
572
573
https://pulpito.ceph.com/yuriw-2022-06-15_17:11:32-fs-wip-yuri10-testing-2022-06-15-0732-pacific-distro-default-smithi/
574
575
* https://tracker.ceph.com/issues/52624
576
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
577
* https://tracker.ceph.com/issues/53360
578
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
579
* https://tracker.ceph.com/issues/51183
580
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
581
582 62 Venky Shankar
h3. 2022 Jun 16
583
584
https://pulpito.ceph.com/yuriw-2022-06-15_17:08:32-fs-wip-yuri-testing-2022-06-14-0744-pacific-distro-default-smithi/
585
586
* https://tracker.ceph.com/issues/52624
587
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
588
* https://tracker.ceph.com/issues/53360
589
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
590
* https://tracker.ceph.com/issues/55449
591
        pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh
592
* https://tracker.ceph.com/issues/51267
593
        CommandFailedError: Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status 1:...
594
* https://tracker.ceph.com/issues/55332
595
        Failure in snaptest-git-ceph.sh (it's an async unlink/create bug)
596
597 61 Venky Shankar
h3. 2022 Jun 15
598
599
http://pulpito.front.sepia.ceph.com/yuriw-2022-06-09_13:32:29-fs-wip-yuri10-testing-2022-06-08-0730-pacific-distro-default-smithi/
600
601
* https://tracker.ceph.com/issues/52624
602
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
603
* https://tracker.ceph.com/issues/53360
604
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
605
* https://tracker.ceph.com/issues/55449
606
        pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh
607
608
609 60 Venky Shankar
h3. 2022 Jun 10
610
611
https://pulpito.ceph.com/?branch=wip-yuri5-testing-2022-06-07-1529-pacific
612
613
* https://tracker.ceph.com/issues/52624
614
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
615
* https://tracker.ceph.com/issues/53360
616
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
617
* https://tracker.ceph.com/issues/55449
618
        pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh
619
620 59 Venky Shankar
h3. 2022 Jun 09
621
622
https://pulpito.ceph.com/yuriw-2022-06-07_16:05:25-fs-wip-yuri4-testing-2022-06-01-1350-pacific-distro-default-smithi/
623
624
* https://tracker.ceph.com/issues/52624
625
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
626
* https://tracker.ceph.com/issues/53360
627
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
628
* https://tracker.ceph.com/issues/55449
629
        pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh
630
631 58 Venky Shankar
h3. 2022 May 06
632
633
https://pulpito.ceph.com/?sha1=73636a1b00037ff974bcdc969b009c5ecec626cc
634
635
* https://tracker.ceph.com/issues/52624
636
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
637
* https://tracker.ceph.com/issues/53360
638
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
639
640 57 Venky Shankar
h3. 2022 April 18
641
642
http://pulpito.front.sepia.ceph.com/?branch=wip-yuri4-testing-2022-04-18-0609-pacific
643
(only mgr/snap_schedule backport pr)
644
645
* https://tracker.ceph.com/issues/52624
646
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
647 56 Venky Shankar
648
h3. 2022 March 28
649
650
http://pulpito.front.sepia.ceph.com/yuriw-2022-03-25_20:52:40-fs-wip-yuri4-testing-2022-03-25-1203-pacific-distro-default-smithi/
651
http://pulpito.front.sepia.ceph.com/yuriw-2022-03-26_19:52:48-fs-wip-yuri4-testing-2022-03-25-1203-pacific-distro-default-smithi/
652
653
* https://tracker.ceph.com/issues/52624
654
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
655
* https://tracker.ceph.com/issues/53360
656
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
657
* https://tracker.ceph.com/issues/54411
658
	mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available;
659
	33 daemons have recently crashed" during suites/fsstress.sh
660
661 55 Venky Shankar
h3. 2022 March 25
662
663
https://pulpito.ceph.com/yuriw-2022-03-22_18:42:28-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/
664
https://pulpito.ceph.com/yuriw-2022-03-23_14:04:56-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/
665
666
* https://tracker.ceph.com/issues/52624
667
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
668
* https://tracker.ceph.com/issues/52606
669
        qa: test_dirfrag_limit
670
* https://tracker.ceph.com/issues/51905
671
        qa: "error reading sessionmap 'mds1_sessionmap'"
672
* https://tracker.ceph.com/issues/53360
673
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
674
* https://tracker.ceph.com/issues/51183
675
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
676 47 Patrick Donnelly
677 54 Venky Shankar
h3. 2022 March 22
678
679
https://pulpito.ceph.com/yuriw-2022-03-17_15:03:06-fs-wip-yuri10-testing-2022-03-16-1432-pacific-distro-default-smithi/
680
681
* https://tracker.ceph.com/issues/52624
682
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
683
* https://tracker.ceph.com/issues/52606
684
        qa: test_dirfrag_limit
685
* https://tracker.ceph.com/issues/51183
686
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
687
* https://tracker.ceph.com/issues/51905
688
        qa: "error reading sessionmap 'mds1_sessionmap'"
689
* https://tracker.ceph.com/issues/53360
690
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
691
* https://tracker.ceph.com/issues/54411
692
	mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available;
693
	33 daemons have recently crashed" during suites/fsstress.sh
694 47 Patrick Donnelly
695 46 Kotresh Hiremath Ravishankar
h3. 2021 November 22
696
697
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-12_00:33:28-fs-wip-yuri7-testing-2021-11-11-1339-pacific-distro-basic-smithi
698
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-08_20:21:11-fs-wip-yuri5-testing-2021-11-08-1003-pacific-distro-basic-smithi
699
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-08_15:19:37-fs-wip-yuri2-testing-2021-11-06-1322-pacific-distro-basic-smithi
700
701
702
* https://tracker.ceph.com/issues/53300
703
	qa: cluster [WRN] Scrub error on inode
704
* https://tracker.ceph.com/issues/53302
705
	qa: sudo logrotate /etc/logrotate.d/ceph-test.conf failed with status 1
706
* https://tracker.ceph.com/issues/53314
707
	qa: fs/upgrade/mds_upgrade_sequence test timeout
708
* https://tracker.ceph.com/issues/53316
709
	qa: (smithi150) slow request osd_op, currently waiting for sub ops warning
710
* https://tracker.ceph.com/issues/52624
711
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
712
* https://tracker.ceph.com/issues/52396
713
	pacific: qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
714
* https://tracker.ceph.com/issues/52875
715
	pacific: qa: test_dirfrag_limit
716
* https://tracker.ceph.com/issues/51705
717 1 Patrick Donnelly
	pacific: qa: tasks.cephfs.fuse_mount:mount command failed
718
* https://tracker.ceph.com/issues/39634
719
	qa: test_full_same_file timeout
720 46 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/49748
721
	gibba: Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds
722 48 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/51964
723
	qa: test_cephfs_mirror_restart_sync_on_blocklist failure
724 49 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/50223
725
	qa: "client.4737 isn't responding to mclientcaps(revoke)"
726 48 Kotresh Hiremath Ravishankar
727 47 Patrick Donnelly
728
h3. 2021 November 20
729
730
https://pulpito.ceph.com/?branch=wip-pdonnell-testing-20211120.024903-pacific
731
732
* https://tracker.ceph.com/issues/53360
733
    pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
734 46 Kotresh Hiremath Ravishankar
735 41 Patrick Donnelly
h3. 2021 September 14 (QE)
736
737
https://pulpito.ceph.com/yuriw-2021-09-10_15:01:12-fs-pacific-distro-basic-smithi/
738 1 Patrick Donnelly
http://pulpito.front.sepia.ceph.com/yuriw-2021-09-13_14:09:59-fs-pacific-distro-basic-smithi/
739 43 Ramana Raja
https://pulpito.ceph.com/yuriw-2021-09-14_15:09:45-fs-pacific-distro-basic-smithi/
740 41 Patrick Donnelly
741
* https://tracker.ceph.com/issues/52606
742
    qa: test_dirfrag_limit
743
* https://tracker.ceph.com/issues/52607
744
    qa: "mon.a (mon.0) 1022 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY)"
745 44 Ramana Raja
* https://tracker.ceph.com/issues/51705
746
    qa: tasks.cephfs.fuse_mount:mount command failed
747 41 Patrick Donnelly
748 40 Ramana Raja
h3. 2021 Sep 7
749
750
https://pulpito.ceph.com/yuriw-2021-09-07_17:38:38-fs-wip-yuri-testing-2021-09-07-0905-pacific-distro-basic-smithi/
751
https://pulpito.ceph.com/yuriw-2021-09-07_23:57:28-fs-wip-yuri-testing-2021-09-07-0905-pacific-distro-basic-smithi/
752
753
754
* https://tracker.ceph.com/issues/52396
755
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
756
* https://tracker.ceph.com/issues/51705
757
    qa: tasks.cephfs.fuse_mount:mount command failed
758 4 Patrick Donnelly
759 34 Ramana Raja
h3. 2021 Aug 30
760
761
https://pulpito.ceph.com/?branch=wip-yuri8-testing-2021-08-30-0930-pacific
762
763
* https://tracker.ceph.com/issues/45434
764
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
765
* https://tracker.ceph.com/issues/51705
766
    qa: tasks.cephfs.fuse_mount:mount command failed
767
* https://tracker.ceph.com/issues/52396
768
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
769 35 Ramana Raja
* https://tracker.ceph.com/issues/52487
770 36 Ramana Raja
    qa: Test failure: test_deep_split (tasks.cephfs.test_fragment.TestFragmentation)
771
* https://tracker.ceph.com/issues/51267
772 37 Ramana Raja
   Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh)
773
* https://tracker.ceph.com/issues/48772
774
   qa: pjd: not ok 9, 44, 80
775 35 Ramana Raja
776 34 Ramana Raja
777 31 Ramana Raja
h3. 2021 Aug 23
778
779
https://pulpito.ceph.com/yuriw-2021-08-23_19:33:26-fs-wip-yuri4-testing-2021-08-23-0812-pacific-distro-basic-smithi/
780
781
* https://tracker.ceph.com/issues/45434
782
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
783
* https://tracker.ceph.com/issues/51705
784
    qa: tasks.cephfs.fuse_mount:mount command failed
785 32 Ramana Raja
* https://tracker.ceph.com/issues/52396
786 33 Ramana Raja
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
787
* https://tracker.ceph.com/issues/52397
788
    qa: test_acls (tasks.cephfs.test_acls.TestACLs) failed
789
790 31 Ramana Raja
791 29 Ramana Raja
h3. 2021 Aug 11
792
793
https://pulpito.ceph.com/yuriw-2021-08-11_14:28:23-fs-wip-yuri8-testing-2021-08-09-0844-pacific-distro-basic-smithi/
794
https://pulpito.ceph.com/yuriw-2021-08-12_15:32:35-fs-wip-yuri8-testing-2021-08-09-0844-pacific-distro-basic-smithi/
795
796
* https://tracker.ceph.com/issues/45434
797
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
798
* https://tracker.ceph.com/issues/51705
799
    qa: tasks.cephfs.fuse_mount:mount command failed
800 30 Ramana Raja
* https://tracker.ceph.com/issues/50222
801
    osd: 5.2s0 deep-scrub : stat mismatch
802 29 Ramana Raja
803 19 Jos Collin
h3. 2021 July 15
804 20 Jos Collin
805 19 Jos Collin
https://pulpito.ceph.com/yuriw-2021-07-13_17:37:59-fs-wip-yuri-testing-2021-07-13-0812-pacific-distro-basic-smithi/
806
807 26 Jos Collin
* https://tracker.ceph.com/issues/45434
808
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
809
* https://tracker.ceph.com/issues/51705
810
    qa: tasks.cephfs.fuse_mount:mount command failed
811 27 Jos Collin
* https://tracker.ceph.com/issues/51183
812
    Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
813 28 Jos Collin
* https://tracker.ceph.com/issues/50528
814
    qa: fs:thrash: pjd suite not ok 80
815
* https://tracker.ceph.com/issues/51706
816
    qa: osd deep-scrub stat mismatch
817 26 Jos Collin
818 19 Jos Collin
h3. 2021 July 13
819 20 Jos Collin
820 19 Jos Collin
https://pulpito.ceph.com/yuriw-2021-07-08_23:33:26-fs-wip-yuri2-testing-2021-07-08-1142-pacific-distro-basic-smithi/
821
822 21 Jos Collin
* https://tracker.ceph.com/issues/51704
823
    Test failure: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth)
824 23 Jos Collin
* https://tracker.ceph.com/issues/45434
825
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
826 24 Jos Collin
* https://tracker.ceph.com/issues/51705
827 25 Jos Collin
    qa: tasks.cephfs.fuse_mount:mount command failed
828
* https://tracker.ceph.com/issues/48640
829 24 Jos Collin
    qa: snapshot mismatch during mds thrashing
830 21 Jos Collin
831 18 Patrick Donnelly
h3. 2021 June 29 (Integration Branch)
832
833
https://pulpito.ceph.com/yuriw-2021-06-29_00:15:33-fs-wip-yuri3-testing-2021-06-28-1259-pacific-distro-basic-smithi/
834
835
Some failures caused by incomplete backup of: https://github.com/ceph/ceph/pull/42065
836
Some package failures caused by nautilus missing package, like: https://pulpito.ceph.com/yuriw-2021-06-29_00:15:33-fs-wip-yuri3-testing-2021-06-28-1259-pacific-distro-basic-smithi/6241300/
837
838
* https://tracker.ceph.com/issues/45434
839
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
840
* https://tracker.ceph.com/issues/50260
841
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
842
* https://tracker.ceph.com/issues/51183
843
    qa: FileNotFoundError: [Errno 2] No such file or directory: '/sys/kernel/debug/ceph/3fab6bea-f243-47a4-a956-8c03a62b61b5.client4721/mds_sessions'
844
845
846 16 Jeff Layton
h3. 2021 June 28
847
848
https://pulpito.ceph.com/yuriw-2021-06-29_00:14:14-fs-wip-yuri-testing-2021-06-28-1259-pacific-distro-basic-smithi/
849
850
* https://tracker.ceph.com/issues/45434
851
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
852
* https://tracker.ceph.com/issues/51440
853 17 Jeff Layton
    fallocate fails with EACCES
854 16 Jeff Layton
* https://tracker.ceph.com/issues/51264
855
    TestVolumeClient failure
856
* https://tracker.ceph.com/issues/51266
857
    test cleanup failure
858
* https://tracker.ceph.com/issues/51183
859
    Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) 
860
861 12 Jeff Layton
h3. 2021 June 14
862
863
https://pulpito.ceph.com/yuriw-2021-06-14_16:21:07-fs-wip-yuri2-testing-2021-06-14-0717-pacific-distro-basic-smithi/
864
 
865
* https://tracker.ceph.com/issues/45434
866
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
867
* https://bugzilla.redhat.com/show_bug.cgi?id=1973276
868
    Could not reconnect to ubuntu@smithi076.front.sepia.ceph.com
869
* https://tracker.ceph.com/issues/51263
870
    pjdfstest rename test 10.t failed with EACCES
871
* https://tracker.ceph.com/issues/51264
872
    TestVolumeClient failure
873 13 Jeff Layton
* https://tracker.ceph.com/issues/51266
874
    Command failed on smithi204 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 
875 14 Jeff Layton
* https://tracker.ceph.com/issues/50279
876
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
877 15 Jeff Layton
* https://tracker.ceph.com/issues/51267
878
    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status 1
879 11 Patrick Donnelly
880
h3. 2021 June 07 (Integration Branch)
881
882
http://pulpito.front.sepia.ceph.com/?branch=wip-yuri4-testing-2021-06-07-0955-pacific
883
884
* https://tracker.ceph.com/issues/45434
885
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
886
* https://tracker.ceph.com/issues/50279
887
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
888
* https://tracker.ceph.com/issues/48773
889
    qa: scrub does not complete
890
* https://tracker.ceph.com/issues/51170
891
    pacific: qa: AttributeError: 'RemoteProcess' object has no attribute 'split'
892
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
893
    qa: quota failure
894
895
896 6 Patrick Donnelly
h3. 2021 Apr 28 (QE pre-release)
897
898
https://pulpito.ceph.com/yuriw-2021-04-28_19:27:30-fs-pacific-distro-basic-smithi/
899
https://pulpito.ceph.com/yuriw-2021-04-30_13:46:58-fs-pacific-distro-basic-smithi/
900
901
* https://tracker.ceph.com/issues/45434
902
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
903
* https://tracker.ceph.com/issues/50258
904
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
905
* https://tracker.ceph.com/issues/50260
906
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
907
* https://tracker.ceph.com/issues/49962
908
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
909
* https://tracker.ceph.com/issues/50016
910
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"
911
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
912
    qa: quota failure
913
* https://tracker.ceph.com/issues/50528
914
    pacific: qa: fs:thrash: pjd suite not ok 20
915
916
917 3 Patrick Donnelly
h3. 2021 Apr 22 (Integration Branch)
918
919
https://pulpito.ceph.com/yuriw-2021-04-22_16:46:11-fs-wip-yuri5-testing-2021-04-20-0819-pacific-distro-basic-smithi/
920
921
* https://tracker.ceph.com/issues/50527
922
    pacific: qa: untar_snap_rm timeout during osd thrashing (ceph-fuse)
923
* https://tracker.ceph.com/issues/50528
924
    pacific: qa: fs:thrash: pjd suite not ok 20
925
* https://tracker.ceph.com/issues/49500 (fixed in another integration run)
926
    qa: "Assertion `cb_done' failed."
927
* https://tracker.ceph.com/issues/49500
928
    qa: "Assertion `cb_done' failed."
929
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
930
    qa: quota failure
931
* https://tracker.ceph.com/issues/45434
932
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
933
* https://tracker.ceph.com/issues/50279
934
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
935
* https://tracker.ceph.com/issues/50258
936
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
937
* https://tracker.ceph.com/issues/49962
938
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
939
* https://tracker.ceph.com/issues/49962
940
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
941
* https://tracker.ceph.com/issues/50530
942
    pacific: client: abort after MDS blocklist
943
944 2 Patrick Donnelly
945
h3. 2021 Apr 21 (Integration Branch)
946
947
https://pulpito.ceph.com/yuriw-2021-04-21_19:25:00-fs-wip-yuri3-testing-2021-04-21-0937-pacific-distro-basic-smithi/
948
949
* https://tracker.ceph.com/issues/45434
950
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
951
* https://tracker.ceph.com/issues/50250
952
    mds: "log [WRN] : Scrub error on inode 0x10000004506 (/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.a log and `damage ls` output for details"
953
* https://tracker.ceph.com/issues/50258
954
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
955
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
956
    qa: quota failure
957
* https://tracker.ceph.com/issues/50016
958
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"
959
* https://tracker.ceph.com/issues/50495
960
    pacific: client: shutdown race fails with status 141
961
962
963 1 Patrick Donnelly
h3. 2021 Apr 07 (Integration Branch)
964
965
https://pulpito.ceph.com/yuriw-2021-04-07_17:37:43-fs-wip-yuri-testing-2021-04-07-0905-pacific-distro-basic-smithi/
966
967
* https://tracker.ceph.com/issues/45434
968
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
969
* https://tracker.ceph.com/issues/48805
970
    mds: "cluster [WRN] Scrub error on inode 0x1000000039d (/client.0/tmp/blogbench-1.0/src/blogtest_in) see mds.a log and `damage ls` output for details"
971
* https://tracker.ceph.com/issues/49500
972
    qa: "Assertion `cb_done' failed."
973
* https://tracker.ceph.com/issues/50258 (new)
974
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
975
* https://tracker.ceph.com/issues/49962
976
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
977
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
978
    qa: quota failure
979
* https://tracker.ceph.com/issues/50260
980
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
981
* https://tracker.ceph.com/issues/50016
982
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"