Project

General

Profile

Pacific » History » Version 58

Venky Shankar, 05/06/2022 11:18 AM

1 1 Patrick Donnelly
h1. Pacific
2
3 4 Patrick Donnelly
h2. On-call Schedule
4
5 52 Venky Shankar
* Feb: Patrick
6 53 Venky Shankar
* Mar: Jeff
7
* Apr: Jos Collin
8
* May: Ramana
9
* Jun: Xiubo
10
* Jul: Rishabh
11
* Aug: Kotresh
12
* Sep: Venky
13
* Oct: Milind
14 38 Ramana Raja
15 1 Patrick Donnelly
h2. Reviews
16 40 Ramana Raja
17 58 Venky Shankar
h3. 2022 May 06
18
19
https://pulpito.ceph.com/?sha1=73636a1b00037ff974bcdc969b009c5ecec626cc
20
21
* https://tracker.ceph.com/issues/52624
22
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
23
* https://tracker.ceph.com/issues/53360
24
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
25
26 57 Venky Shankar
h3. 2022 April 18
27
28
http://pulpito.front.sepia.ceph.com/?branch=wip-yuri4-testing-2022-04-18-0609-pacific
29
(only mgr/snap_schedule backport pr)
30
31
* https://tracker.ceph.com/issues/52624
32
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
33 56 Venky Shankar
34
h3. 2022 March 28
35
36
http://pulpito.front.sepia.ceph.com/yuriw-2022-03-25_20:52:40-fs-wip-yuri4-testing-2022-03-25-1203-pacific-distro-default-smithi/
37
http://pulpito.front.sepia.ceph.com/yuriw-2022-03-26_19:52:48-fs-wip-yuri4-testing-2022-03-25-1203-pacific-distro-default-smithi/
38
39
* https://tracker.ceph.com/issues/52624
40
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
41
* https://tracker.ceph.com/issues/53360
42
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
43
* https://tracker.ceph.com/issues/54411
44
	mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available;
45
	33 daemons have recently crashed" during suites/fsstress.sh
46
47 55 Venky Shankar
h3. 2022 March 25
48
49
https://pulpito.ceph.com/yuriw-2022-03-22_18:42:28-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/
50
https://pulpito.ceph.com/yuriw-2022-03-23_14:04:56-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/
51
52
* https://tracker.ceph.com/issues/52624
53
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
54
* https://tracker.ceph.com/issues/52606
55
        qa: test_dirfrag_limit
56
* https://tracker.ceph.com/issues/51905
57
        qa: "error reading sessionmap 'mds1_sessionmap'"
58
* https://tracker.ceph.com/issues/53360
59
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
60
* https://tracker.ceph.com/issues/51183
61
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
62 47 Patrick Donnelly
63 54 Venky Shankar
h3. 2022 March 22
64
65
https://pulpito.ceph.com/yuriw-2022-03-17_15:03:06-fs-wip-yuri10-testing-2022-03-16-1432-pacific-distro-default-smithi/
66
67
* https://tracker.ceph.com/issues/52624
68
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
69
* https://tracker.ceph.com/issues/52606
70
        qa: test_dirfrag_limit
71
* https://tracker.ceph.com/issues/51183
72
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
73
* https://tracker.ceph.com/issues/51905
74
        qa: "error reading sessionmap 'mds1_sessionmap'"
75
* https://tracker.ceph.com/issues/53360
76
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
77
* https://tracker.ceph.com/issues/54411
78
	mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available;
79
	33 daemons have recently crashed" during suites/fsstress.sh
80 47 Patrick Donnelly
81 46 Kotresh Hiremath Ravishankar
h3. 2021 November 22
82
83
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-12_00:33:28-fs-wip-yuri7-testing-2021-11-11-1339-pacific-distro-basic-smithi
84
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-08_20:21:11-fs-wip-yuri5-testing-2021-11-08-1003-pacific-distro-basic-smithi
85
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-08_15:19:37-fs-wip-yuri2-testing-2021-11-06-1322-pacific-distro-basic-smithi
86
87
88
* https://tracker.ceph.com/issues/53300
89
	qa: cluster [WRN] Scrub error on inode
90
* https://tracker.ceph.com/issues/53302
91
	qa: sudo logrotate /etc/logrotate.d/ceph-test.conf failed with status 1
92
* https://tracker.ceph.com/issues/53314
93
	qa: fs/upgrade/mds_upgrade_sequence test timeout
94
* https://tracker.ceph.com/issues/53316
95
	qa: (smithi150) slow request osd_op, currently waiting for sub ops warning
96
* https://tracker.ceph.com/issues/52624
97
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
98
* https://tracker.ceph.com/issues/52396
99
	pacific: qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
100
* https://tracker.ceph.com/issues/52875
101
	pacific: qa: test_dirfrag_limit
102
* https://tracker.ceph.com/issues/51705
103 1 Patrick Donnelly
	pacific: qa: tasks.cephfs.fuse_mount:mount command failed
104
* https://tracker.ceph.com/issues/39634
105
	qa: test_full_same_file timeout
106 46 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/49748
107
	gibba: Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds
108 48 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/51964
109
	qa: test_cephfs_mirror_restart_sync_on_blocklist failure
110 49 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/50223
111
	qa: "client.4737 isn't responding to mclientcaps(revoke)"
112 48 Kotresh Hiremath Ravishankar
113 47 Patrick Donnelly
114
h3. 2021 November 20
115
116
https://pulpito.ceph.com/?branch=wip-pdonnell-testing-20211120.024903-pacific
117
118
* https://tracker.ceph.com/issues/53360
119
    pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
120 46 Kotresh Hiremath Ravishankar
121 41 Patrick Donnelly
h3. 2021 September 14 (QE)
122
123
https://pulpito.ceph.com/yuriw-2021-09-10_15:01:12-fs-pacific-distro-basic-smithi/
124 1 Patrick Donnelly
http://pulpito.front.sepia.ceph.com/yuriw-2021-09-13_14:09:59-fs-pacific-distro-basic-smithi/
125 43 Ramana Raja
https://pulpito.ceph.com/yuriw-2021-09-14_15:09:45-fs-pacific-distro-basic-smithi/
126 41 Patrick Donnelly
127
* https://tracker.ceph.com/issues/52606
128
    qa: test_dirfrag_limit
129
* https://tracker.ceph.com/issues/52607
130
    qa: "mon.a (mon.0) 1022 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY)"
131 44 Ramana Raja
* https://tracker.ceph.com/issues/51705
132
    qa: tasks.cephfs.fuse_mount:mount command failed
133 41 Patrick Donnelly
134 40 Ramana Raja
h3. 2021 Sep 7
135
136
https://pulpito.ceph.com/yuriw-2021-09-07_17:38:38-fs-wip-yuri-testing-2021-09-07-0905-pacific-distro-basic-smithi/
137
https://pulpito.ceph.com/yuriw-2021-09-07_23:57:28-fs-wip-yuri-testing-2021-09-07-0905-pacific-distro-basic-smithi/
138
139
140
* https://tracker.ceph.com/issues/52396
141
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
142
* https://tracker.ceph.com/issues/51705
143
    qa: tasks.cephfs.fuse_mount:mount command failed
144 4 Patrick Donnelly
145 34 Ramana Raja
h3. 2021 Aug 30
146
147
https://pulpito.ceph.com/?branch=wip-yuri8-testing-2021-08-30-0930-pacific
148
149
* https://tracker.ceph.com/issues/45434
150
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
151
* https://tracker.ceph.com/issues/51705
152
    qa: tasks.cephfs.fuse_mount:mount command failed
153
* https://tracker.ceph.com/issues/52396
154
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
155 35 Ramana Raja
* https://tracker.ceph.com/issues/52487
156 36 Ramana Raja
    qa: Test failure: test_deep_split (tasks.cephfs.test_fragment.TestFragmentation)
157
* https://tracker.ceph.com/issues/51267
158 37 Ramana Raja
   Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh)
159
* https://tracker.ceph.com/issues/48772
160
   qa: pjd: not ok 9, 44, 80
161 35 Ramana Raja
162 34 Ramana Raja
163 31 Ramana Raja
h3. 2021 Aug 23
164
165
https://pulpito.ceph.com/yuriw-2021-08-23_19:33:26-fs-wip-yuri4-testing-2021-08-23-0812-pacific-distro-basic-smithi/
166
167
* https://tracker.ceph.com/issues/45434
168
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
169
* https://tracker.ceph.com/issues/51705
170
    qa: tasks.cephfs.fuse_mount:mount command failed
171 32 Ramana Raja
* https://tracker.ceph.com/issues/52396
172 33 Ramana Raja
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
173
* https://tracker.ceph.com/issues/52397
174
    qa: test_acls (tasks.cephfs.test_acls.TestACLs) failed
175
176 31 Ramana Raja
177 29 Ramana Raja
h3. 2021 Aug 11
178
179
https://pulpito.ceph.com/yuriw-2021-08-11_14:28:23-fs-wip-yuri8-testing-2021-08-09-0844-pacific-distro-basic-smithi/
180
https://pulpito.ceph.com/yuriw-2021-08-12_15:32:35-fs-wip-yuri8-testing-2021-08-09-0844-pacific-distro-basic-smithi/
181
182
* https://tracker.ceph.com/issues/45434
183
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
184
* https://tracker.ceph.com/issues/51705
185
    qa: tasks.cephfs.fuse_mount:mount command failed
186 30 Ramana Raja
* https://tracker.ceph.com/issues/50222
187
    osd: 5.2s0 deep-scrub : stat mismatch
188 29 Ramana Raja
189 19 Jos Collin
h3. 2021 July 15
190 20 Jos Collin
191 19 Jos Collin
https://pulpito.ceph.com/yuriw-2021-07-13_17:37:59-fs-wip-yuri-testing-2021-07-13-0812-pacific-distro-basic-smithi/
192
193 26 Jos Collin
* https://tracker.ceph.com/issues/45434
194
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
195
* https://tracker.ceph.com/issues/51705
196
    qa: tasks.cephfs.fuse_mount:mount command failed
197 27 Jos Collin
* https://tracker.ceph.com/issues/51183
198
    Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
199 28 Jos Collin
* https://tracker.ceph.com/issues/50528
200
    qa: fs:thrash: pjd suite not ok 80
201
* https://tracker.ceph.com/issues/51706
202
    qa: osd deep-scrub stat mismatch
203 26 Jos Collin
204 19 Jos Collin
h3. 2021 July 13
205 20 Jos Collin
206 19 Jos Collin
https://pulpito.ceph.com/yuriw-2021-07-08_23:33:26-fs-wip-yuri2-testing-2021-07-08-1142-pacific-distro-basic-smithi/
207
208 21 Jos Collin
* https://tracker.ceph.com/issues/51704
209
    Test failure: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth)
210 23 Jos Collin
* https://tracker.ceph.com/issues/45434
211
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
212 24 Jos Collin
* https://tracker.ceph.com/issues/51705
213 25 Jos Collin
    qa: tasks.cephfs.fuse_mount:mount command failed
214
* https://tracker.ceph.com/issues/48640
215 24 Jos Collin
    qa: snapshot mismatch during mds thrashing
216 21 Jos Collin
217 18 Patrick Donnelly
h3. 2021 June 29 (Integration Branch)
218
219
https://pulpito.ceph.com/yuriw-2021-06-29_00:15:33-fs-wip-yuri3-testing-2021-06-28-1259-pacific-distro-basic-smithi/
220
221
Some failures caused by incomplete backup of: https://github.com/ceph/ceph/pull/42065
222
Some package failures caused by nautilus missing package, like: https://pulpito.ceph.com/yuriw-2021-06-29_00:15:33-fs-wip-yuri3-testing-2021-06-28-1259-pacific-distro-basic-smithi/6241300/
223
224
* https://tracker.ceph.com/issues/45434
225
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
226
* https://tracker.ceph.com/issues/50260
227
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
228
* https://tracker.ceph.com/issues/51183
229
    qa: FileNotFoundError: [Errno 2] No such file or directory: '/sys/kernel/debug/ceph/3fab6bea-f243-47a4-a956-8c03a62b61b5.client4721/mds_sessions'
230
231
232 16 Jeff Layton
h3. 2021 June 28
233
234
https://pulpito.ceph.com/yuriw-2021-06-29_00:14:14-fs-wip-yuri-testing-2021-06-28-1259-pacific-distro-basic-smithi/
235
236
* https://tracker.ceph.com/issues/45434
237
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
238
* https://tracker.ceph.com/issues/51440
239 17 Jeff Layton
    fallocate fails with EACCES
240 16 Jeff Layton
* https://tracker.ceph.com/issues/51264
241
    TestVolumeClient failure
242
* https://tracker.ceph.com/issues/51266
243
    test cleanup failure
244
* https://tracker.ceph.com/issues/51183
245
    Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) 
246
247 12 Jeff Layton
h3. 2021 June 14
248
249
https://pulpito.ceph.com/yuriw-2021-06-14_16:21:07-fs-wip-yuri2-testing-2021-06-14-0717-pacific-distro-basic-smithi/
250
 
251
* https://tracker.ceph.com/issues/45434
252
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
253
* https://bugzilla.redhat.com/show_bug.cgi?id=1973276
254
    Could not reconnect to ubuntu@smithi076.front.sepia.ceph.com
255
* https://tracker.ceph.com/issues/51263
256
    pjdfstest rename test 10.t failed with EACCES
257
* https://tracker.ceph.com/issues/51264
258
    TestVolumeClient failure
259 13 Jeff Layton
* https://tracker.ceph.com/issues/51266
260
    Command failed on smithi204 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 
261 14 Jeff Layton
* https://tracker.ceph.com/issues/50279
262
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
263 15 Jeff Layton
* https://tracker.ceph.com/issues/51267
264
    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status 1
265 11 Patrick Donnelly
266
h3. 2021 June 07 (Integration Branch)
267
268
http://pulpito.front.sepia.ceph.com/?branch=wip-yuri4-testing-2021-06-07-0955-pacific
269
270
* https://tracker.ceph.com/issues/45434
271
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
272
* https://tracker.ceph.com/issues/50279
273
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
274
* https://tracker.ceph.com/issues/48773
275
    qa: scrub does not complete
276
* https://tracker.ceph.com/issues/51170
277
    pacific: qa: AttributeError: 'RemoteProcess' object has no attribute 'split'
278
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
279
    qa: quota failure
280
281
282 6 Patrick Donnelly
h3. 2021 Apr 28 (QE pre-release)
283
284
https://pulpito.ceph.com/yuriw-2021-04-28_19:27:30-fs-pacific-distro-basic-smithi/
285
https://pulpito.ceph.com/yuriw-2021-04-30_13:46:58-fs-pacific-distro-basic-smithi/
286
287
* https://tracker.ceph.com/issues/45434
288
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
289
* https://tracker.ceph.com/issues/50258
290
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
291
* https://tracker.ceph.com/issues/50260
292
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
293
* https://tracker.ceph.com/issues/49962
294
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
295
* https://tracker.ceph.com/issues/50016
296
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"
297
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
298
    qa: quota failure
299
* https://tracker.ceph.com/issues/50528
300
    pacific: qa: fs:thrash: pjd suite not ok 20
301
302
303 3 Patrick Donnelly
h3. 2021 Apr 22 (Integration Branch)
304
305
https://pulpito.ceph.com/yuriw-2021-04-22_16:46:11-fs-wip-yuri5-testing-2021-04-20-0819-pacific-distro-basic-smithi/
306
307
* https://tracker.ceph.com/issues/50527
308
    pacific: qa: untar_snap_rm timeout during osd thrashing (ceph-fuse)
309
* https://tracker.ceph.com/issues/50528
310
    pacific: qa: fs:thrash: pjd suite not ok 20
311
* https://tracker.ceph.com/issues/49500 (fixed in another integration run)
312
    qa: "Assertion `cb_done' failed."
313
* https://tracker.ceph.com/issues/49500
314
    qa: "Assertion `cb_done' failed."
315
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
316
    qa: quota failure
317
* https://tracker.ceph.com/issues/45434
318
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
319
* https://tracker.ceph.com/issues/50279
320
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
321
* https://tracker.ceph.com/issues/50258
322
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
323
* https://tracker.ceph.com/issues/49962
324
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
325
* https://tracker.ceph.com/issues/49962
326
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
327
* https://tracker.ceph.com/issues/50530
328
    pacific: client: abort after MDS blocklist
329
330 2 Patrick Donnelly
331
h3. 2021 Apr 21 (Integration Branch)
332
333
https://pulpito.ceph.com/yuriw-2021-04-21_19:25:00-fs-wip-yuri3-testing-2021-04-21-0937-pacific-distro-basic-smithi/
334
335
* https://tracker.ceph.com/issues/45434
336
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
337
* https://tracker.ceph.com/issues/50250
338
    mds: "log [WRN] : Scrub error on inode 0x10000004506 (/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.a log and `damage ls` output for details"
339
* https://tracker.ceph.com/issues/50258
340
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
341
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
342
    qa: quota failure
343
* https://tracker.ceph.com/issues/50016
344
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"
345
* https://tracker.ceph.com/issues/50495
346
    pacific: client: shutdown race fails with status 141
347
348
349 1 Patrick Donnelly
h3. 2021 Apr 07 (Integration Branch)
350
351
https://pulpito.ceph.com/yuriw-2021-04-07_17:37:43-fs-wip-yuri-testing-2021-04-07-0905-pacific-distro-basic-smithi/
352
353
* https://tracker.ceph.com/issues/45434
354
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
355
* https://tracker.ceph.com/issues/48805
356
    mds: "cluster [WRN] Scrub error on inode 0x1000000039d (/client.0/tmp/blogbench-1.0/src/blogtest_in) see mds.a log and `damage ls` output for details"
357
* https://tracker.ceph.com/issues/49500
358
    qa: "Assertion `cb_done' failed."
359
* https://tracker.ceph.com/issues/50258 (new)
360
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
361
* https://tracker.ceph.com/issues/49962
362
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
363
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
364
    qa: quota failure
365
* https://tracker.ceph.com/issues/50260
366
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
367
* https://tracker.ceph.com/issues/50016
368
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"