Project

General

Profile

Pacific » History » Version 56

Venky Shankar, 03/28/2022 09:33 AM

1 1 Patrick Donnelly
h1. Pacific
2
3 4 Patrick Donnelly
h2. On-call Schedule
4
5 52 Venky Shankar
* Feb: Patrick
6 53 Venky Shankar
* Mar: Jeff
7
* Apr: Jos Collin
8
* May: Ramana
9
* Jun: Xiubo
10
* Jul: Rishabh
11
* Aug: Kotresh
12
* Sep: Venky
13
* Oct: Milind
14 38 Ramana Raja
15 1 Patrick Donnelly
h2. Reviews
16 40 Ramana Raja
17 56 Venky Shankar
18
h3. 2022 March 28
19
20
http://pulpito.front.sepia.ceph.com/yuriw-2022-03-25_20:52:40-fs-wip-yuri4-testing-2022-03-25-1203-pacific-distro-default-smithi/
21
http://pulpito.front.sepia.ceph.com/yuriw-2022-03-26_19:52:48-fs-wip-yuri4-testing-2022-03-25-1203-pacific-distro-default-smithi/
22
23
* https://tracker.ceph.com/issues/52624
24
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
25
* https://tracker.ceph.com/issues/53360
26
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
27
* https://tracker.ceph.com/issues/54411
28
	mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available;
29
	33 daemons have recently crashed" during suites/fsstress.sh
30
31 55 Venky Shankar
h3. 2022 March 25
32
33
https://pulpito.ceph.com/yuriw-2022-03-22_18:42:28-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/
34
https://pulpito.ceph.com/yuriw-2022-03-23_14:04:56-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/
35
36
* https://tracker.ceph.com/issues/52624
37
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
38
* https://tracker.ceph.com/issues/52606
39
        qa: test_dirfrag_limit
40
* https://tracker.ceph.com/issues/51905
41
        qa: "error reading sessionmap 'mds1_sessionmap'"
42
* https://tracker.ceph.com/issues/53360
43
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
44
* https://tracker.ceph.com/issues/51183
45
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
46 47 Patrick Donnelly
47 54 Venky Shankar
h3. 2022 March 22
48
49
https://pulpito.ceph.com/yuriw-2022-03-17_15:03:06-fs-wip-yuri10-testing-2022-03-16-1432-pacific-distro-default-smithi/
50
51
* https://tracker.ceph.com/issues/52624
52
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
53
* https://tracker.ceph.com/issues/52606
54
        qa: test_dirfrag_limit
55
* https://tracker.ceph.com/issues/51183
56
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
57
* https://tracker.ceph.com/issues/51905
58
        qa: "error reading sessionmap 'mds1_sessionmap'"
59
* https://tracker.ceph.com/issues/53360
60
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
61
* https://tracker.ceph.com/issues/54411
62
	mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available;
63
	33 daemons have recently crashed" during suites/fsstress.sh
64 47 Patrick Donnelly
65 46 Kotresh Hiremath Ravishankar
h3. 2021 November 22
66
67
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-12_00:33:28-fs-wip-yuri7-testing-2021-11-11-1339-pacific-distro-basic-smithi
68
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-08_20:21:11-fs-wip-yuri5-testing-2021-11-08-1003-pacific-distro-basic-smithi
69
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-08_15:19:37-fs-wip-yuri2-testing-2021-11-06-1322-pacific-distro-basic-smithi
70
71
72
* https://tracker.ceph.com/issues/53300
73
	qa: cluster [WRN] Scrub error on inode
74
* https://tracker.ceph.com/issues/53302
75
	qa: sudo logrotate /etc/logrotate.d/ceph-test.conf failed with status 1
76
* https://tracker.ceph.com/issues/53314
77
	qa: fs/upgrade/mds_upgrade_sequence test timeout
78
* https://tracker.ceph.com/issues/53316
79
	qa: (smithi150) slow request osd_op, currently waiting for sub ops warning
80
* https://tracker.ceph.com/issues/52624
81
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
82
* https://tracker.ceph.com/issues/52396
83
	pacific: qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
84
* https://tracker.ceph.com/issues/52875
85
	pacific: qa: test_dirfrag_limit
86
* https://tracker.ceph.com/issues/51705
87 1 Patrick Donnelly
	pacific: qa: tasks.cephfs.fuse_mount:mount command failed
88
* https://tracker.ceph.com/issues/39634
89
	qa: test_full_same_file timeout
90 46 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/49748
91
	gibba: Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds
92 48 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/51964
93
	qa: test_cephfs_mirror_restart_sync_on_blocklist failure
94 49 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/50223
95
	qa: "client.4737 isn't responding to mclientcaps(revoke)"
96 48 Kotresh Hiremath Ravishankar
97 47 Patrick Donnelly
98
h3. 2021 November 20
99
100
https://pulpito.ceph.com/?branch=wip-pdonnell-testing-20211120.024903-pacific
101
102
* https://tracker.ceph.com/issues/53360
103
    pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
104 46 Kotresh Hiremath Ravishankar
105 41 Patrick Donnelly
h3. 2021 September 14 (QE)
106
107
https://pulpito.ceph.com/yuriw-2021-09-10_15:01:12-fs-pacific-distro-basic-smithi/
108 1 Patrick Donnelly
http://pulpito.front.sepia.ceph.com/yuriw-2021-09-13_14:09:59-fs-pacific-distro-basic-smithi/
109 43 Ramana Raja
https://pulpito.ceph.com/yuriw-2021-09-14_15:09:45-fs-pacific-distro-basic-smithi/
110 41 Patrick Donnelly
111
* https://tracker.ceph.com/issues/52606
112
    qa: test_dirfrag_limit
113
* https://tracker.ceph.com/issues/52607
114
    qa: "mon.a (mon.0) 1022 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY)"
115 44 Ramana Raja
* https://tracker.ceph.com/issues/51705
116
    qa: tasks.cephfs.fuse_mount:mount command failed
117 41 Patrick Donnelly
118 40 Ramana Raja
h3. 2021 Sep 7
119
120
https://pulpito.ceph.com/yuriw-2021-09-07_17:38:38-fs-wip-yuri-testing-2021-09-07-0905-pacific-distro-basic-smithi/
121
https://pulpito.ceph.com/yuriw-2021-09-07_23:57:28-fs-wip-yuri-testing-2021-09-07-0905-pacific-distro-basic-smithi/
122
123
124
* https://tracker.ceph.com/issues/52396
125
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
126
* https://tracker.ceph.com/issues/51705
127
    qa: tasks.cephfs.fuse_mount:mount command failed
128 4 Patrick Donnelly
129 34 Ramana Raja
h3. 2021 Aug 30
130
131
https://pulpito.ceph.com/?branch=wip-yuri8-testing-2021-08-30-0930-pacific
132
133
* https://tracker.ceph.com/issues/45434
134
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
135
* https://tracker.ceph.com/issues/51705
136
    qa: tasks.cephfs.fuse_mount:mount command failed
137
* https://tracker.ceph.com/issues/52396
138
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
139 35 Ramana Raja
* https://tracker.ceph.com/issues/52487
140 36 Ramana Raja
    qa: Test failure: test_deep_split (tasks.cephfs.test_fragment.TestFragmentation)
141
* https://tracker.ceph.com/issues/51267
142 37 Ramana Raja
   Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh)
143
* https://tracker.ceph.com/issues/48772
144
   qa: pjd: not ok 9, 44, 80
145 35 Ramana Raja
146 34 Ramana Raja
147 31 Ramana Raja
h3. 2021 Aug 23
148
149
https://pulpito.ceph.com/yuriw-2021-08-23_19:33:26-fs-wip-yuri4-testing-2021-08-23-0812-pacific-distro-basic-smithi/
150
151
* https://tracker.ceph.com/issues/45434
152
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
153
* https://tracker.ceph.com/issues/51705
154
    qa: tasks.cephfs.fuse_mount:mount command failed
155 32 Ramana Raja
* https://tracker.ceph.com/issues/52396
156 33 Ramana Raja
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
157
* https://tracker.ceph.com/issues/52397
158
    qa: test_acls (tasks.cephfs.test_acls.TestACLs) failed
159
160 31 Ramana Raja
161 29 Ramana Raja
h3. 2021 Aug 11
162
163
https://pulpito.ceph.com/yuriw-2021-08-11_14:28:23-fs-wip-yuri8-testing-2021-08-09-0844-pacific-distro-basic-smithi/
164
https://pulpito.ceph.com/yuriw-2021-08-12_15:32:35-fs-wip-yuri8-testing-2021-08-09-0844-pacific-distro-basic-smithi/
165
166
* https://tracker.ceph.com/issues/45434
167
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
168
* https://tracker.ceph.com/issues/51705
169
    qa: tasks.cephfs.fuse_mount:mount command failed
170 30 Ramana Raja
* https://tracker.ceph.com/issues/50222
171
    osd: 5.2s0 deep-scrub : stat mismatch
172 29 Ramana Raja
173 19 Jos Collin
h3. 2021 July 15
174 20 Jos Collin
175 19 Jos Collin
https://pulpito.ceph.com/yuriw-2021-07-13_17:37:59-fs-wip-yuri-testing-2021-07-13-0812-pacific-distro-basic-smithi/
176
177 26 Jos Collin
* https://tracker.ceph.com/issues/45434
178
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
179
* https://tracker.ceph.com/issues/51705
180
    qa: tasks.cephfs.fuse_mount:mount command failed
181 27 Jos Collin
* https://tracker.ceph.com/issues/51183
182
    Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
183 28 Jos Collin
* https://tracker.ceph.com/issues/50528
184
    qa: fs:thrash: pjd suite not ok 80
185
* https://tracker.ceph.com/issues/51706
186
    qa: osd deep-scrub stat mismatch
187 26 Jos Collin
188 19 Jos Collin
h3. 2021 July 13
189 20 Jos Collin
190 19 Jos Collin
https://pulpito.ceph.com/yuriw-2021-07-08_23:33:26-fs-wip-yuri2-testing-2021-07-08-1142-pacific-distro-basic-smithi/
191
192 21 Jos Collin
* https://tracker.ceph.com/issues/51704
193
    Test failure: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth)
194 23 Jos Collin
* https://tracker.ceph.com/issues/45434
195
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
196 24 Jos Collin
* https://tracker.ceph.com/issues/51705
197 25 Jos Collin
    qa: tasks.cephfs.fuse_mount:mount command failed
198
* https://tracker.ceph.com/issues/48640
199 24 Jos Collin
    qa: snapshot mismatch during mds thrashing
200 21 Jos Collin
201 18 Patrick Donnelly
h3. 2021 June 29 (Integration Branch)
202
203
https://pulpito.ceph.com/yuriw-2021-06-29_00:15:33-fs-wip-yuri3-testing-2021-06-28-1259-pacific-distro-basic-smithi/
204
205
Some failures caused by incomplete backup of: https://github.com/ceph/ceph/pull/42065
206
Some package failures caused by nautilus missing package, like: https://pulpito.ceph.com/yuriw-2021-06-29_00:15:33-fs-wip-yuri3-testing-2021-06-28-1259-pacific-distro-basic-smithi/6241300/
207
208
* https://tracker.ceph.com/issues/45434
209
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
210
* https://tracker.ceph.com/issues/50260
211
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
212
* https://tracker.ceph.com/issues/51183
213
    qa: FileNotFoundError: [Errno 2] No such file or directory: '/sys/kernel/debug/ceph/3fab6bea-f243-47a4-a956-8c03a62b61b5.client4721/mds_sessions'
214
215
216 16 Jeff Layton
h3. 2021 June 28
217
218
https://pulpito.ceph.com/yuriw-2021-06-29_00:14:14-fs-wip-yuri-testing-2021-06-28-1259-pacific-distro-basic-smithi/
219
220
* https://tracker.ceph.com/issues/45434
221
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
222
* https://tracker.ceph.com/issues/51440
223 17 Jeff Layton
    fallocate fails with EACCES
224 16 Jeff Layton
* https://tracker.ceph.com/issues/51264
225
    TestVolumeClient failure
226
* https://tracker.ceph.com/issues/51266
227
    test cleanup failure
228
* https://tracker.ceph.com/issues/51183
229
    Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) 
230
231 12 Jeff Layton
h3. 2021 June 14
232
233
https://pulpito.ceph.com/yuriw-2021-06-14_16:21:07-fs-wip-yuri2-testing-2021-06-14-0717-pacific-distro-basic-smithi/
234
 
235
* https://tracker.ceph.com/issues/45434
236
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
237
* https://bugzilla.redhat.com/show_bug.cgi?id=1973276
238
    Could not reconnect to ubuntu@smithi076.front.sepia.ceph.com
239
* https://tracker.ceph.com/issues/51263
240
    pjdfstest rename test 10.t failed with EACCES
241
* https://tracker.ceph.com/issues/51264
242
    TestVolumeClient failure
243 13 Jeff Layton
* https://tracker.ceph.com/issues/51266
244
    Command failed on smithi204 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 
245 14 Jeff Layton
* https://tracker.ceph.com/issues/50279
246
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
247 15 Jeff Layton
* https://tracker.ceph.com/issues/51267
248
    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status 1
249 11 Patrick Donnelly
250
h3. 2021 June 07 (Integration Branch)
251
252
http://pulpito.front.sepia.ceph.com/?branch=wip-yuri4-testing-2021-06-07-0955-pacific
253
254
* https://tracker.ceph.com/issues/45434
255
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
256
* https://tracker.ceph.com/issues/50279
257
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
258
* https://tracker.ceph.com/issues/48773
259
    qa: scrub does not complete
260
* https://tracker.ceph.com/issues/51170
261
    pacific: qa: AttributeError: 'RemoteProcess' object has no attribute 'split'
262
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
263
    qa: quota failure
264
265
266 6 Patrick Donnelly
h3. 2021 Apr 28 (QE pre-release)
267
268
https://pulpito.ceph.com/yuriw-2021-04-28_19:27:30-fs-pacific-distro-basic-smithi/
269
https://pulpito.ceph.com/yuriw-2021-04-30_13:46:58-fs-pacific-distro-basic-smithi/
270
271
* https://tracker.ceph.com/issues/45434
272
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
273
* https://tracker.ceph.com/issues/50258
274
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
275
* https://tracker.ceph.com/issues/50260
276
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
277
* https://tracker.ceph.com/issues/49962
278
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
279
* https://tracker.ceph.com/issues/50016
280
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"
281
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
282
    qa: quota failure
283
* https://tracker.ceph.com/issues/50528
284
    pacific: qa: fs:thrash: pjd suite not ok 20
285
286
287 3 Patrick Donnelly
h3. 2021 Apr 22 (Integration Branch)
288
289
https://pulpito.ceph.com/yuriw-2021-04-22_16:46:11-fs-wip-yuri5-testing-2021-04-20-0819-pacific-distro-basic-smithi/
290
291
* https://tracker.ceph.com/issues/50527
292
    pacific: qa: untar_snap_rm timeout during osd thrashing (ceph-fuse)
293
* https://tracker.ceph.com/issues/50528
294
    pacific: qa: fs:thrash: pjd suite not ok 20
295
* https://tracker.ceph.com/issues/49500 (fixed in another integration run)
296
    qa: "Assertion `cb_done' failed."
297
* https://tracker.ceph.com/issues/49500
298
    qa: "Assertion `cb_done' failed."
299
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
300
    qa: quota failure
301
* https://tracker.ceph.com/issues/45434
302
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
303
* https://tracker.ceph.com/issues/50279
304
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
305
* https://tracker.ceph.com/issues/50258
306
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
307
* https://tracker.ceph.com/issues/49962
308
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
309
* https://tracker.ceph.com/issues/49962
310
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
311
* https://tracker.ceph.com/issues/50530
312
    pacific: client: abort after MDS blocklist
313
314 2 Patrick Donnelly
315
h3. 2021 Apr 21 (Integration Branch)
316
317
https://pulpito.ceph.com/yuriw-2021-04-21_19:25:00-fs-wip-yuri3-testing-2021-04-21-0937-pacific-distro-basic-smithi/
318
319
* https://tracker.ceph.com/issues/45434
320
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
321
* https://tracker.ceph.com/issues/50250
322
    mds: "log [WRN] : Scrub error on inode 0x10000004506 (/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.a log and `damage ls` output for details"
323
* https://tracker.ceph.com/issues/50258
324
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
325
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
326
    qa: quota failure
327
* https://tracker.ceph.com/issues/50016
328
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"
329
* https://tracker.ceph.com/issues/50495
330
    pacific: client: shutdown race fails with status 141
331
332
333 1 Patrick Donnelly
h3. 2021 Apr 07 (Integration Branch)
334
335
https://pulpito.ceph.com/yuriw-2021-04-07_17:37:43-fs-wip-yuri-testing-2021-04-07-0905-pacific-distro-basic-smithi/
336
337
* https://tracker.ceph.com/issues/45434
338
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
339
* https://tracker.ceph.com/issues/48805
340
    mds: "cluster [WRN] Scrub error on inode 0x1000000039d (/client.0/tmp/blogbench-1.0/src/blogtest_in) see mds.a log and `damage ls` output for details"
341
* https://tracker.ceph.com/issues/49500
342
    qa: "Assertion `cb_done' failed."
343
* https://tracker.ceph.com/issues/50258 (new)
344
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
345
* https://tracker.ceph.com/issues/49962
346
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
347
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
348
    qa: quota failure
349
* https://tracker.ceph.com/issues/50260
350
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
351
* https://tracker.ceph.com/issues/50016
352
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"