Project

General

Profile

Pacific » History » Version 83

Venky Shankar, 12/07/2022 11:48 AM

1 1 Patrick Donnelly
h1. Pacific
2
3 4 Patrick Donnelly
h2. On-call Schedule
4
5 81 Venky Shankar
* Dec: Venky
6
* Jan: Patrick
7
* Feb: Jos
8 79 Venky Shankar
* Mar: Xiubo
9
* Apr: Rishabh
10
* May: Kotresh
11 81 Venky Shankar
* Jun: Milind
12
13 38 Ramana Raja
14 1 Patrick Donnelly
h2. Reviews
15 40 Ramana Raja
16 83 Venky Shankar
h3. 2022 Dec 02
17
18
many transient git.ceph.com related timeouts
19
many transient 'Failed to connect to the host via ssh' failures
20
21
* https://tracker.ceph.com/issues/57723
22
  pacific: qa: test_subvolume_snapshot_info_if_orphan_clone fails
23
* https://tracker.ceph.com/issues/52624
24
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
25
26 82 Venky Shankar
h3. 2022 Dec 01
27
28
many transient git.ceph.com related timeouts
29
30
* https://tracker.ceph.com/issues/57723
31
  pacific: qa: test_subvolume_snapshot_info_if_orphan_clone fails
32
* https://tracker.ceph.com/issues/52624
33
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
34
35 80 Patrick Donnelly
h3. 2022 Nov 18
36
37
https://trello.com/c/ecysAxl6/1656-wip-yuri-testing-2022-11-18-1500-pacific-old-wip-yuri-testing-2022-10-23-0729-pacific-old-wip-yuri-testing-2022-10-21-0927-pacif
38
https://pulpito.ceph.com/yuriw-2022-11-28_16:28:56-fs-wip-yuri-testing-2022-11-18-1500-pacific-distro-default-smithi/
39
40
2 ansible dead failures.
41
12 transient git.ceph.com related timeouts
42
43
* https://tracker.ceph.com/issues/57723
44
  pacific: qa: test_subvolume_snapshot_info_if_orphan_clone fails
45
* https://tracker.ceph.com/issues/52624
46
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
47
48 78 Milind Changire
h3. 2022 Oct 19
49
50
http://pulpito.front.sepia.ceph.com/yuriw-2022-10-10_17:18:40-fs-wip-yuri5-testing-2022-10-10-0837-pacific-distro-default-smithi/
51
52
* https://tracker.ceph.com/issues/57723
53
  pacific: qa: test_subvolume_snapshot_info_if_orphan_clone fails
54
* https://tracker.ceph.com/issues/52624
55
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
56
* https://tracker.ceph.com/issues/56644
57
  qa: test_rapid_creation fails with "No space left on device"
58
* https://tracker.ceph.com/issues/54460
59
  snaptest-multiple-capsnaps.sh test failure
60
* https://tracker.ceph.com/issues/57892
61
  sudo dd of=/home/ubuntu/cephtest/valgrind.supp failure during node setup phase
62
63
64
65 76 Venky Shankar
h3. 2022 Oct 06
66
67
http://pulpito.front.sepia.ceph.com/yuriw-2022-09-21_13:07:25-rados-wip-yuri5-testing-2022-09-20-1347-pacific-distro-default-smithi/
68
69
* https://tracker.ceph.com/issues/52624
70
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
71
* https://tracker.ceph.com/issues/50223
72
	qa: "client.4737 isn't responding to mclientcaps(revoke)"
73 77 Milind Changire
* https://tracker.ceph.com/issues/56507
74
  Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
75 76 Venky Shankar
76 75 Venky Shankar
h3. 2022 Sep 27
77
78
http://pulpito.front.sepia.ceph.com/yuriw-2022-09-22_22:35:04-fs-wip-yuri2-testing-2022-09-22-1400-pacific-distro-default-smithi/
79
80
* https://tracker.ceph.com/issues/52624
81
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
82
* https://tracker.ceph.com/issues/48773
83
    Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete
84
* https://tracker.ceph.com/issues/50224
85
    test_mirroring_init_failure_with_recovery - (tasks.cephfs.test_mirroring.TestMirroring)
86
* https://tracker.ceph.com/issues/56507
87
    Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
88
* https://tracker.ceph.com/issues/50223
89
	qa: "client.4737 isn't responding to mclientcaps(revoke)"
90
91 74 Venky Shankar
h3. 2022 Sep 22
92
93
http://pulpito.front.sepia.ceph.com/?branch=wip-yuri2-testing-2022-09-06-1007-pacific
94
95
* https://tracker.ceph.com/issues/52624
96
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
97
* https://tracker.ceph.com/issues/51282
98
    cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log
99
* https://tracker.ceph.com/issues/53360
100
    pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
101
102 73 Venky Shankar
h3. 2022 Sep 19
103
104
http://pulpito.front.sepia.ceph.com/yuriw-2022-09-16_19:05:25-fs-wip-yuri11-testing-2022-09-16-0958-pacific-distro-default-smithi/
105
106
* https://tracker.ceph.com/issues/52624
107
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
108
* https://tracker.ceph.com/issues/57594
109
    pacific: Test failure: test_rebuild_moved_dir (tasks.cephfs.test_data_scan.TestDataScan)
110
111 71 Venky Shankar
h3. 2022 Sep 15
112
113
http://pulpito.front.sepia.ceph.com/yuriw-2022-09-13_14:22:44-fs-wip-yuri5-testing-2022-09-09-1109-pacific-distro-default-smithi/
114
115
* https://tracker.ceph.com/issues/51282
116
    cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log
117
* https://tracker.ceph.com/issues/52624
118
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
119
* https://tracker.ceph.com/issues/53360
120
    pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
121 72 Venky Shankar
* https://tracker.ceph.com/issues/48773
122
    Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete
123 71 Venky Shankar
124 70 Kotresh Hiremath Ravishankar
h3. 2022 AUG 18
125
126
https://pulpito.ceph.com/yuriw-2022-08-18_23:16:33-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-default-smithi/
127
* https://tracker.ceph.com/issues/52624 - cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
128
* https://tracker.ceph.com/issues/51267 - tasks/workunit/snaps failure - Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status
129
* https://tracker.ceph.com/issues/48773 - Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete
130
* https://tracker.ceph.com/issues/51964 - qa: test_cephfs_mirror_restart_sync_on_blocklist failure
131
* https://tracker.ceph.com/issues/57218 - qa: tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} fails
132
* https://tracker.ceph.com/issues/56507 - Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
133
134
135
Re-run1: https://pulpito.ceph.com/yuriw-2022-08-19_21:01:11-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-default-smithi/
136
* https://tracker.ceph.com/issues/52624 - cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
137
* https://tracker.ceph.com/issues/57219 - mds crashed while running workunit test fs/misc/dirfrag.sh
138
139
140 69 Kotresh Hiremath Ravishankar
h3. 2022 AUG 11
141
142
https://pulpito.ceph.com/yuriw-2022-08-11_16:57:01-fs-wip-yuri3-testing-2022-08-11-0809-pacific-distro-default-smithi/
143
* Most of the failures are passed in re-run. Please check rerun failures below.
144
  - https://tracker.ceph.com/issues/57147 - test_full_fsync (tasks.cephfs.test_full.TestClusterFull) - mds stuck in up:creating state - osd crash
145
  - https://tracker.ceph.com/issues/52624 - cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
146
  - https://tracker.ceph.com/issues/51282 - cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log
147
  - https://tracker.ceph.com/issues/50224 - test_mirroring_init_failure_with_recovery - (tasks.cephfs.test_mirroring.TestMirroring)
148
  - https://tracker.ceph.com/issues/57083 - https://tracker.ceph.com/issues/53360 - tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} 
149
  - https://tracker.ceph.com/issues/51183 - Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
150
  - https://tracker.ceph.com/issues/56507 - Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)   
151
152
Re-run-1: https://pulpito.ceph.com/yuriw-2022-08-15_13:45:54-fs-wip-yuri3-testing-2022-08-11-0809-pacific-distro-default-smithi/
153
* https://tracker.ceph.com/issues/52624
154
  cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
155
* tasks/mirror - Command failed on smithi078 with status 123: "sudo find /var/log/ceph -name '*.log' -print0 ..." - Asked for re-run.
156
* https://tracker.ceph.com/issues/57083
157
* https://tracker.ceph.com/issues/53360
158
  tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} - tasks.cephfs.fuse_mount:mount command failed
159
  client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
160
* https://tracker.ceph.com/issues/51183
161
  Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
162
* https://tracker.ceph.com/issues/56507
163
  Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
164
165 68 Kotresh Hiremath Ravishankar
166
h3. 2022 AUG 04
167
168
https://pulpito.ceph.com/yuriw-2022-08-04_20:54:08-fs-wip-yuri6-testing-2022-08-04-0617-pacific-distro-default-smithi/
169
170
* https://tracker.ceph.com/issues/57087
171
  test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan)
172
* https://tracker.ceph.com/issues/52624
173
  cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
174
* https://tracker.ceph.com/issues/51267
175
  tasks/workunit/snaps failure - Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status      
176
* https://tracker.ceph.com/issues/53360
177
* https://tracker.ceph.com/issues/57083
178
  qa/import-legacy tasks.cephfs.fuse_mount:mount command failed - :ModuleNotFoundError: No module named 'ceph_volume_client'
179
* https://tracker.ceph.com/issues/56507
180
  Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
181
182
183 67 Kotresh Hiremath Ravishankar
h3. 2022 July 15
184
185
https://pulpito.ceph.com/yuriw-2022-07-21_22:57:30-fs-wip-yuri2-testing-2022-07-15-0755-pacific-distro-default-smithi
186
Re-run: https://pulpito.ceph.com/yuriw-2022-07-24_15:34:38-fs-wip-yuri2-testing-2022-07-15-0755-pacific-distro-default-smithi
187
188
* https://tracker.ceph.com/issues/52624
189
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
190
* https://tracker.ceph.com/issues/57083
191
* https://tracker.ceph.com/issues/53360
192
        tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} - tasks.cephfs.fuse_mount:mount command failed
193
        client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" 
194
* https://tracker.ceph.com/issues/51183
195
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
196
* https://tracker.ceph.com/issues/56507
197
        pacific: Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
198
199
200 66 Venky Shankar
h3. 2022 July 08
201
202
* https://tracker.ceph.com/issues/52624
203
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
204
* https://tracker.ceph.com/issues/53360
205
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
206
* https://tracker.ceph.com/issues/51183
207
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
208
* https://tracker.ceph.com/issues/56506
209
        pacific: Test failure: test_rebuild_backtraceless (tasks.cephfs.test_data_scan.TestDataScan)
210
* https://tracker.ceph.com/issues/56507
211
        pacific: Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
212
213 65 Venky Shankar
h3. 2022 Jun 28
214
215
https://pulpito.ceph.com/?branch=wip-yuri3-testing-2022-06-22-1121-pacific
216
217
* https://tracker.ceph.com/issues/52624
218
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
219
* https://tracker.ceph.com/issues/53360
220
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
221
* https://tracker.ceph.com/issues/51183
222
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
223
224 64 Venky Shankar
h3. 2022 Jun 22
225
226
https://pulpito.ceph.com/?branch=wip-yuri4-testing-2022-06-21-0704-pacific
227
228
* https://tracker.ceph.com/issues/52624
229
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
230
* https://tracker.ceph.com/issues/53360
231
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
232
* https://tracker.ceph.com/issues/51183
233
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
234
235 63 Venky Shankar
h3. 2022 Jun 17
236
237
https://pulpito.ceph.com/yuriw-2022-06-15_17:11:32-fs-wip-yuri10-testing-2022-06-15-0732-pacific-distro-default-smithi/
238
239
* https://tracker.ceph.com/issues/52624
240
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
241
* https://tracker.ceph.com/issues/53360
242
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
243
* https://tracker.ceph.com/issues/51183
244
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
245
246 62 Venky Shankar
h3. 2022 Jun 16
247
248
https://pulpito.ceph.com/yuriw-2022-06-15_17:08:32-fs-wip-yuri-testing-2022-06-14-0744-pacific-distro-default-smithi/
249
250
* https://tracker.ceph.com/issues/52624
251
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
252
* https://tracker.ceph.com/issues/53360
253
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
254
* https://tracker.ceph.com/issues/55449
255
        pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh
256
* https://tracker.ceph.com/issues/51267
257
        CommandFailedError: Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status 1:...
258
* https://tracker.ceph.com/issues/55332
259
        Failure in snaptest-git-ceph.sh (it's an async unlink/create bug)
260
261 61 Venky Shankar
h3. 2022 Jun 15
262
263
http://pulpito.front.sepia.ceph.com/yuriw-2022-06-09_13:32:29-fs-wip-yuri10-testing-2022-06-08-0730-pacific-distro-default-smithi/
264
265
* https://tracker.ceph.com/issues/52624
266
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
267
* https://tracker.ceph.com/issues/53360
268
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
269
* https://tracker.ceph.com/issues/55449
270
        pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh
271
272
273 60 Venky Shankar
h3. 2022 Jun 10
274
275
https://pulpito.ceph.com/?branch=wip-yuri5-testing-2022-06-07-1529-pacific
276
277
* https://tracker.ceph.com/issues/52624
278
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
279
* https://tracker.ceph.com/issues/53360
280
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
281
* https://tracker.ceph.com/issues/55449
282
        pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh
283
284 59 Venky Shankar
h3. 2022 Jun 09
285
286
https://pulpito.ceph.com/yuriw-2022-06-07_16:05:25-fs-wip-yuri4-testing-2022-06-01-1350-pacific-distro-default-smithi/
287
288
* https://tracker.ceph.com/issues/52624
289
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
290
* https://tracker.ceph.com/issues/53360
291
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
292
* https://tracker.ceph.com/issues/55449
293
        pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh
294
295 58 Venky Shankar
h3. 2022 May 06
296
297
https://pulpito.ceph.com/?sha1=73636a1b00037ff974bcdc969b009c5ecec626cc
298
299
* https://tracker.ceph.com/issues/52624
300
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
301
* https://tracker.ceph.com/issues/53360
302
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
303
304 57 Venky Shankar
h3. 2022 April 18
305
306
http://pulpito.front.sepia.ceph.com/?branch=wip-yuri4-testing-2022-04-18-0609-pacific
307
(only mgr/snap_schedule backport pr)
308
309
* https://tracker.ceph.com/issues/52624
310
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
311 56 Venky Shankar
312
h3. 2022 March 28
313
314
http://pulpito.front.sepia.ceph.com/yuriw-2022-03-25_20:52:40-fs-wip-yuri4-testing-2022-03-25-1203-pacific-distro-default-smithi/
315
http://pulpito.front.sepia.ceph.com/yuriw-2022-03-26_19:52:48-fs-wip-yuri4-testing-2022-03-25-1203-pacific-distro-default-smithi/
316
317
* https://tracker.ceph.com/issues/52624
318
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
319
* https://tracker.ceph.com/issues/53360
320
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
321
* https://tracker.ceph.com/issues/54411
322
	mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available;
323
	33 daemons have recently crashed" during suites/fsstress.sh
324
325 55 Venky Shankar
h3. 2022 March 25
326
327
https://pulpito.ceph.com/yuriw-2022-03-22_18:42:28-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/
328
https://pulpito.ceph.com/yuriw-2022-03-23_14:04:56-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/
329
330
* https://tracker.ceph.com/issues/52624
331
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
332
* https://tracker.ceph.com/issues/52606
333
        qa: test_dirfrag_limit
334
* https://tracker.ceph.com/issues/51905
335
        qa: "error reading sessionmap 'mds1_sessionmap'"
336
* https://tracker.ceph.com/issues/53360
337
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
338
* https://tracker.ceph.com/issues/51183
339
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
340 47 Patrick Donnelly
341 54 Venky Shankar
h3. 2022 March 22
342
343
https://pulpito.ceph.com/yuriw-2022-03-17_15:03:06-fs-wip-yuri10-testing-2022-03-16-1432-pacific-distro-default-smithi/
344
345
* https://tracker.ceph.com/issues/52624
346
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
347
* https://tracker.ceph.com/issues/52606
348
        qa: test_dirfrag_limit
349
* https://tracker.ceph.com/issues/51183
350
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
351
* https://tracker.ceph.com/issues/51905
352
        qa: "error reading sessionmap 'mds1_sessionmap'"
353
* https://tracker.ceph.com/issues/53360
354
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
355
* https://tracker.ceph.com/issues/54411
356
	mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available;
357
	33 daemons have recently crashed" during suites/fsstress.sh
358 47 Patrick Donnelly
359 46 Kotresh Hiremath Ravishankar
h3. 2021 November 22
360
361
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-12_00:33:28-fs-wip-yuri7-testing-2021-11-11-1339-pacific-distro-basic-smithi
362
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-08_20:21:11-fs-wip-yuri5-testing-2021-11-08-1003-pacific-distro-basic-smithi
363
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-08_15:19:37-fs-wip-yuri2-testing-2021-11-06-1322-pacific-distro-basic-smithi
364
365
366
* https://tracker.ceph.com/issues/53300
367
	qa: cluster [WRN] Scrub error on inode
368
* https://tracker.ceph.com/issues/53302
369
	qa: sudo logrotate /etc/logrotate.d/ceph-test.conf failed with status 1
370
* https://tracker.ceph.com/issues/53314
371
	qa: fs/upgrade/mds_upgrade_sequence test timeout
372
* https://tracker.ceph.com/issues/53316
373
	qa: (smithi150) slow request osd_op, currently waiting for sub ops warning
374
* https://tracker.ceph.com/issues/52624
375
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
376
* https://tracker.ceph.com/issues/52396
377
	pacific: qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
378
* https://tracker.ceph.com/issues/52875
379
	pacific: qa: test_dirfrag_limit
380
* https://tracker.ceph.com/issues/51705
381 1 Patrick Donnelly
	pacific: qa: tasks.cephfs.fuse_mount:mount command failed
382
* https://tracker.ceph.com/issues/39634
383
	qa: test_full_same_file timeout
384 46 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/49748
385
	gibba: Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds
386 48 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/51964
387
	qa: test_cephfs_mirror_restart_sync_on_blocklist failure
388 49 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/50223
389
	qa: "client.4737 isn't responding to mclientcaps(revoke)"
390 48 Kotresh Hiremath Ravishankar
391 47 Patrick Donnelly
392
h3. 2021 November 20
393
394
https://pulpito.ceph.com/?branch=wip-pdonnell-testing-20211120.024903-pacific
395
396
* https://tracker.ceph.com/issues/53360
397
    pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
398 46 Kotresh Hiremath Ravishankar
399 41 Patrick Donnelly
h3. 2021 September 14 (QE)
400
401
https://pulpito.ceph.com/yuriw-2021-09-10_15:01:12-fs-pacific-distro-basic-smithi/
402 1 Patrick Donnelly
http://pulpito.front.sepia.ceph.com/yuriw-2021-09-13_14:09:59-fs-pacific-distro-basic-smithi/
403 43 Ramana Raja
https://pulpito.ceph.com/yuriw-2021-09-14_15:09:45-fs-pacific-distro-basic-smithi/
404 41 Patrick Donnelly
405
* https://tracker.ceph.com/issues/52606
406
    qa: test_dirfrag_limit
407
* https://tracker.ceph.com/issues/52607
408
    qa: "mon.a (mon.0) 1022 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY)"
409 44 Ramana Raja
* https://tracker.ceph.com/issues/51705
410
    qa: tasks.cephfs.fuse_mount:mount command failed
411 41 Patrick Donnelly
412 40 Ramana Raja
h3. 2021 Sep 7
413
414
https://pulpito.ceph.com/yuriw-2021-09-07_17:38:38-fs-wip-yuri-testing-2021-09-07-0905-pacific-distro-basic-smithi/
415
https://pulpito.ceph.com/yuriw-2021-09-07_23:57:28-fs-wip-yuri-testing-2021-09-07-0905-pacific-distro-basic-smithi/
416
417
418
* https://tracker.ceph.com/issues/52396
419
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
420
* https://tracker.ceph.com/issues/51705
421
    qa: tasks.cephfs.fuse_mount:mount command failed
422 4 Patrick Donnelly
423 34 Ramana Raja
h3. 2021 Aug 30
424
425
https://pulpito.ceph.com/?branch=wip-yuri8-testing-2021-08-30-0930-pacific
426
427
* https://tracker.ceph.com/issues/45434
428
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
429
* https://tracker.ceph.com/issues/51705
430
    qa: tasks.cephfs.fuse_mount:mount command failed
431
* https://tracker.ceph.com/issues/52396
432
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
433 35 Ramana Raja
* https://tracker.ceph.com/issues/52487
434 36 Ramana Raja
    qa: Test failure: test_deep_split (tasks.cephfs.test_fragment.TestFragmentation)
435
* https://tracker.ceph.com/issues/51267
436 37 Ramana Raja
   Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh)
437
* https://tracker.ceph.com/issues/48772
438
   qa: pjd: not ok 9, 44, 80
439 35 Ramana Raja
440 34 Ramana Raja
441 31 Ramana Raja
h3. 2021 Aug 23
442
443
https://pulpito.ceph.com/yuriw-2021-08-23_19:33:26-fs-wip-yuri4-testing-2021-08-23-0812-pacific-distro-basic-smithi/
444
445
* https://tracker.ceph.com/issues/45434
446
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
447
* https://tracker.ceph.com/issues/51705
448
    qa: tasks.cephfs.fuse_mount:mount command failed
449 32 Ramana Raja
* https://tracker.ceph.com/issues/52396
450 33 Ramana Raja
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
451
* https://tracker.ceph.com/issues/52397
452
    qa: test_acls (tasks.cephfs.test_acls.TestACLs) failed
453
454 31 Ramana Raja
455 29 Ramana Raja
h3. 2021 Aug 11
456
457
https://pulpito.ceph.com/yuriw-2021-08-11_14:28:23-fs-wip-yuri8-testing-2021-08-09-0844-pacific-distro-basic-smithi/
458
https://pulpito.ceph.com/yuriw-2021-08-12_15:32:35-fs-wip-yuri8-testing-2021-08-09-0844-pacific-distro-basic-smithi/
459
460
* https://tracker.ceph.com/issues/45434
461
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
462
* https://tracker.ceph.com/issues/51705
463
    qa: tasks.cephfs.fuse_mount:mount command failed
464 30 Ramana Raja
* https://tracker.ceph.com/issues/50222
465
    osd: 5.2s0 deep-scrub : stat mismatch
466 29 Ramana Raja
467 19 Jos Collin
h3. 2021 July 15
468 20 Jos Collin
469 19 Jos Collin
https://pulpito.ceph.com/yuriw-2021-07-13_17:37:59-fs-wip-yuri-testing-2021-07-13-0812-pacific-distro-basic-smithi/
470
471 26 Jos Collin
* https://tracker.ceph.com/issues/45434
472
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
473
* https://tracker.ceph.com/issues/51705
474
    qa: tasks.cephfs.fuse_mount:mount command failed
475 27 Jos Collin
* https://tracker.ceph.com/issues/51183
476
    Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
477 28 Jos Collin
* https://tracker.ceph.com/issues/50528
478
    qa: fs:thrash: pjd suite not ok 80
479
* https://tracker.ceph.com/issues/51706
480
    qa: osd deep-scrub stat mismatch
481 26 Jos Collin
482 19 Jos Collin
h3. 2021 July 13
483 20 Jos Collin
484 19 Jos Collin
https://pulpito.ceph.com/yuriw-2021-07-08_23:33:26-fs-wip-yuri2-testing-2021-07-08-1142-pacific-distro-basic-smithi/
485
486 21 Jos Collin
* https://tracker.ceph.com/issues/51704
487
    Test failure: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth)
488 23 Jos Collin
* https://tracker.ceph.com/issues/45434
489
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
490 24 Jos Collin
* https://tracker.ceph.com/issues/51705
491 25 Jos Collin
    qa: tasks.cephfs.fuse_mount:mount command failed
492
* https://tracker.ceph.com/issues/48640
493 24 Jos Collin
    qa: snapshot mismatch during mds thrashing
494 21 Jos Collin
495 18 Patrick Donnelly
h3. 2021 June 29 (Integration Branch)
496
497
https://pulpito.ceph.com/yuriw-2021-06-29_00:15:33-fs-wip-yuri3-testing-2021-06-28-1259-pacific-distro-basic-smithi/
498
499
Some failures caused by incomplete backup of: https://github.com/ceph/ceph/pull/42065
500
Some package failures caused by nautilus missing package, like: https://pulpito.ceph.com/yuriw-2021-06-29_00:15:33-fs-wip-yuri3-testing-2021-06-28-1259-pacific-distro-basic-smithi/6241300/
501
502
* https://tracker.ceph.com/issues/45434
503
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
504
* https://tracker.ceph.com/issues/50260
505
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
506
* https://tracker.ceph.com/issues/51183
507
    qa: FileNotFoundError: [Errno 2] No such file or directory: '/sys/kernel/debug/ceph/3fab6bea-f243-47a4-a956-8c03a62b61b5.client4721/mds_sessions'
508
509
510 16 Jeff Layton
h3. 2021 June 28
511
512
https://pulpito.ceph.com/yuriw-2021-06-29_00:14:14-fs-wip-yuri-testing-2021-06-28-1259-pacific-distro-basic-smithi/
513
514
* https://tracker.ceph.com/issues/45434
515
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
516
* https://tracker.ceph.com/issues/51440
517 17 Jeff Layton
    fallocate fails with EACCES
518 16 Jeff Layton
* https://tracker.ceph.com/issues/51264
519
    TestVolumeClient failure
520
* https://tracker.ceph.com/issues/51266
521
    test cleanup failure
522
* https://tracker.ceph.com/issues/51183
523
    Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) 
524
525 12 Jeff Layton
h3. 2021 June 14
526
527
https://pulpito.ceph.com/yuriw-2021-06-14_16:21:07-fs-wip-yuri2-testing-2021-06-14-0717-pacific-distro-basic-smithi/
528
 
529
* https://tracker.ceph.com/issues/45434
530
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
531
* https://bugzilla.redhat.com/show_bug.cgi?id=1973276
532
    Could not reconnect to ubuntu@smithi076.front.sepia.ceph.com
533
* https://tracker.ceph.com/issues/51263
534
    pjdfstest rename test 10.t failed with EACCES
535
* https://tracker.ceph.com/issues/51264
536
    TestVolumeClient failure
537 13 Jeff Layton
* https://tracker.ceph.com/issues/51266
538
    Command failed on smithi204 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 
539 14 Jeff Layton
* https://tracker.ceph.com/issues/50279
540
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
541 15 Jeff Layton
* https://tracker.ceph.com/issues/51267
542
    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status 1
543 11 Patrick Donnelly
544
h3. 2021 June 07 (Integration Branch)
545
546
http://pulpito.front.sepia.ceph.com/?branch=wip-yuri4-testing-2021-06-07-0955-pacific
547
548
* https://tracker.ceph.com/issues/45434
549
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
550
* https://tracker.ceph.com/issues/50279
551
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
552
* https://tracker.ceph.com/issues/48773
553
    qa: scrub does not complete
554
* https://tracker.ceph.com/issues/51170
555
    pacific: qa: AttributeError: 'RemoteProcess' object has no attribute 'split'
556
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
557
    qa: quota failure
558
559
560 6 Patrick Donnelly
h3. 2021 Apr 28 (QE pre-release)
561
562
https://pulpito.ceph.com/yuriw-2021-04-28_19:27:30-fs-pacific-distro-basic-smithi/
563
https://pulpito.ceph.com/yuriw-2021-04-30_13:46:58-fs-pacific-distro-basic-smithi/
564
565
* https://tracker.ceph.com/issues/45434
566
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
567
* https://tracker.ceph.com/issues/50258
568
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
569
* https://tracker.ceph.com/issues/50260
570
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
571
* https://tracker.ceph.com/issues/49962
572
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
573
* https://tracker.ceph.com/issues/50016
574
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"
575
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
576
    qa: quota failure
577
* https://tracker.ceph.com/issues/50528
578
    pacific: qa: fs:thrash: pjd suite not ok 20
579
580
581 3 Patrick Donnelly
h3. 2021 Apr 22 (Integration Branch)
582
583
https://pulpito.ceph.com/yuriw-2021-04-22_16:46:11-fs-wip-yuri5-testing-2021-04-20-0819-pacific-distro-basic-smithi/
584
585
* https://tracker.ceph.com/issues/50527
586
    pacific: qa: untar_snap_rm timeout during osd thrashing (ceph-fuse)
587
* https://tracker.ceph.com/issues/50528
588
    pacific: qa: fs:thrash: pjd suite not ok 20
589
* https://tracker.ceph.com/issues/49500 (fixed in another integration run)
590
    qa: "Assertion `cb_done' failed."
591
* https://tracker.ceph.com/issues/49500
592
    qa: "Assertion `cb_done' failed."
593
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
594
    qa: quota failure
595
* https://tracker.ceph.com/issues/45434
596
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
597
* https://tracker.ceph.com/issues/50279
598
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
599
* https://tracker.ceph.com/issues/50258
600
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
601
* https://tracker.ceph.com/issues/49962
602
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
603
* https://tracker.ceph.com/issues/49962
604
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
605
* https://tracker.ceph.com/issues/50530
606
    pacific: client: abort after MDS blocklist
607
608 2 Patrick Donnelly
609
h3. 2021 Apr 21 (Integration Branch)
610
611
https://pulpito.ceph.com/yuriw-2021-04-21_19:25:00-fs-wip-yuri3-testing-2021-04-21-0937-pacific-distro-basic-smithi/
612
613
* https://tracker.ceph.com/issues/45434
614
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
615
* https://tracker.ceph.com/issues/50250
616
    mds: "log [WRN] : Scrub error on inode 0x10000004506 (/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.a log and `damage ls` output for details"
617
* https://tracker.ceph.com/issues/50258
618
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
619
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
620
    qa: quota failure
621
* https://tracker.ceph.com/issues/50016
622
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"
623
* https://tracker.ceph.com/issues/50495
624
    pacific: client: shutdown race fails with status 141
625
626
627 1 Patrick Donnelly
h3. 2021 Apr 07 (Integration Branch)
628
629
https://pulpito.ceph.com/yuriw-2021-04-07_17:37:43-fs-wip-yuri-testing-2021-04-07-0905-pacific-distro-basic-smithi/
630
631
* https://tracker.ceph.com/issues/45434
632
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
633
* https://tracker.ceph.com/issues/48805
634
    mds: "cluster [WRN] Scrub error on inode 0x1000000039d (/client.0/tmp/blogbench-1.0/src/blogtest_in) see mds.a log and `damage ls` output for details"
635
* https://tracker.ceph.com/issues/49500
636
    qa: "Assertion `cb_done' failed."
637
* https://tracker.ceph.com/issues/50258 (new)
638
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
639
* https://tracker.ceph.com/issues/49962
640
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
641
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
642
    qa: quota failure
643
* https://tracker.ceph.com/issues/50260
644
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
645
* https://tracker.ceph.com/issues/50016
646
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"