Project

General

Profile

Pacific » History » Version 89

Rishabh Dave, 05/04/2023 11:00 AM

1 1 Patrick Donnelly
h1. Pacific
2
3 4 Patrick Donnelly
h2. On-call Schedule
4
5 81 Venky Shankar
* Dec: Venky
6
* Jan: Patrick
7
* Feb: Jos
8 79 Venky Shankar
* Mar: Xiubo
9
* Apr: Rishabh
10
* May: Kotresh
11 81 Venky Shankar
* Jun: Milind
12
13 1 Patrick Donnelly
h2. Reviews
14 85 Rishabh Dave
15 87 Rishabh Dave
h3. NEW ENTRY
16
17 86 Rishabh Dave
h3. 2023 Apr 13
18
19
https://pulpito.ceph.com/yuriw-2023-04-04_15:06:57-fs-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/
20
21
https://tracker.ceph.com/issues/52624
22
cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
23
* https://tracker.ceph.com/issues/57594
24
  Test failure: test_rebuild_moved_dir (tasks.cephfs.test_data_scan.TestDataScan)
25
* https://tracker.ceph.com/issues/54108
26
  qa: iogen workunit: "The following counters failed to be set on mds daemons: {'mds.exported', 'mds.imported'}" 
27
* https://tracker.ceph.com/issues/58340
28
  fsstress.sh failed with errno 125
29
* https://tracker.ceph.com/issues/54462
30
  Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi055 with status 128
31
* https://tracker.ceph.com/issues/49287     
32
  cephadm: podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found
33
* https://tracker.ceph.com/issues/58726
34
  test_acls: expected a yum based or a apt based system
35 40 Ramana Raja
36 84 Venky Shankar
h3. 2022 Dec 07
37
38
https://pulpito.ceph.com/yuriw-2022-12-07_15:45:30-fs-wip-yuri4-testing-2022-12-05-0921-pacific-distro-default-smithi/
39
40
many transient git.ceph.com related timeouts
41
42
* https://tracker.ceph.com/issues/52624
43
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
44
* https://tracker.ceph.com/issues/50224
45
    test_mirroring_init_failure_with_recovery - (tasks.cephfs.test_mirroring.TestMirroring)
46
* https://tracker.ceph.com/issues/56644
47
    qa: test_rapid_creation fails with "No space left on device"
48
* https://tracker.ceph.com/issues/58221
49
    pacific: Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan)
50
51 83 Venky Shankar
h3. 2022 Dec 02
52
53
many transient git.ceph.com related timeouts
54
many transient 'Failed to connect to the host via ssh' failures
55
56
* https://tracker.ceph.com/issues/57723
57
  pacific: qa: test_subvolume_snapshot_info_if_orphan_clone fails
58
* https://tracker.ceph.com/issues/52624
59
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
60
61 82 Venky Shankar
h3. 2022 Dec 01
62
63
many transient git.ceph.com related timeouts
64
65
* https://tracker.ceph.com/issues/57723
66
  pacific: qa: test_subvolume_snapshot_info_if_orphan_clone fails
67
* https://tracker.ceph.com/issues/52624
68
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
69
70 80 Patrick Donnelly
h3. 2022 Nov 18
71
72
https://trello.com/c/ecysAxl6/1656-wip-yuri-testing-2022-11-18-1500-pacific-old-wip-yuri-testing-2022-10-23-0729-pacific-old-wip-yuri-testing-2022-10-21-0927-pacif
73
https://pulpito.ceph.com/yuriw-2022-11-28_16:28:56-fs-wip-yuri-testing-2022-11-18-1500-pacific-distro-default-smithi/
74
75
2 ansible dead failures.
76
12 transient git.ceph.com related timeouts
77
78
* https://tracker.ceph.com/issues/57723
79
  pacific: qa: test_subvolume_snapshot_info_if_orphan_clone fails
80
* https://tracker.ceph.com/issues/52624
81
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
82
83 78 Milind Changire
h3. 2022 Oct 19
84
85
http://pulpito.front.sepia.ceph.com/yuriw-2022-10-10_17:18:40-fs-wip-yuri5-testing-2022-10-10-0837-pacific-distro-default-smithi/
86
87
* https://tracker.ceph.com/issues/57723
88
  pacific: qa: test_subvolume_snapshot_info_if_orphan_clone fails
89
* https://tracker.ceph.com/issues/52624
90
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
91
* https://tracker.ceph.com/issues/56644
92
  qa: test_rapid_creation fails with "No space left on device"
93
* https://tracker.ceph.com/issues/54460
94
  snaptest-multiple-capsnaps.sh test failure
95
* https://tracker.ceph.com/issues/57892
96
  sudo dd of=/home/ubuntu/cephtest/valgrind.supp failure during node setup phase
97
98
99
100 76 Venky Shankar
h3. 2022 Oct 06
101
102
http://pulpito.front.sepia.ceph.com/yuriw-2022-09-21_13:07:25-rados-wip-yuri5-testing-2022-09-20-1347-pacific-distro-default-smithi/
103
104
* https://tracker.ceph.com/issues/52624
105
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
106
* https://tracker.ceph.com/issues/50223
107
	qa: "client.4737 isn't responding to mclientcaps(revoke)"
108 77 Milind Changire
* https://tracker.ceph.com/issues/56507
109
  Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
110 76 Venky Shankar
111 75 Venky Shankar
h3. 2022 Sep 27
112
113
http://pulpito.front.sepia.ceph.com/yuriw-2022-09-22_22:35:04-fs-wip-yuri2-testing-2022-09-22-1400-pacific-distro-default-smithi/
114
115
* https://tracker.ceph.com/issues/52624
116
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
117
* https://tracker.ceph.com/issues/48773
118
    Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete
119
* https://tracker.ceph.com/issues/50224
120
    test_mirroring_init_failure_with_recovery - (tasks.cephfs.test_mirroring.TestMirroring)
121
* https://tracker.ceph.com/issues/56507
122
    Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
123
* https://tracker.ceph.com/issues/50223
124
	qa: "client.4737 isn't responding to mclientcaps(revoke)"
125
126 74 Venky Shankar
h3. 2022 Sep 22
127
128
http://pulpito.front.sepia.ceph.com/?branch=wip-yuri2-testing-2022-09-06-1007-pacific
129
130
* https://tracker.ceph.com/issues/52624
131
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
132
* https://tracker.ceph.com/issues/51282
133
    cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log
134
* https://tracker.ceph.com/issues/53360
135
    pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
136
137 73 Venky Shankar
h3. 2022 Sep 19
138
139
http://pulpito.front.sepia.ceph.com/yuriw-2022-09-16_19:05:25-fs-wip-yuri11-testing-2022-09-16-0958-pacific-distro-default-smithi/
140
141
* https://tracker.ceph.com/issues/52624
142
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
143
* https://tracker.ceph.com/issues/57594
144
    pacific: Test failure: test_rebuild_moved_dir (tasks.cephfs.test_data_scan.TestDataScan)
145
146 71 Venky Shankar
h3. 2022 Sep 15
147
148
http://pulpito.front.sepia.ceph.com/yuriw-2022-09-13_14:22:44-fs-wip-yuri5-testing-2022-09-09-1109-pacific-distro-default-smithi/
149
150
* https://tracker.ceph.com/issues/51282
151
    cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log
152
* https://tracker.ceph.com/issues/52624
153
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
154
* https://tracker.ceph.com/issues/53360
155
    pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
156 72 Venky Shankar
* https://tracker.ceph.com/issues/48773
157
    Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete
158 71 Venky Shankar
159 70 Kotresh Hiremath Ravishankar
h3. 2022 AUG 18
160
161
https://pulpito.ceph.com/yuriw-2022-08-18_23:16:33-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-default-smithi/
162
* https://tracker.ceph.com/issues/52624 - cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
163
* https://tracker.ceph.com/issues/51267 - tasks/workunit/snaps failure - Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status
164
* https://tracker.ceph.com/issues/48773 - Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete
165
* https://tracker.ceph.com/issues/51964 - qa: test_cephfs_mirror_restart_sync_on_blocklist failure
166
* https://tracker.ceph.com/issues/57218 - qa: tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} fails
167
* https://tracker.ceph.com/issues/56507 - Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
168
169
170
Re-run1: https://pulpito.ceph.com/yuriw-2022-08-19_21:01:11-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-default-smithi/
171
* https://tracker.ceph.com/issues/52624 - cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
172
* https://tracker.ceph.com/issues/57219 - mds crashed while running workunit test fs/misc/dirfrag.sh
173
174
175 69 Kotresh Hiremath Ravishankar
h3. 2022 AUG 11
176
177
https://pulpito.ceph.com/yuriw-2022-08-11_16:57:01-fs-wip-yuri3-testing-2022-08-11-0809-pacific-distro-default-smithi/
178
* Most of the failures are passed in re-run. Please check rerun failures below.
179
  - https://tracker.ceph.com/issues/57147 - test_full_fsync (tasks.cephfs.test_full.TestClusterFull) - mds stuck in up:creating state - osd crash
180
  - https://tracker.ceph.com/issues/52624 - cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
181
  - https://tracker.ceph.com/issues/51282 - cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log
182
  - https://tracker.ceph.com/issues/50224 - test_mirroring_init_failure_with_recovery - (tasks.cephfs.test_mirroring.TestMirroring)
183
  - https://tracker.ceph.com/issues/57083 - https://tracker.ceph.com/issues/53360 - tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} 
184
  - https://tracker.ceph.com/issues/51183 - Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
185
  - https://tracker.ceph.com/issues/56507 - Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)   
186
187
Re-run-1: https://pulpito.ceph.com/yuriw-2022-08-15_13:45:54-fs-wip-yuri3-testing-2022-08-11-0809-pacific-distro-default-smithi/
188
* https://tracker.ceph.com/issues/52624
189
  cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
190
* tasks/mirror - Command failed on smithi078 with status 123: "sudo find /var/log/ceph -name '*.log' -print0 ..." - Asked for re-run.
191
* https://tracker.ceph.com/issues/57083
192
* https://tracker.ceph.com/issues/53360
193
  tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} - tasks.cephfs.fuse_mount:mount command failed
194
  client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
195
* https://tracker.ceph.com/issues/51183
196
  Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
197
* https://tracker.ceph.com/issues/56507
198
  Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
199
200 68 Kotresh Hiremath Ravishankar
201
h3. 2022 AUG 04
202
203
https://pulpito.ceph.com/yuriw-2022-08-04_20:54:08-fs-wip-yuri6-testing-2022-08-04-0617-pacific-distro-default-smithi/
204
205
* https://tracker.ceph.com/issues/57087
206
  test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan)
207
* https://tracker.ceph.com/issues/52624
208
  cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
209
* https://tracker.ceph.com/issues/51267
210
  tasks/workunit/snaps failure - Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status      
211
* https://tracker.ceph.com/issues/53360
212
* https://tracker.ceph.com/issues/57083
213
  qa/import-legacy tasks.cephfs.fuse_mount:mount command failed - :ModuleNotFoundError: No module named 'ceph_volume_client'
214
* https://tracker.ceph.com/issues/56507
215
  Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
216
217
218 67 Kotresh Hiremath Ravishankar
h3. 2022 July 15
219
220
https://pulpito.ceph.com/yuriw-2022-07-21_22:57:30-fs-wip-yuri2-testing-2022-07-15-0755-pacific-distro-default-smithi
221
Re-run: https://pulpito.ceph.com/yuriw-2022-07-24_15:34:38-fs-wip-yuri2-testing-2022-07-15-0755-pacific-distro-default-smithi
222
223
* https://tracker.ceph.com/issues/52624
224
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
225
* https://tracker.ceph.com/issues/57083
226
* https://tracker.ceph.com/issues/53360
227
        tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} - tasks.cephfs.fuse_mount:mount command failed
228
        client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" 
229
* https://tracker.ceph.com/issues/51183
230
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
231
* https://tracker.ceph.com/issues/56507
232
        pacific: Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
233
234
235 66 Venky Shankar
h3. 2022 July 08
236
237
* https://tracker.ceph.com/issues/52624
238
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
239
* https://tracker.ceph.com/issues/53360
240
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
241
* https://tracker.ceph.com/issues/51183
242
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
243
* https://tracker.ceph.com/issues/56506
244
        pacific: Test failure: test_rebuild_backtraceless (tasks.cephfs.test_data_scan.TestDataScan)
245
* https://tracker.ceph.com/issues/56507
246
        pacific: Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
247
248 65 Venky Shankar
h3. 2022 Jun 28
249
250
https://pulpito.ceph.com/?branch=wip-yuri3-testing-2022-06-22-1121-pacific
251
252
* https://tracker.ceph.com/issues/52624
253
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
254
* https://tracker.ceph.com/issues/53360
255
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
256
* https://tracker.ceph.com/issues/51183
257
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
258
259 64 Venky Shankar
h3. 2022 Jun 22
260
261
https://pulpito.ceph.com/?branch=wip-yuri4-testing-2022-06-21-0704-pacific
262
263
* https://tracker.ceph.com/issues/52624
264
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
265
* https://tracker.ceph.com/issues/53360
266
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
267
* https://tracker.ceph.com/issues/51183
268
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
269
270 63 Venky Shankar
h3. 2022 Jun 17
271
272
https://pulpito.ceph.com/yuriw-2022-06-15_17:11:32-fs-wip-yuri10-testing-2022-06-15-0732-pacific-distro-default-smithi/
273
274
* https://tracker.ceph.com/issues/52624
275
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
276
* https://tracker.ceph.com/issues/53360
277
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
278
* https://tracker.ceph.com/issues/51183
279
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
280
281 62 Venky Shankar
h3. 2022 Jun 16
282
283
https://pulpito.ceph.com/yuriw-2022-06-15_17:08:32-fs-wip-yuri-testing-2022-06-14-0744-pacific-distro-default-smithi/
284
285
* https://tracker.ceph.com/issues/52624
286
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
287
* https://tracker.ceph.com/issues/53360
288
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
289
* https://tracker.ceph.com/issues/55449
290
        pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh
291
* https://tracker.ceph.com/issues/51267
292
        CommandFailedError: Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status 1:...
293
* https://tracker.ceph.com/issues/55332
294
        Failure in snaptest-git-ceph.sh (it's an async unlink/create bug)
295
296 61 Venky Shankar
h3. 2022 Jun 15
297
298
http://pulpito.front.sepia.ceph.com/yuriw-2022-06-09_13:32:29-fs-wip-yuri10-testing-2022-06-08-0730-pacific-distro-default-smithi/
299
300
* https://tracker.ceph.com/issues/52624
301
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
302
* https://tracker.ceph.com/issues/53360
303
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
304
* https://tracker.ceph.com/issues/55449
305
        pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh
306
307
308 60 Venky Shankar
h3. 2022 Jun 10
309
310
https://pulpito.ceph.com/?branch=wip-yuri5-testing-2022-06-07-1529-pacific
311
312
* https://tracker.ceph.com/issues/52624
313
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
314
* https://tracker.ceph.com/issues/53360
315
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
316
* https://tracker.ceph.com/issues/55449
317
        pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh
318
319 59 Venky Shankar
h3. 2022 Jun 09
320
321
https://pulpito.ceph.com/yuriw-2022-06-07_16:05:25-fs-wip-yuri4-testing-2022-06-01-1350-pacific-distro-default-smithi/
322
323
* https://tracker.ceph.com/issues/52624
324
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
325
* https://tracker.ceph.com/issues/53360
326
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
327
* https://tracker.ceph.com/issues/55449
328
        pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh
329
330 58 Venky Shankar
h3. 2022 May 06
331
332
https://pulpito.ceph.com/?sha1=73636a1b00037ff974bcdc969b009c5ecec626cc
333
334
* https://tracker.ceph.com/issues/52624
335
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
336
* https://tracker.ceph.com/issues/53360
337
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
338
339 57 Venky Shankar
h3. 2022 April 18
340
341
http://pulpito.front.sepia.ceph.com/?branch=wip-yuri4-testing-2022-04-18-0609-pacific
342
(only mgr/snap_schedule backport pr)
343
344
* https://tracker.ceph.com/issues/52624
345
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
346 56 Venky Shankar
347
h3. 2022 March 28
348
349
http://pulpito.front.sepia.ceph.com/yuriw-2022-03-25_20:52:40-fs-wip-yuri4-testing-2022-03-25-1203-pacific-distro-default-smithi/
350
http://pulpito.front.sepia.ceph.com/yuriw-2022-03-26_19:52:48-fs-wip-yuri4-testing-2022-03-25-1203-pacific-distro-default-smithi/
351
352
* https://tracker.ceph.com/issues/52624
353
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
354
* https://tracker.ceph.com/issues/53360
355
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
356
* https://tracker.ceph.com/issues/54411
357
	mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available;
358
	33 daemons have recently crashed" during suites/fsstress.sh
359
360 55 Venky Shankar
h3. 2022 March 25
361
362
https://pulpito.ceph.com/yuriw-2022-03-22_18:42:28-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/
363
https://pulpito.ceph.com/yuriw-2022-03-23_14:04:56-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/
364
365
* https://tracker.ceph.com/issues/52624
366
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
367
* https://tracker.ceph.com/issues/52606
368
        qa: test_dirfrag_limit
369
* https://tracker.ceph.com/issues/51905
370
        qa: "error reading sessionmap 'mds1_sessionmap'"
371
* https://tracker.ceph.com/issues/53360
372
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
373
* https://tracker.ceph.com/issues/51183
374
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
375 47 Patrick Donnelly
376 54 Venky Shankar
h3. 2022 March 22
377
378
https://pulpito.ceph.com/yuriw-2022-03-17_15:03:06-fs-wip-yuri10-testing-2022-03-16-1432-pacific-distro-default-smithi/
379
380
* https://tracker.ceph.com/issues/52624
381
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
382
* https://tracker.ceph.com/issues/52606
383
        qa: test_dirfrag_limit
384
* https://tracker.ceph.com/issues/51183
385
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
386
* https://tracker.ceph.com/issues/51905
387
        qa: "error reading sessionmap 'mds1_sessionmap'"
388
* https://tracker.ceph.com/issues/53360
389
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
390
* https://tracker.ceph.com/issues/54411
391
	mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available;
392
	33 daemons have recently crashed" during suites/fsstress.sh
393 47 Patrick Donnelly
394 46 Kotresh Hiremath Ravishankar
h3. 2021 November 22
395
396
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-12_00:33:28-fs-wip-yuri7-testing-2021-11-11-1339-pacific-distro-basic-smithi
397
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-08_20:21:11-fs-wip-yuri5-testing-2021-11-08-1003-pacific-distro-basic-smithi
398
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-08_15:19:37-fs-wip-yuri2-testing-2021-11-06-1322-pacific-distro-basic-smithi
399
400
401
* https://tracker.ceph.com/issues/53300
402
	qa: cluster [WRN] Scrub error on inode
403
* https://tracker.ceph.com/issues/53302
404
	qa: sudo logrotate /etc/logrotate.d/ceph-test.conf failed with status 1
405
* https://tracker.ceph.com/issues/53314
406
	qa: fs/upgrade/mds_upgrade_sequence test timeout
407
* https://tracker.ceph.com/issues/53316
408
	qa: (smithi150) slow request osd_op, currently waiting for sub ops warning
409
* https://tracker.ceph.com/issues/52624
410
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
411
* https://tracker.ceph.com/issues/52396
412
	pacific: qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
413
* https://tracker.ceph.com/issues/52875
414
	pacific: qa: test_dirfrag_limit
415
* https://tracker.ceph.com/issues/51705
416 1 Patrick Donnelly
	pacific: qa: tasks.cephfs.fuse_mount:mount command failed
417
* https://tracker.ceph.com/issues/39634
418
	qa: test_full_same_file timeout
419 46 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/49748
420
	gibba: Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds
421 48 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/51964
422
	qa: test_cephfs_mirror_restart_sync_on_blocklist failure
423 49 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/50223
424
	qa: "client.4737 isn't responding to mclientcaps(revoke)"
425 48 Kotresh Hiremath Ravishankar
426 47 Patrick Donnelly
427
h3. 2021 November 20
428
429
https://pulpito.ceph.com/?branch=wip-pdonnell-testing-20211120.024903-pacific
430
431
* https://tracker.ceph.com/issues/53360
432
    pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
433 46 Kotresh Hiremath Ravishankar
434 41 Patrick Donnelly
h3. 2021 September 14 (QE)
435
436
https://pulpito.ceph.com/yuriw-2021-09-10_15:01:12-fs-pacific-distro-basic-smithi/
437 1 Patrick Donnelly
http://pulpito.front.sepia.ceph.com/yuriw-2021-09-13_14:09:59-fs-pacific-distro-basic-smithi/
438 43 Ramana Raja
https://pulpito.ceph.com/yuriw-2021-09-14_15:09:45-fs-pacific-distro-basic-smithi/
439 41 Patrick Donnelly
440
* https://tracker.ceph.com/issues/52606
441
    qa: test_dirfrag_limit
442
* https://tracker.ceph.com/issues/52607
443
    qa: "mon.a (mon.0) 1022 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY)"
444 44 Ramana Raja
* https://tracker.ceph.com/issues/51705
445
    qa: tasks.cephfs.fuse_mount:mount command failed
446 41 Patrick Donnelly
447 40 Ramana Raja
h3. 2021 Sep 7
448
449
https://pulpito.ceph.com/yuriw-2021-09-07_17:38:38-fs-wip-yuri-testing-2021-09-07-0905-pacific-distro-basic-smithi/
450
https://pulpito.ceph.com/yuriw-2021-09-07_23:57:28-fs-wip-yuri-testing-2021-09-07-0905-pacific-distro-basic-smithi/
451
452
453
* https://tracker.ceph.com/issues/52396
454
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
455
* https://tracker.ceph.com/issues/51705
456
    qa: tasks.cephfs.fuse_mount:mount command failed
457 4 Patrick Donnelly
458 34 Ramana Raja
h3. 2021 Aug 30
459
460
https://pulpito.ceph.com/?branch=wip-yuri8-testing-2021-08-30-0930-pacific
461
462
* https://tracker.ceph.com/issues/45434
463
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
464
* https://tracker.ceph.com/issues/51705
465
    qa: tasks.cephfs.fuse_mount:mount command failed
466
* https://tracker.ceph.com/issues/52396
467
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
468 35 Ramana Raja
* https://tracker.ceph.com/issues/52487
469 36 Ramana Raja
    qa: Test failure: test_deep_split (tasks.cephfs.test_fragment.TestFragmentation)
470
* https://tracker.ceph.com/issues/51267
471 37 Ramana Raja
   Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh)
472
* https://tracker.ceph.com/issues/48772
473
   qa: pjd: not ok 9, 44, 80
474 35 Ramana Raja
475 34 Ramana Raja
476 31 Ramana Raja
h3. 2021 Aug 23
477
478
https://pulpito.ceph.com/yuriw-2021-08-23_19:33:26-fs-wip-yuri4-testing-2021-08-23-0812-pacific-distro-basic-smithi/
479
480
* https://tracker.ceph.com/issues/45434
481
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
482
* https://tracker.ceph.com/issues/51705
483
    qa: tasks.cephfs.fuse_mount:mount command failed
484 32 Ramana Raja
* https://tracker.ceph.com/issues/52396
485 33 Ramana Raja
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
486
* https://tracker.ceph.com/issues/52397
487
    qa: test_acls (tasks.cephfs.test_acls.TestACLs) failed
488
489 31 Ramana Raja
490 29 Ramana Raja
h3. 2021 Aug 11
491
492
https://pulpito.ceph.com/yuriw-2021-08-11_14:28:23-fs-wip-yuri8-testing-2021-08-09-0844-pacific-distro-basic-smithi/
493
https://pulpito.ceph.com/yuriw-2021-08-12_15:32:35-fs-wip-yuri8-testing-2021-08-09-0844-pacific-distro-basic-smithi/
494
495
* https://tracker.ceph.com/issues/45434
496
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
497
* https://tracker.ceph.com/issues/51705
498
    qa: tasks.cephfs.fuse_mount:mount command failed
499 30 Ramana Raja
* https://tracker.ceph.com/issues/50222
500
    osd: 5.2s0 deep-scrub : stat mismatch
501 29 Ramana Raja
502 19 Jos Collin
h3. 2021 July 15
503 20 Jos Collin
504 19 Jos Collin
https://pulpito.ceph.com/yuriw-2021-07-13_17:37:59-fs-wip-yuri-testing-2021-07-13-0812-pacific-distro-basic-smithi/
505
506 26 Jos Collin
* https://tracker.ceph.com/issues/45434
507
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
508
* https://tracker.ceph.com/issues/51705
509
    qa: tasks.cephfs.fuse_mount:mount command failed
510 27 Jos Collin
* https://tracker.ceph.com/issues/51183
511
    Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
512 28 Jos Collin
* https://tracker.ceph.com/issues/50528
513
    qa: fs:thrash: pjd suite not ok 80
514
* https://tracker.ceph.com/issues/51706
515
    qa: osd deep-scrub stat mismatch
516 26 Jos Collin
517 19 Jos Collin
h3. 2021 July 13
518 20 Jos Collin
519 19 Jos Collin
https://pulpito.ceph.com/yuriw-2021-07-08_23:33:26-fs-wip-yuri2-testing-2021-07-08-1142-pacific-distro-basic-smithi/
520
521 21 Jos Collin
* https://tracker.ceph.com/issues/51704
522
    Test failure: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth)
523 23 Jos Collin
* https://tracker.ceph.com/issues/45434
524
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
525 24 Jos Collin
* https://tracker.ceph.com/issues/51705
526 25 Jos Collin
    qa: tasks.cephfs.fuse_mount:mount command failed
527
* https://tracker.ceph.com/issues/48640
528 24 Jos Collin
    qa: snapshot mismatch during mds thrashing
529 21 Jos Collin
530 18 Patrick Donnelly
h3. 2021 June 29 (Integration Branch)
531
532
https://pulpito.ceph.com/yuriw-2021-06-29_00:15:33-fs-wip-yuri3-testing-2021-06-28-1259-pacific-distro-basic-smithi/
533
534
Some failures caused by incomplete backup of: https://github.com/ceph/ceph/pull/42065
535
Some package failures caused by nautilus missing package, like: https://pulpito.ceph.com/yuriw-2021-06-29_00:15:33-fs-wip-yuri3-testing-2021-06-28-1259-pacific-distro-basic-smithi/6241300/
536
537
* https://tracker.ceph.com/issues/45434
538
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
539
* https://tracker.ceph.com/issues/50260
540
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
541
* https://tracker.ceph.com/issues/51183
542
    qa: FileNotFoundError: [Errno 2] No such file or directory: '/sys/kernel/debug/ceph/3fab6bea-f243-47a4-a956-8c03a62b61b5.client4721/mds_sessions'
543
544
545 16 Jeff Layton
h3. 2021 June 28
546
547
https://pulpito.ceph.com/yuriw-2021-06-29_00:14:14-fs-wip-yuri-testing-2021-06-28-1259-pacific-distro-basic-smithi/
548
549
* https://tracker.ceph.com/issues/45434
550
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
551
* https://tracker.ceph.com/issues/51440
552 17 Jeff Layton
    fallocate fails with EACCES
553 16 Jeff Layton
* https://tracker.ceph.com/issues/51264
554
    TestVolumeClient failure
555
* https://tracker.ceph.com/issues/51266
556
    test cleanup failure
557
* https://tracker.ceph.com/issues/51183
558
    Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) 
559
560 12 Jeff Layton
h3. 2021 June 14
561
562
https://pulpito.ceph.com/yuriw-2021-06-14_16:21:07-fs-wip-yuri2-testing-2021-06-14-0717-pacific-distro-basic-smithi/
563
 
564
* https://tracker.ceph.com/issues/45434
565
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
566
* https://bugzilla.redhat.com/show_bug.cgi?id=1973276
567
    Could not reconnect to ubuntu@smithi076.front.sepia.ceph.com
568
* https://tracker.ceph.com/issues/51263
569
    pjdfstest rename test 10.t failed with EACCES
570
* https://tracker.ceph.com/issues/51264
571
    TestVolumeClient failure
572 13 Jeff Layton
* https://tracker.ceph.com/issues/51266
573
    Command failed on smithi204 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 
574 14 Jeff Layton
* https://tracker.ceph.com/issues/50279
575
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
576 15 Jeff Layton
* https://tracker.ceph.com/issues/51267
577
    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status 1
578 11 Patrick Donnelly
579
h3. 2021 June 07 (Integration Branch)
580
581
http://pulpito.front.sepia.ceph.com/?branch=wip-yuri4-testing-2021-06-07-0955-pacific
582
583
* https://tracker.ceph.com/issues/45434
584
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
585
* https://tracker.ceph.com/issues/50279
586
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
587
* https://tracker.ceph.com/issues/48773
588
    qa: scrub does not complete
589
* https://tracker.ceph.com/issues/51170
590
    pacific: qa: AttributeError: 'RemoteProcess' object has no attribute 'split'
591
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
592
    qa: quota failure
593
594
595 6 Patrick Donnelly
h3. 2021 Apr 28 (QE pre-release)
596
597
https://pulpito.ceph.com/yuriw-2021-04-28_19:27:30-fs-pacific-distro-basic-smithi/
598
https://pulpito.ceph.com/yuriw-2021-04-30_13:46:58-fs-pacific-distro-basic-smithi/
599
600
* https://tracker.ceph.com/issues/45434
601
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
602
* https://tracker.ceph.com/issues/50258
603
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
604
* https://tracker.ceph.com/issues/50260
605
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
606
* https://tracker.ceph.com/issues/49962
607
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
608
* https://tracker.ceph.com/issues/50016
609
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"
610
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
611
    qa: quota failure
612
* https://tracker.ceph.com/issues/50528
613
    pacific: qa: fs:thrash: pjd suite not ok 20
614
615
616 3 Patrick Donnelly
h3. 2021 Apr 22 (Integration Branch)
617
618
https://pulpito.ceph.com/yuriw-2021-04-22_16:46:11-fs-wip-yuri5-testing-2021-04-20-0819-pacific-distro-basic-smithi/
619
620
* https://tracker.ceph.com/issues/50527
621
    pacific: qa: untar_snap_rm timeout during osd thrashing (ceph-fuse)
622
* https://tracker.ceph.com/issues/50528
623
    pacific: qa: fs:thrash: pjd suite not ok 20
624
* https://tracker.ceph.com/issues/49500 (fixed in another integration run)
625
    qa: "Assertion `cb_done' failed."
626
* https://tracker.ceph.com/issues/49500
627
    qa: "Assertion `cb_done' failed."
628
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
629
    qa: quota failure
630
* https://tracker.ceph.com/issues/45434
631
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
632
* https://tracker.ceph.com/issues/50279
633
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
634
* https://tracker.ceph.com/issues/50258
635
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
636
* https://tracker.ceph.com/issues/49962
637
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
638
* https://tracker.ceph.com/issues/49962
639
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
640
* https://tracker.ceph.com/issues/50530
641
    pacific: client: abort after MDS blocklist
642
643 2 Patrick Donnelly
644
h3. 2021 Apr 21 (Integration Branch)
645
646
https://pulpito.ceph.com/yuriw-2021-04-21_19:25:00-fs-wip-yuri3-testing-2021-04-21-0937-pacific-distro-basic-smithi/
647
648
* https://tracker.ceph.com/issues/45434
649
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
650
* https://tracker.ceph.com/issues/50250
651
    mds: "log [WRN] : Scrub error on inode 0x10000004506 (/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.a log and `damage ls` output for details"
652
* https://tracker.ceph.com/issues/50258
653
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
654
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
655
    qa: quota failure
656
* https://tracker.ceph.com/issues/50016
657
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"
658
* https://tracker.ceph.com/issues/50495
659
    pacific: client: shutdown race fails with status 141
660
661
662 1 Patrick Donnelly
h3. 2021 Apr 07 (Integration Branch)
663
664
https://pulpito.ceph.com/yuriw-2021-04-07_17:37:43-fs-wip-yuri-testing-2021-04-07-0905-pacific-distro-basic-smithi/
665
666
* https://tracker.ceph.com/issues/45434
667
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
668
* https://tracker.ceph.com/issues/48805
669
    mds: "cluster [WRN] Scrub error on inode 0x1000000039d (/client.0/tmp/blogbench-1.0/src/blogtest_in) see mds.a log and `damage ls` output for details"
670
* https://tracker.ceph.com/issues/49500
671
    qa: "Assertion `cb_done' failed."
672
* https://tracker.ceph.com/issues/50258 (new)
673
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
674
* https://tracker.ceph.com/issues/49962
675
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
676
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
677
    qa: quota failure
678
* https://tracker.ceph.com/issues/50260
679
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
680
* https://tracker.ceph.com/issues/50016
681
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"