Project

General

Profile

Pacific » History » Version 90

Rishabh Dave, 05/04/2023 11:05 AM

1 1 Patrick Donnelly
h1. Pacific
2
3 4 Patrick Donnelly
h2. On-call Schedule
4
5 81 Venky Shankar
* Dec: Venky
6
* Jan: Patrick
7
* Feb: Jos
8 79 Venky Shankar
* Mar: Xiubo
9
* Apr: Rishabh
10
* May: Kotresh
11 81 Venky Shankar
* Jun: Milind
12
13 1 Patrick Donnelly
h2. Reviews
14 85 Rishabh Dave
15 87 Rishabh Dave
h3. NEW ENTRY
16
17 90 Rishabh Dave
h3. 2023 May 4
18
19
https://pulpito.ceph.com/yuriw-2023-04-25_19:03:49-fs-wip-yuri5-testing-2023-04-25-0837-pacific-distro-default-smithi/
20
21
* https://tracker.ceph.com/issues/59560
22
  qa: RuntimeError: more than one file system available
23
* https://tracker.ceph.com/issues/59626
24
  FSMissing: File system xxxx does not exist in the map
25
* https://tracker.ceph.com/issues/58340
26
  fsstress.sh failed with errno 124
27
* https://tracker.ceph.com/issues/58992
28
  test_acls
29
* https://tracker.ceph.com/issues/48773
30
  Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete
31
* https://tracker.ceph.com/issues/57676
32
  qa: error during scrub thrashing: rank damage found: {'backtrace'}
33
34
35 86 Rishabh Dave
h3. 2023 Apr 13
36
37
https://pulpito.ceph.com/yuriw-2023-04-04_15:06:57-fs-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/
38
39
https://tracker.ceph.com/issues/52624
40
cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
41
* https://tracker.ceph.com/issues/57594
42
  Test failure: test_rebuild_moved_dir (tasks.cephfs.test_data_scan.TestDataScan)
43
* https://tracker.ceph.com/issues/54108
44
  qa: iogen workunit: "The following counters failed to be set on mds daemons: {'mds.exported', 'mds.imported'}" 
45
* https://tracker.ceph.com/issues/58340
46
  fsstress.sh failed with errno 125
47
* https://tracker.ceph.com/issues/54462
48
  Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi055 with status 128
49
* https://tracker.ceph.com/issues/49287     
50
  cephadm: podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found
51
* https://tracker.ceph.com/issues/58726
52
  test_acls: expected a yum based or a apt based system
53 40 Ramana Raja
54 84 Venky Shankar
h3. 2022 Dec 07
55
56
https://pulpito.ceph.com/yuriw-2022-12-07_15:45:30-fs-wip-yuri4-testing-2022-12-05-0921-pacific-distro-default-smithi/
57
58
many transient git.ceph.com related timeouts
59
60
* https://tracker.ceph.com/issues/52624
61
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
62
* https://tracker.ceph.com/issues/50224
63
    test_mirroring_init_failure_with_recovery - (tasks.cephfs.test_mirroring.TestMirroring)
64
* https://tracker.ceph.com/issues/56644
65
    qa: test_rapid_creation fails with "No space left on device"
66
* https://tracker.ceph.com/issues/58221
67
    pacific: Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan)
68
69 83 Venky Shankar
h3. 2022 Dec 02
70
71
many transient git.ceph.com related timeouts
72
many transient 'Failed to connect to the host via ssh' failures
73
74
* https://tracker.ceph.com/issues/57723
75
  pacific: qa: test_subvolume_snapshot_info_if_orphan_clone fails
76
* https://tracker.ceph.com/issues/52624
77
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
78
79 82 Venky Shankar
h3. 2022 Dec 01
80
81
many transient git.ceph.com related timeouts
82
83
* https://tracker.ceph.com/issues/57723
84
  pacific: qa: test_subvolume_snapshot_info_if_orphan_clone fails
85
* https://tracker.ceph.com/issues/52624
86
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
87
88 80 Patrick Donnelly
h3. 2022 Nov 18
89
90
https://trello.com/c/ecysAxl6/1656-wip-yuri-testing-2022-11-18-1500-pacific-old-wip-yuri-testing-2022-10-23-0729-pacific-old-wip-yuri-testing-2022-10-21-0927-pacif
91
https://pulpito.ceph.com/yuriw-2022-11-28_16:28:56-fs-wip-yuri-testing-2022-11-18-1500-pacific-distro-default-smithi/
92
93
2 ansible dead failures.
94
12 transient git.ceph.com related timeouts
95
96
* https://tracker.ceph.com/issues/57723
97
  pacific: qa: test_subvolume_snapshot_info_if_orphan_clone fails
98
* https://tracker.ceph.com/issues/52624
99
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
100
101 78 Milind Changire
h3. 2022 Oct 19
102
103
http://pulpito.front.sepia.ceph.com/yuriw-2022-10-10_17:18:40-fs-wip-yuri5-testing-2022-10-10-0837-pacific-distro-default-smithi/
104
105
* https://tracker.ceph.com/issues/57723
106
  pacific: qa: test_subvolume_snapshot_info_if_orphan_clone fails
107
* https://tracker.ceph.com/issues/52624
108
  cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
109
* https://tracker.ceph.com/issues/56644
110
  qa: test_rapid_creation fails with "No space left on device"
111
* https://tracker.ceph.com/issues/54460
112
  snaptest-multiple-capsnaps.sh test failure
113
* https://tracker.ceph.com/issues/57892
114
  sudo dd of=/home/ubuntu/cephtest/valgrind.supp failure during node setup phase
115
116
117
118 76 Venky Shankar
h3. 2022 Oct 06
119
120
http://pulpito.front.sepia.ceph.com/yuriw-2022-09-21_13:07:25-rados-wip-yuri5-testing-2022-09-20-1347-pacific-distro-default-smithi/
121
122
* https://tracker.ceph.com/issues/52624
123
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
124
* https://tracker.ceph.com/issues/50223
125
	qa: "client.4737 isn't responding to mclientcaps(revoke)"
126 77 Milind Changire
* https://tracker.ceph.com/issues/56507
127
  Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
128 76 Venky Shankar
129 75 Venky Shankar
h3. 2022 Sep 27
130
131
http://pulpito.front.sepia.ceph.com/yuriw-2022-09-22_22:35:04-fs-wip-yuri2-testing-2022-09-22-1400-pacific-distro-default-smithi/
132
133
* https://tracker.ceph.com/issues/52624
134
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
135
* https://tracker.ceph.com/issues/48773
136
    Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete
137
* https://tracker.ceph.com/issues/50224
138
    test_mirroring_init_failure_with_recovery - (tasks.cephfs.test_mirroring.TestMirroring)
139
* https://tracker.ceph.com/issues/56507
140
    Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
141
* https://tracker.ceph.com/issues/50223
142
	qa: "client.4737 isn't responding to mclientcaps(revoke)"
143
144 74 Venky Shankar
h3. 2022 Sep 22
145
146
http://pulpito.front.sepia.ceph.com/?branch=wip-yuri2-testing-2022-09-06-1007-pacific
147
148
* https://tracker.ceph.com/issues/52624
149
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
150
* https://tracker.ceph.com/issues/51282
151
    cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log
152
* https://tracker.ceph.com/issues/53360
153
    pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
154
155 73 Venky Shankar
h3. 2022 Sep 19
156
157
http://pulpito.front.sepia.ceph.com/yuriw-2022-09-16_19:05:25-fs-wip-yuri11-testing-2022-09-16-0958-pacific-distro-default-smithi/
158
159
* https://tracker.ceph.com/issues/52624
160
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
161
* https://tracker.ceph.com/issues/57594
162
    pacific: Test failure: test_rebuild_moved_dir (tasks.cephfs.test_data_scan.TestDataScan)
163
164 71 Venky Shankar
h3. 2022 Sep 15
165
166
http://pulpito.front.sepia.ceph.com/yuriw-2022-09-13_14:22:44-fs-wip-yuri5-testing-2022-09-09-1109-pacific-distro-default-smithi/
167
168
* https://tracker.ceph.com/issues/51282
169
    cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log
170
* https://tracker.ceph.com/issues/52624
171
    cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
172
* https://tracker.ceph.com/issues/53360
173
    pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
174 72 Venky Shankar
* https://tracker.ceph.com/issues/48773
175
    Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete
176 71 Venky Shankar
177 70 Kotresh Hiremath Ravishankar
h3. 2022 AUG 18
178
179
https://pulpito.ceph.com/yuriw-2022-08-18_23:16:33-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-default-smithi/
180
* https://tracker.ceph.com/issues/52624 - cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
181
* https://tracker.ceph.com/issues/51267 - tasks/workunit/snaps failure - Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status
182
* https://tracker.ceph.com/issues/48773 - Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete
183
* https://tracker.ceph.com/issues/51964 - qa: test_cephfs_mirror_restart_sync_on_blocklist failure
184
* https://tracker.ceph.com/issues/57218 - qa: tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} fails
185
* https://tracker.ceph.com/issues/56507 - Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
186
187
188
Re-run1: https://pulpito.ceph.com/yuriw-2022-08-19_21:01:11-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-default-smithi/
189
* https://tracker.ceph.com/issues/52624 - cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
190
* https://tracker.ceph.com/issues/57219 - mds crashed while running workunit test fs/misc/dirfrag.sh
191
192
193 69 Kotresh Hiremath Ravishankar
h3. 2022 AUG 11
194
195
https://pulpito.ceph.com/yuriw-2022-08-11_16:57:01-fs-wip-yuri3-testing-2022-08-11-0809-pacific-distro-default-smithi/
196
* Most of the failures are passed in re-run. Please check rerun failures below.
197
  - https://tracker.ceph.com/issues/57147 - test_full_fsync (tasks.cephfs.test_full.TestClusterFull) - mds stuck in up:creating state - osd crash
198
  - https://tracker.ceph.com/issues/52624 - cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
199
  - https://tracker.ceph.com/issues/51282 - cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log
200
  - https://tracker.ceph.com/issues/50224 - test_mirroring_init_failure_with_recovery - (tasks.cephfs.test_mirroring.TestMirroring)
201
  - https://tracker.ceph.com/issues/57083 - https://tracker.ceph.com/issues/53360 - tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} 
202
  - https://tracker.ceph.com/issues/51183 - Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
203
  - https://tracker.ceph.com/issues/56507 - Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)   
204
205
Re-run-1: https://pulpito.ceph.com/yuriw-2022-08-15_13:45:54-fs-wip-yuri3-testing-2022-08-11-0809-pacific-distro-default-smithi/
206
* https://tracker.ceph.com/issues/52624
207
  cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
208
* tasks/mirror - Command failed on smithi078 with status 123: "sudo find /var/log/ceph -name '*.log' -print0 ..." - Asked for re-run.
209
* https://tracker.ceph.com/issues/57083
210
* https://tracker.ceph.com/issues/53360
211
  tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} - tasks.cephfs.fuse_mount:mount command failed
212
  client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
213
* https://tracker.ceph.com/issues/51183
214
  Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
215
* https://tracker.ceph.com/issues/56507
216
  Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
217
218 68 Kotresh Hiremath Ravishankar
219
h3. 2022 AUG 04
220
221
https://pulpito.ceph.com/yuriw-2022-08-04_20:54:08-fs-wip-yuri6-testing-2022-08-04-0617-pacific-distro-default-smithi/
222
223
* https://tracker.ceph.com/issues/57087
224
  test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan)
225
* https://tracker.ceph.com/issues/52624
226
  cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
227
* https://tracker.ceph.com/issues/51267
228
  tasks/workunit/snaps failure - Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status      
229
* https://tracker.ceph.com/issues/53360
230
* https://tracker.ceph.com/issues/57083
231
  qa/import-legacy tasks.cephfs.fuse_mount:mount command failed - :ModuleNotFoundError: No module named 'ceph_volume_client'
232
* https://tracker.ceph.com/issues/56507
233
  Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
234
235
236 67 Kotresh Hiremath Ravishankar
h3. 2022 July 15
237
238
https://pulpito.ceph.com/yuriw-2022-07-21_22:57:30-fs-wip-yuri2-testing-2022-07-15-0755-pacific-distro-default-smithi
239
Re-run: https://pulpito.ceph.com/yuriw-2022-07-24_15:34:38-fs-wip-yuri2-testing-2022-07-15-0755-pacific-distro-default-smithi
240
241
* https://tracker.ceph.com/issues/52624
242
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
243
* https://tracker.ceph.com/issues/57083
244
* https://tracker.ceph.com/issues/53360
245
        tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} - tasks.cephfs.fuse_mount:mount command failed
246
        client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" 
247
* https://tracker.ceph.com/issues/51183
248
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
249
* https://tracker.ceph.com/issues/56507
250
        pacific: Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
251
252
253 66 Venky Shankar
h3. 2022 July 08
254
255
* https://tracker.ceph.com/issues/52624
256
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
257
* https://tracker.ceph.com/issues/53360
258
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
259
* https://tracker.ceph.com/issues/51183
260
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
261
* https://tracker.ceph.com/issues/56506
262
        pacific: Test failure: test_rebuild_backtraceless (tasks.cephfs.test_data_scan.TestDataScan)
263
* https://tracker.ceph.com/issues/56507
264
        pacific: Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)
265
266 65 Venky Shankar
h3. 2022 Jun 28
267
268
https://pulpito.ceph.com/?branch=wip-yuri3-testing-2022-06-22-1121-pacific
269
270
* https://tracker.ceph.com/issues/52624
271
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
272
* https://tracker.ceph.com/issues/53360
273
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
274
* https://tracker.ceph.com/issues/51183
275
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
276
277 64 Venky Shankar
h3. 2022 Jun 22
278
279
https://pulpito.ceph.com/?branch=wip-yuri4-testing-2022-06-21-0704-pacific
280
281
* https://tracker.ceph.com/issues/52624
282
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
283
* https://tracker.ceph.com/issues/53360
284
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
285
* https://tracker.ceph.com/issues/51183
286
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
287
288 63 Venky Shankar
h3. 2022 Jun 17
289
290
https://pulpito.ceph.com/yuriw-2022-06-15_17:11:32-fs-wip-yuri10-testing-2022-06-15-0732-pacific-distro-default-smithi/
291
292
* https://tracker.ceph.com/issues/52624
293
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
294
* https://tracker.ceph.com/issues/53360
295
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
296
* https://tracker.ceph.com/issues/51183
297
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
298
299 62 Venky Shankar
h3. 2022 Jun 16
300
301
https://pulpito.ceph.com/yuriw-2022-06-15_17:08:32-fs-wip-yuri-testing-2022-06-14-0744-pacific-distro-default-smithi/
302
303
* https://tracker.ceph.com/issues/52624
304
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
305
* https://tracker.ceph.com/issues/53360
306
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
307
* https://tracker.ceph.com/issues/55449
308
        pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh
309
* https://tracker.ceph.com/issues/51267
310
        CommandFailedError: Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status 1:...
311
* https://tracker.ceph.com/issues/55332
312
        Failure in snaptest-git-ceph.sh (it's an async unlink/create bug)
313
314 61 Venky Shankar
h3. 2022 Jun 15
315
316
http://pulpito.front.sepia.ceph.com/yuriw-2022-06-09_13:32:29-fs-wip-yuri10-testing-2022-06-08-0730-pacific-distro-default-smithi/
317
318
* https://tracker.ceph.com/issues/52624
319
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
320
* https://tracker.ceph.com/issues/53360
321
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
322
* https://tracker.ceph.com/issues/55449
323
        pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh
324
325
326 60 Venky Shankar
h3. 2022 Jun 10
327
328
https://pulpito.ceph.com/?branch=wip-yuri5-testing-2022-06-07-1529-pacific
329
330
* https://tracker.ceph.com/issues/52624
331
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
332
* https://tracker.ceph.com/issues/53360
333
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
334
* https://tracker.ceph.com/issues/55449
335
        pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh
336
337 59 Venky Shankar
h3. 2022 Jun 09
338
339
https://pulpito.ceph.com/yuriw-2022-06-07_16:05:25-fs-wip-yuri4-testing-2022-06-01-1350-pacific-distro-default-smithi/
340
341
* https://tracker.ceph.com/issues/52624
342
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
343
* https://tracker.ceph.com/issues/53360
344
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
345
* https://tracker.ceph.com/issues/55449
346
        pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh
347
348 58 Venky Shankar
h3. 2022 May 06
349
350
https://pulpito.ceph.com/?sha1=73636a1b00037ff974bcdc969b009c5ecec626cc
351
352
* https://tracker.ceph.com/issues/52624
353
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
354
* https://tracker.ceph.com/issues/53360
355
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
356
357 57 Venky Shankar
h3. 2022 April 18
358
359
http://pulpito.front.sepia.ceph.com/?branch=wip-yuri4-testing-2022-04-18-0609-pacific
360
(only mgr/snap_schedule backport pr)
361
362
* https://tracker.ceph.com/issues/52624
363
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
364 56 Venky Shankar
365
h3. 2022 March 28
366
367
http://pulpito.front.sepia.ceph.com/yuriw-2022-03-25_20:52:40-fs-wip-yuri4-testing-2022-03-25-1203-pacific-distro-default-smithi/
368
http://pulpito.front.sepia.ceph.com/yuriw-2022-03-26_19:52:48-fs-wip-yuri4-testing-2022-03-25-1203-pacific-distro-default-smithi/
369
370
* https://tracker.ceph.com/issues/52624
371
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
372
* https://tracker.ceph.com/issues/53360
373
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
374
* https://tracker.ceph.com/issues/54411
375
	mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available;
376
	33 daemons have recently crashed" during suites/fsstress.sh
377
378 55 Venky Shankar
h3. 2022 March 25
379
380
https://pulpito.ceph.com/yuriw-2022-03-22_18:42:28-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/
381
https://pulpito.ceph.com/yuriw-2022-03-23_14:04:56-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/
382
383
* https://tracker.ceph.com/issues/52624
384
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
385
* https://tracker.ceph.com/issues/52606
386
        qa: test_dirfrag_limit
387
* https://tracker.ceph.com/issues/51905
388
        qa: "error reading sessionmap 'mds1_sessionmap'"
389
* https://tracker.ceph.com/issues/53360
390
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
391
* https://tracker.ceph.com/issues/51183
392
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
393 47 Patrick Donnelly
394 54 Venky Shankar
h3. 2022 March 22
395
396
https://pulpito.ceph.com/yuriw-2022-03-17_15:03:06-fs-wip-yuri10-testing-2022-03-16-1432-pacific-distro-default-smithi/
397
398
* https://tracker.ceph.com/issues/52624
399
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
400
* https://tracker.ceph.com/issues/52606
401
        qa: test_dirfrag_limit
402
* https://tracker.ceph.com/issues/51183
403
        Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
404
* https://tracker.ceph.com/issues/51905
405
        qa: "error reading sessionmap 'mds1_sessionmap'"
406
* https://tracker.ceph.com/issues/53360
407
        pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
408
* https://tracker.ceph.com/issues/54411
409
	mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available;
410
	33 daemons have recently crashed" during suites/fsstress.sh
411 47 Patrick Donnelly
412 46 Kotresh Hiremath Ravishankar
h3. 2021 November 22
413
414
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-12_00:33:28-fs-wip-yuri7-testing-2021-11-11-1339-pacific-distro-basic-smithi
415
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-08_20:21:11-fs-wip-yuri5-testing-2021-11-08-1003-pacific-distro-basic-smithi
416
http://pulpito.front.sepia.ceph.com/yuriw-2021-11-08_15:19:37-fs-wip-yuri2-testing-2021-11-06-1322-pacific-distro-basic-smithi
417
418
419
* https://tracker.ceph.com/issues/53300
420
	qa: cluster [WRN] Scrub error on inode
421
* https://tracker.ceph.com/issues/53302
422
	qa: sudo logrotate /etc/logrotate.d/ceph-test.conf failed with status 1
423
* https://tracker.ceph.com/issues/53314
424
	qa: fs/upgrade/mds_upgrade_sequence test timeout
425
* https://tracker.ceph.com/issues/53316
426
	qa: (smithi150) slow request osd_op, currently waiting for sub ops warning
427
* https://tracker.ceph.com/issues/52624
428
	qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"
429
* https://tracker.ceph.com/issues/52396
430
	pacific: qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
431
* https://tracker.ceph.com/issues/52875
432
	pacific: qa: test_dirfrag_limit
433
* https://tracker.ceph.com/issues/51705
434 1 Patrick Donnelly
	pacific: qa: tasks.cephfs.fuse_mount:mount command failed
435
* https://tracker.ceph.com/issues/39634
436
	qa: test_full_same_file timeout
437 46 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/49748
438
	gibba: Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds
439 48 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/51964
440
	qa: test_cephfs_mirror_restart_sync_on_blocklist failure
441 49 Kotresh Hiremath Ravishankar
* https://tracker.ceph.com/issues/50223
442
	qa: "client.4737 isn't responding to mclientcaps(revoke)"
443 48 Kotresh Hiremath Ravishankar
444 47 Patrick Donnelly
445
h3. 2021 November 20
446
447
https://pulpito.ceph.com/?branch=wip-pdonnell-testing-20211120.024903-pacific
448
449
* https://tracker.ceph.com/issues/53360
450
    pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]"
451 46 Kotresh Hiremath Ravishankar
452 41 Patrick Donnelly
h3. 2021 September 14 (QE)
453
454
https://pulpito.ceph.com/yuriw-2021-09-10_15:01:12-fs-pacific-distro-basic-smithi/
455 1 Patrick Donnelly
http://pulpito.front.sepia.ceph.com/yuriw-2021-09-13_14:09:59-fs-pacific-distro-basic-smithi/
456 43 Ramana Raja
https://pulpito.ceph.com/yuriw-2021-09-14_15:09:45-fs-pacific-distro-basic-smithi/
457 41 Patrick Donnelly
458
* https://tracker.ceph.com/issues/52606
459
    qa: test_dirfrag_limit
460
* https://tracker.ceph.com/issues/52607
461
    qa: "mon.a (mon.0) 1022 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY)"
462 44 Ramana Raja
* https://tracker.ceph.com/issues/51705
463
    qa: tasks.cephfs.fuse_mount:mount command failed
464 41 Patrick Donnelly
465 40 Ramana Raja
h3. 2021 Sep 7
466
467
https://pulpito.ceph.com/yuriw-2021-09-07_17:38:38-fs-wip-yuri-testing-2021-09-07-0905-pacific-distro-basic-smithi/
468
https://pulpito.ceph.com/yuriw-2021-09-07_23:57:28-fs-wip-yuri-testing-2021-09-07-0905-pacific-distro-basic-smithi/
469
470
471
* https://tracker.ceph.com/issues/52396
472
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
473
* https://tracker.ceph.com/issues/51705
474
    qa: tasks.cephfs.fuse_mount:mount command failed
475 4 Patrick Donnelly
476 34 Ramana Raja
h3. 2021 Aug 30
477
478
https://pulpito.ceph.com/?branch=wip-yuri8-testing-2021-08-30-0930-pacific
479
480
* https://tracker.ceph.com/issues/45434
481
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
482
* https://tracker.ceph.com/issues/51705
483
    qa: tasks.cephfs.fuse_mount:mount command failed
484
* https://tracker.ceph.com/issues/52396
485
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
486 35 Ramana Raja
* https://tracker.ceph.com/issues/52487
487 36 Ramana Raja
    qa: Test failure: test_deep_split (tasks.cephfs.test_fragment.TestFragmentation)
488
* https://tracker.ceph.com/issues/51267
489 37 Ramana Raja
   Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh)
490
* https://tracker.ceph.com/issues/48772
491
   qa: pjd: not ok 9, 44, 80
492 35 Ramana Raja
493 34 Ramana Raja
494 31 Ramana Raja
h3. 2021 Aug 23
495
496
https://pulpito.ceph.com/yuriw-2021-08-23_19:33:26-fs-wip-yuri4-testing-2021-08-23-0812-pacific-distro-basic-smithi/
497
498
* https://tracker.ceph.com/issues/45434
499
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
500
* https://tracker.ceph.com/issues/51705
501
    qa: tasks.cephfs.fuse_mount:mount command failed
502 32 Ramana Raja
* https://tracker.ceph.com/issues/52396
503 33 Ramana Raja
    qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable)
504
* https://tracker.ceph.com/issues/52397
505
    qa: test_acls (tasks.cephfs.test_acls.TestACLs) failed
506
507 31 Ramana Raja
508 29 Ramana Raja
h3. 2021 Aug 11
509
510
https://pulpito.ceph.com/yuriw-2021-08-11_14:28:23-fs-wip-yuri8-testing-2021-08-09-0844-pacific-distro-basic-smithi/
511
https://pulpito.ceph.com/yuriw-2021-08-12_15:32:35-fs-wip-yuri8-testing-2021-08-09-0844-pacific-distro-basic-smithi/
512
513
* https://tracker.ceph.com/issues/45434
514
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
515
* https://tracker.ceph.com/issues/51705
516
    qa: tasks.cephfs.fuse_mount:mount command failed
517 30 Ramana Raja
* https://tracker.ceph.com/issues/50222
518
    osd: 5.2s0 deep-scrub : stat mismatch
519 29 Ramana Raja
520 19 Jos Collin
h3. 2021 July 15
521 20 Jos Collin
522 19 Jos Collin
https://pulpito.ceph.com/yuriw-2021-07-13_17:37:59-fs-wip-yuri-testing-2021-07-13-0812-pacific-distro-basic-smithi/
523
524 26 Jos Collin
* https://tracker.ceph.com/issues/45434
525
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
526
* https://tracker.ceph.com/issues/51705
527
    qa: tasks.cephfs.fuse_mount:mount command failed
528 27 Jos Collin
* https://tracker.ceph.com/issues/51183
529
    Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
530 28 Jos Collin
* https://tracker.ceph.com/issues/50528
531
    qa: fs:thrash: pjd suite not ok 80
532
* https://tracker.ceph.com/issues/51706
533
    qa: osd deep-scrub stat mismatch
534 26 Jos Collin
535 19 Jos Collin
h3. 2021 July 13
536 20 Jos Collin
537 19 Jos Collin
https://pulpito.ceph.com/yuriw-2021-07-08_23:33:26-fs-wip-yuri2-testing-2021-07-08-1142-pacific-distro-basic-smithi/
538
539 21 Jos Collin
* https://tracker.ceph.com/issues/51704
540
    Test failure: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth)
541 23 Jos Collin
* https://tracker.ceph.com/issues/45434
542
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
543 24 Jos Collin
* https://tracker.ceph.com/issues/51705
544 25 Jos Collin
    qa: tasks.cephfs.fuse_mount:mount command failed
545
* https://tracker.ceph.com/issues/48640
546 24 Jos Collin
    qa: snapshot mismatch during mds thrashing
547 21 Jos Collin
548 18 Patrick Donnelly
h3. 2021 June 29 (Integration Branch)
549
550
https://pulpito.ceph.com/yuriw-2021-06-29_00:15:33-fs-wip-yuri3-testing-2021-06-28-1259-pacific-distro-basic-smithi/
551
552
Some failures caused by incomplete backup of: https://github.com/ceph/ceph/pull/42065
553
Some package failures caused by nautilus missing package, like: https://pulpito.ceph.com/yuriw-2021-06-29_00:15:33-fs-wip-yuri3-testing-2021-06-28-1259-pacific-distro-basic-smithi/6241300/
554
555
* https://tracker.ceph.com/issues/45434
556
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
557
* https://tracker.ceph.com/issues/50260
558
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
559
* https://tracker.ceph.com/issues/51183
560
    qa: FileNotFoundError: [Errno 2] No such file or directory: '/sys/kernel/debug/ceph/3fab6bea-f243-47a4-a956-8c03a62b61b5.client4721/mds_sessions'
561
562
563 16 Jeff Layton
h3. 2021 June 28
564
565
https://pulpito.ceph.com/yuriw-2021-06-29_00:14:14-fs-wip-yuri-testing-2021-06-28-1259-pacific-distro-basic-smithi/
566
567
* https://tracker.ceph.com/issues/45434
568
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
569
* https://tracker.ceph.com/issues/51440
570 17 Jeff Layton
    fallocate fails with EACCES
571 16 Jeff Layton
* https://tracker.ceph.com/issues/51264
572
    TestVolumeClient failure
573
* https://tracker.ceph.com/issues/51266
574
    test cleanup failure
575
* https://tracker.ceph.com/issues/51183
576
    Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) 
577
578 12 Jeff Layton
h3. 2021 June 14
579
580
https://pulpito.ceph.com/yuriw-2021-06-14_16:21:07-fs-wip-yuri2-testing-2021-06-14-0717-pacific-distro-basic-smithi/
581
 
582
* https://tracker.ceph.com/issues/45434
583
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
584
* https://bugzilla.redhat.com/show_bug.cgi?id=1973276
585
    Could not reconnect to ubuntu@smithi076.front.sepia.ceph.com
586
* https://tracker.ceph.com/issues/51263
587
    pjdfstest rename test 10.t failed with EACCES
588
* https://tracker.ceph.com/issues/51264
589
    TestVolumeClient failure
590 13 Jeff Layton
* https://tracker.ceph.com/issues/51266
591
    Command failed on smithi204 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 
592 14 Jeff Layton
* https://tracker.ceph.com/issues/50279
593
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
594 15 Jeff Layton
* https://tracker.ceph.com/issues/51267
595
    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status 1
596 11 Patrick Donnelly
597
h3. 2021 June 07 (Integration Branch)
598
599
http://pulpito.front.sepia.ceph.com/?branch=wip-yuri4-testing-2021-06-07-0955-pacific
600
601
* https://tracker.ceph.com/issues/45434
602
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
603
* https://tracker.ceph.com/issues/50279
604
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
605
* https://tracker.ceph.com/issues/48773
606
    qa: scrub does not complete
607
* https://tracker.ceph.com/issues/51170
608
    pacific: qa: AttributeError: 'RemoteProcess' object has no attribute 'split'
609
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
610
    qa: quota failure
611
612
613 6 Patrick Donnelly
h3. 2021 Apr 28 (QE pre-release)
614
615
https://pulpito.ceph.com/yuriw-2021-04-28_19:27:30-fs-pacific-distro-basic-smithi/
616
https://pulpito.ceph.com/yuriw-2021-04-30_13:46:58-fs-pacific-distro-basic-smithi/
617
618
* https://tracker.ceph.com/issues/45434
619
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
620
* https://tracker.ceph.com/issues/50258
621
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
622
* https://tracker.ceph.com/issues/50260
623
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
624
* https://tracker.ceph.com/issues/49962
625
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
626
* https://tracker.ceph.com/issues/50016
627
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"
628
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
629
    qa: quota failure
630
* https://tracker.ceph.com/issues/50528
631
    pacific: qa: fs:thrash: pjd suite not ok 20
632
633
634 3 Patrick Donnelly
h3. 2021 Apr 22 (Integration Branch)
635
636
https://pulpito.ceph.com/yuriw-2021-04-22_16:46:11-fs-wip-yuri5-testing-2021-04-20-0819-pacific-distro-basic-smithi/
637
638
* https://tracker.ceph.com/issues/50527
639
    pacific: qa: untar_snap_rm timeout during osd thrashing (ceph-fuse)
640
* https://tracker.ceph.com/issues/50528
641
    pacific: qa: fs:thrash: pjd suite not ok 20
642
* https://tracker.ceph.com/issues/49500 (fixed in another integration run)
643
    qa: "Assertion `cb_done' failed."
644
* https://tracker.ceph.com/issues/49500
645
    qa: "Assertion `cb_done' failed."
646
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
647
    qa: quota failure
648
* https://tracker.ceph.com/issues/45434
649
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
650
* https://tracker.ceph.com/issues/50279
651
    qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
652
* https://tracker.ceph.com/issues/50258
653
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
654
* https://tracker.ceph.com/issues/49962
655
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
656
* https://tracker.ceph.com/issues/49962
657
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
658
* https://tracker.ceph.com/issues/50530
659
    pacific: client: abort after MDS blocklist
660
661 2 Patrick Donnelly
662
h3. 2021 Apr 21 (Integration Branch)
663
664
https://pulpito.ceph.com/yuriw-2021-04-21_19:25:00-fs-wip-yuri3-testing-2021-04-21-0937-pacific-distro-basic-smithi/
665
666
* https://tracker.ceph.com/issues/45434
667
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
668
* https://tracker.ceph.com/issues/50250
669
    mds: "log [WRN] : Scrub error on inode 0x10000004506 (/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.a log and `damage ls` output for details"
670
* https://tracker.ceph.com/issues/50258
671
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
672
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
673
    qa: quota failure
674
* https://tracker.ceph.com/issues/50016
675
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"
676
* https://tracker.ceph.com/issues/50495
677
    pacific: client: shutdown race fails with status 141
678
679
680 1 Patrick Donnelly
h3. 2021 Apr 07 (Integration Branch)
681
682
https://pulpito.ceph.com/yuriw-2021-04-07_17:37:43-fs-wip-yuri-testing-2021-04-07-0905-pacific-distro-basic-smithi/
683
684
* https://tracker.ceph.com/issues/45434
685
    qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed
686
* https://tracker.ceph.com/issues/48805
687
    mds: "cluster [WRN] Scrub error on inode 0x1000000039d (/client.0/tmp/blogbench-1.0/src/blogtest_in) see mds.a log and `damage ls` output for details"
688
* https://tracker.ceph.com/issues/49500
689
    qa: "Assertion `cb_done' failed."
690
* https://tracker.ceph.com/issues/50258 (new)
691
    pacific: qa: "run() got an unexpected keyword argument 'stdin_data'"
692
* https://tracker.ceph.com/issues/49962
693
    'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes
694
* https://tracker.ceph.com/issues/48203 (stock kernel update required)
695
    qa: quota failure
696
* https://tracker.ceph.com/issues/50260
697
    pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
698
* https://tracker.ceph.com/issues/50016
699
    qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes"