Pacific » History » Version 76
Venky Shankar, 10/06/2022 04:44 AM
1 | 1 | Patrick Donnelly | h1. Pacific |
---|---|---|---|
2 | |||
3 | 4 | Patrick Donnelly | h2. On-call Schedule |
4 | |||
5 | 52 | Venky Shankar | * Feb: Patrick |
6 | 53 | Venky Shankar | * Mar: Jeff |
7 | * Apr: Jos Collin |
||
8 | * May: Ramana |
||
9 | * Jun: Xiubo |
||
10 | * Jul: Rishabh |
||
11 | * Aug: Kotresh |
||
12 | * Sep: Venky |
||
13 | * Oct: Milind |
||
14 | 38 | Ramana Raja | |
15 | 1 | Patrick Donnelly | h2. Reviews |
16 | 40 | Ramana Raja | |
17 | 76 | Venky Shankar | h3. 2022 Oct 06 |
18 | |||
19 | http://pulpito.front.sepia.ceph.com/yuriw-2022-09-21_13:07:25-rados-wip-yuri5-testing-2022-09-20-1347-pacific-distro-default-smithi/ |
||
20 | |||
21 | * https://tracker.ceph.com/issues/52624 |
||
22 | cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||
23 | * https://tracker.ceph.com/issues/50223 |
||
24 | qa: "client.4737 isn't responding to mclientcaps(revoke)" |
||
25 | |||
26 | 75 | Venky Shankar | h3. 2022 Sep 27 |
27 | |||
28 | http://pulpito.front.sepia.ceph.com/yuriw-2022-09-22_22:35:04-fs-wip-yuri2-testing-2022-09-22-1400-pacific-distro-default-smithi/ |
||
29 | |||
30 | * https://tracker.ceph.com/issues/52624 |
||
31 | cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||
32 | * https://tracker.ceph.com/issues/48773 |
||
33 | Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete |
||
34 | * https://tracker.ceph.com/issues/50224 |
||
35 | test_mirroring_init_failure_with_recovery - (tasks.cephfs.test_mirroring.TestMirroring) |
||
36 | * https://tracker.ceph.com/issues/56507 |
||
37 | Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation) |
||
38 | * https://tracker.ceph.com/issues/50223 |
||
39 | qa: "client.4737 isn't responding to mclientcaps(revoke)" |
||
40 | |||
41 | 74 | Venky Shankar | h3. 2022 Sep 22 |
42 | |||
43 | http://pulpito.front.sepia.ceph.com/?branch=wip-yuri2-testing-2022-09-06-1007-pacific |
||
44 | |||
45 | * https://tracker.ceph.com/issues/52624 |
||
46 | cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||
47 | * https://tracker.ceph.com/issues/51282 |
||
48 | cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log |
||
49 | * https://tracker.ceph.com/issues/53360 |
||
50 | pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
51 | |||
52 | 73 | Venky Shankar | h3. 2022 Sep 19 |
53 | |||
54 | http://pulpito.front.sepia.ceph.com/yuriw-2022-09-16_19:05:25-fs-wip-yuri11-testing-2022-09-16-0958-pacific-distro-default-smithi/ |
||
55 | |||
56 | * https://tracker.ceph.com/issues/52624 |
||
57 | cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||
58 | * https://tracker.ceph.com/issues/57594 |
||
59 | pacific: Test failure: test_rebuild_moved_dir (tasks.cephfs.test_data_scan.TestDataScan) |
||
60 | |||
61 | 71 | Venky Shankar | h3. 2022 Sep 15 |
62 | |||
63 | http://pulpito.front.sepia.ceph.com/yuriw-2022-09-13_14:22:44-fs-wip-yuri5-testing-2022-09-09-1109-pacific-distro-default-smithi/ |
||
64 | |||
65 | * https://tracker.ceph.com/issues/51282 |
||
66 | cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log |
||
67 | * https://tracker.ceph.com/issues/52624 |
||
68 | cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||
69 | * https://tracker.ceph.com/issues/53360 |
||
70 | pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
71 | 72 | Venky Shankar | * https://tracker.ceph.com/issues/48773 |
72 | Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete |
||
73 | 71 | Venky Shankar | |
74 | 70 | Kotresh Hiremath Ravishankar | h3. 2022 AUG 18 |
75 | |||
76 | https://pulpito.ceph.com/yuriw-2022-08-18_23:16:33-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-default-smithi/ |
||
77 | * https://tracker.ceph.com/issues/52624 - cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||
78 | * https://tracker.ceph.com/issues/51267 - tasks/workunit/snaps failure - Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status |
||
79 | * https://tracker.ceph.com/issues/48773 - Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi103 with status 23: 'mkdir -p...' - qa: scrub does not complete |
||
80 | * https://tracker.ceph.com/issues/51964 - qa: test_cephfs_mirror_restart_sync_on_blocklist failure |
||
81 | * https://tracker.ceph.com/issues/57218 - qa: tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} fails |
||
82 | * https://tracker.ceph.com/issues/56507 - Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation) |
||
83 | |||
84 | |||
85 | Re-run1: https://pulpito.ceph.com/yuriw-2022-08-19_21:01:11-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-default-smithi/ |
||
86 | * https://tracker.ceph.com/issues/52624 - cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||
87 | * https://tracker.ceph.com/issues/57219 - mds crashed while running workunit test fs/misc/dirfrag.sh |
||
88 | |||
89 | |||
90 | 69 | Kotresh Hiremath Ravishankar | h3. 2022 AUG 11 |
91 | |||
92 | https://pulpito.ceph.com/yuriw-2022-08-11_16:57:01-fs-wip-yuri3-testing-2022-08-11-0809-pacific-distro-default-smithi/ |
||
93 | * Most of the failures are passed in re-run. Please check rerun failures below. |
||
94 | - https://tracker.ceph.com/issues/57147 - test_full_fsync (tasks.cephfs.test_full.TestClusterFull) - mds stuck in up:creating state - osd crash |
||
95 | - https://tracker.ceph.com/issues/52624 - cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||
96 | - https://tracker.ceph.com/issues/51282 - cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log |
||
97 | - https://tracker.ceph.com/issues/50224 - test_mirroring_init_failure_with_recovery - (tasks.cephfs.test_mirroring.TestMirroring) |
||
98 | - https://tracker.ceph.com/issues/57083 - https://tracker.ceph.com/issues/53360 - tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} |
||
99 | - https://tracker.ceph.com/issues/51183 - Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) |
||
100 | - https://tracker.ceph.com/issues/56507 - Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation) |
||
101 | |||
102 | Re-run-1: https://pulpito.ceph.com/yuriw-2022-08-15_13:45:54-fs-wip-yuri3-testing-2022-08-11-0809-pacific-distro-default-smithi/ |
||
103 | * https://tracker.ceph.com/issues/52624 |
||
104 | cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||
105 | * tasks/mirror - Command failed on smithi078 with status 123: "sudo find /var/log/ceph -name '*.log' -print0 ..." - Asked for re-run. |
||
106 | * https://tracker.ceph.com/issues/57083 |
||
107 | * https://tracker.ceph.com/issues/53360 |
||
108 | tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} - tasks.cephfs.fuse_mount:mount command failed |
||
109 | client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
110 | * https://tracker.ceph.com/issues/51183 |
||
111 | Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) |
||
112 | * https://tracker.ceph.com/issues/56507 |
||
113 | Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation) |
||
114 | |||
115 | 68 | Kotresh Hiremath Ravishankar | |
116 | h3. 2022 AUG 04 |
||
117 | |||
118 | https://pulpito.ceph.com/yuriw-2022-08-04_20:54:08-fs-wip-yuri6-testing-2022-08-04-0617-pacific-distro-default-smithi/ |
||
119 | |||
120 | * https://tracker.ceph.com/issues/57087 |
||
121 | test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) |
||
122 | * https://tracker.ceph.com/issues/52624 |
||
123 | cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||
124 | * https://tracker.ceph.com/issues/51267 |
||
125 | tasks/workunit/snaps failure - Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status |
||
126 | * https://tracker.ceph.com/issues/53360 |
||
127 | * https://tracker.ceph.com/issues/57083 |
||
128 | qa/import-legacy tasks.cephfs.fuse_mount:mount command failed - :ModuleNotFoundError: No module named 'ceph_volume_client' |
||
129 | * https://tracker.ceph.com/issues/56507 |
||
130 | Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation) |
||
131 | |||
132 | |||
133 | 67 | Kotresh Hiremath Ravishankar | h3. 2022 July 15 |
134 | |||
135 | https://pulpito.ceph.com/yuriw-2022-07-21_22:57:30-fs-wip-yuri2-testing-2022-07-15-0755-pacific-distro-default-smithi |
||
136 | Re-run: https://pulpito.ceph.com/yuriw-2022-07-24_15:34:38-fs-wip-yuri2-testing-2022-07-15-0755-pacific-distro-default-smithi |
||
137 | |||
138 | * https://tracker.ceph.com/issues/52624 |
||
139 | qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" |
||
140 | * https://tracker.ceph.com/issues/57083 |
||
141 | * https://tracker.ceph.com/issues/53360 |
||
142 | tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} - tasks.cephfs.fuse_mount:mount command failed |
||
143 | client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
144 | * https://tracker.ceph.com/issues/51183 |
||
145 | Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) |
||
146 | * https://tracker.ceph.com/issues/56507 |
||
147 | pacific: Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation) |
||
148 | |||
149 | |||
150 | 66 | Venky Shankar | h3. 2022 July 08 |
151 | |||
152 | * https://tracker.ceph.com/issues/52624 |
||
153 | qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" |
||
154 | * https://tracker.ceph.com/issues/53360 |
||
155 | pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
156 | * https://tracker.ceph.com/issues/51183 |
||
157 | Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) |
||
158 | * https://tracker.ceph.com/issues/56506 |
||
159 | pacific: Test failure: test_rebuild_backtraceless (tasks.cephfs.test_data_scan.TestDataScan) |
||
160 | * https://tracker.ceph.com/issues/56507 |
||
161 | pacific: Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation) |
||
162 | |||
163 | 65 | Venky Shankar | h3. 2022 Jun 28 |
164 | |||
165 | https://pulpito.ceph.com/?branch=wip-yuri3-testing-2022-06-22-1121-pacific |
||
166 | |||
167 | * https://tracker.ceph.com/issues/52624 |
||
168 | qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" |
||
169 | * https://tracker.ceph.com/issues/53360 |
||
170 | pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
171 | * https://tracker.ceph.com/issues/51183 |
||
172 | Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) |
||
173 | |||
174 | 64 | Venky Shankar | h3. 2022 Jun 22 |
175 | |||
176 | https://pulpito.ceph.com/?branch=wip-yuri4-testing-2022-06-21-0704-pacific |
||
177 | |||
178 | * https://tracker.ceph.com/issues/52624 |
||
179 | qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" |
||
180 | * https://tracker.ceph.com/issues/53360 |
||
181 | pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
182 | * https://tracker.ceph.com/issues/51183 |
||
183 | Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) |
||
184 | |||
185 | 63 | Venky Shankar | h3. 2022 Jun 17 |
186 | |||
187 | https://pulpito.ceph.com/yuriw-2022-06-15_17:11:32-fs-wip-yuri10-testing-2022-06-15-0732-pacific-distro-default-smithi/ |
||
188 | |||
189 | * https://tracker.ceph.com/issues/52624 |
||
190 | qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" |
||
191 | * https://tracker.ceph.com/issues/53360 |
||
192 | pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
193 | * https://tracker.ceph.com/issues/51183 |
||
194 | Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) |
||
195 | |||
196 | 62 | Venky Shankar | h3. 2022 Jun 16 |
197 | |||
198 | https://pulpito.ceph.com/yuriw-2022-06-15_17:08:32-fs-wip-yuri-testing-2022-06-14-0744-pacific-distro-default-smithi/ |
||
199 | |||
200 | * https://tracker.ceph.com/issues/52624 |
||
201 | qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" |
||
202 | * https://tracker.ceph.com/issues/53360 |
||
203 | pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
204 | * https://tracker.ceph.com/issues/55449 |
||
205 | pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh |
||
206 | * https://tracker.ceph.com/issues/51267 |
||
207 | CommandFailedError: Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status 1:... |
||
208 | * https://tracker.ceph.com/issues/55332 |
||
209 | Failure in snaptest-git-ceph.sh (it's an async unlink/create bug) |
||
210 | |||
211 | 61 | Venky Shankar | h3. 2022 Jun 15 |
212 | |||
213 | http://pulpito.front.sepia.ceph.com/yuriw-2022-06-09_13:32:29-fs-wip-yuri10-testing-2022-06-08-0730-pacific-distro-default-smithi/ |
||
214 | |||
215 | * https://tracker.ceph.com/issues/52624 |
||
216 | qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" |
||
217 | * https://tracker.ceph.com/issues/53360 |
||
218 | pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
219 | * https://tracker.ceph.com/issues/55449 |
||
220 | pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh |
||
221 | |||
222 | |||
223 | 60 | Venky Shankar | h3. 2022 Jun 10 |
224 | |||
225 | https://pulpito.ceph.com/?branch=wip-yuri5-testing-2022-06-07-1529-pacific |
||
226 | |||
227 | * https://tracker.ceph.com/issues/52624 |
||
228 | qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" |
||
229 | * https://tracker.ceph.com/issues/53360 |
||
230 | pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
231 | * https://tracker.ceph.com/issues/55449 |
||
232 | pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh |
||
233 | |||
234 | 59 | Venky Shankar | h3. 2022 Jun 09 |
235 | |||
236 | https://pulpito.ceph.com/yuriw-2022-06-07_16:05:25-fs-wip-yuri4-testing-2022-06-01-1350-pacific-distro-default-smithi/ |
||
237 | |||
238 | * https://tracker.ceph.com/issues/52624 |
||
239 | qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" |
||
240 | * https://tracker.ceph.com/issues/53360 |
||
241 | pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
242 | * https://tracker.ceph.com/issues/55449 |
||
243 | pacific: mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh |
||
244 | |||
245 | 58 | Venky Shankar | h3. 2022 May 06 |
246 | |||
247 | https://pulpito.ceph.com/?sha1=73636a1b00037ff974bcdc969b009c5ecec626cc |
||
248 | |||
249 | * https://tracker.ceph.com/issues/52624 |
||
250 | qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" |
||
251 | * https://tracker.ceph.com/issues/53360 |
||
252 | pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
253 | |||
254 | 57 | Venky Shankar | h3. 2022 April 18 |
255 | |||
256 | http://pulpito.front.sepia.ceph.com/?branch=wip-yuri4-testing-2022-04-18-0609-pacific |
||
257 | (only mgr/snap_schedule backport pr) |
||
258 | |||
259 | * https://tracker.ceph.com/issues/52624 |
||
260 | qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" |
||
261 | 56 | Venky Shankar | |
262 | h3. 2022 March 28 |
||
263 | |||
264 | http://pulpito.front.sepia.ceph.com/yuriw-2022-03-25_20:52:40-fs-wip-yuri4-testing-2022-03-25-1203-pacific-distro-default-smithi/ |
||
265 | http://pulpito.front.sepia.ceph.com/yuriw-2022-03-26_19:52:48-fs-wip-yuri4-testing-2022-03-25-1203-pacific-distro-default-smithi/ |
||
266 | |||
267 | * https://tracker.ceph.com/issues/52624 |
||
268 | qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" |
||
269 | * https://tracker.ceph.com/issues/53360 |
||
270 | pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
271 | * https://tracker.ceph.com/issues/54411 |
||
272 | mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; |
||
273 | 33 daemons have recently crashed" during suites/fsstress.sh |
||
274 | |||
275 | 55 | Venky Shankar | h3. 2022 March 25 |
276 | |||
277 | https://pulpito.ceph.com/yuriw-2022-03-22_18:42:28-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/ |
||
278 | https://pulpito.ceph.com/yuriw-2022-03-23_14:04:56-fs-wip-yuri8-testing-2022-03-22-0910-pacific-distro-default-smithi/ |
||
279 | |||
280 | * https://tracker.ceph.com/issues/52624 |
||
281 | qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" |
||
282 | * https://tracker.ceph.com/issues/52606 |
||
283 | qa: test_dirfrag_limit |
||
284 | * https://tracker.ceph.com/issues/51905 |
||
285 | qa: "error reading sessionmap 'mds1_sessionmap'" |
||
286 | * https://tracker.ceph.com/issues/53360 |
||
287 | pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
288 | * https://tracker.ceph.com/issues/51183 |
||
289 | Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) |
||
290 | 47 | Patrick Donnelly | |
291 | 54 | Venky Shankar | h3. 2022 March 22 |
292 | |||
293 | https://pulpito.ceph.com/yuriw-2022-03-17_15:03:06-fs-wip-yuri10-testing-2022-03-16-1432-pacific-distro-default-smithi/ |
||
294 | |||
295 | * https://tracker.ceph.com/issues/52624 |
||
296 | qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" |
||
297 | * https://tracker.ceph.com/issues/52606 |
||
298 | qa: test_dirfrag_limit |
||
299 | * https://tracker.ceph.com/issues/51183 |
||
300 | Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) |
||
301 | * https://tracker.ceph.com/issues/51905 |
||
302 | qa: "error reading sessionmap 'mds1_sessionmap'" |
||
303 | * https://tracker.ceph.com/issues/53360 |
||
304 | pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
305 | * https://tracker.ceph.com/issues/54411 |
||
306 | mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; |
||
307 | 33 daemons have recently crashed" during suites/fsstress.sh |
||
308 | 47 | Patrick Donnelly | |
309 | 46 | Kotresh Hiremath Ravishankar | h3. 2021 November 22 |
310 | |||
311 | http://pulpito.front.sepia.ceph.com/yuriw-2021-11-12_00:33:28-fs-wip-yuri7-testing-2021-11-11-1339-pacific-distro-basic-smithi |
||
312 | http://pulpito.front.sepia.ceph.com/yuriw-2021-11-08_20:21:11-fs-wip-yuri5-testing-2021-11-08-1003-pacific-distro-basic-smithi |
||
313 | http://pulpito.front.sepia.ceph.com/yuriw-2021-11-08_15:19:37-fs-wip-yuri2-testing-2021-11-06-1322-pacific-distro-basic-smithi |
||
314 | |||
315 | |||
316 | * https://tracker.ceph.com/issues/53300 |
||
317 | qa: cluster [WRN] Scrub error on inode |
||
318 | * https://tracker.ceph.com/issues/53302 |
||
319 | qa: sudo logrotate /etc/logrotate.d/ceph-test.conf failed with status 1 |
||
320 | * https://tracker.ceph.com/issues/53314 |
||
321 | qa: fs/upgrade/mds_upgrade_sequence test timeout |
||
322 | * https://tracker.ceph.com/issues/53316 |
||
323 | qa: (smithi150) slow request osd_op, currently waiting for sub ops warning |
||
324 | * https://tracker.ceph.com/issues/52624 |
||
325 | qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" |
||
326 | * https://tracker.ceph.com/issues/52396 |
||
327 | pacific: qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable) |
||
328 | * https://tracker.ceph.com/issues/52875 |
||
329 | pacific: qa: test_dirfrag_limit |
||
330 | * https://tracker.ceph.com/issues/51705 |
||
331 | 1 | Patrick Donnelly | pacific: qa: tasks.cephfs.fuse_mount:mount command failed |
332 | * https://tracker.ceph.com/issues/39634 |
||
333 | qa: test_full_same_file timeout |
||
334 | 46 | Kotresh Hiremath Ravishankar | * https://tracker.ceph.com/issues/49748 |
335 | gibba: Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds |
||
336 | 48 | Kotresh Hiremath Ravishankar | * https://tracker.ceph.com/issues/51964 |
337 | qa: test_cephfs_mirror_restart_sync_on_blocklist failure |
||
338 | 49 | Kotresh Hiremath Ravishankar | * https://tracker.ceph.com/issues/50223 |
339 | qa: "client.4737 isn't responding to mclientcaps(revoke)" |
||
340 | 48 | Kotresh Hiremath Ravishankar | |
341 | 47 | Patrick Donnelly | |
342 | h3. 2021 November 20 |
||
343 | |||
344 | https://pulpito.ceph.com/?branch=wip-pdonnell-testing-20211120.024903-pacific |
||
345 | |||
346 | * https://tracker.ceph.com/issues/53360 |
||
347 | pacific: client: "handle_auth_bad_method server allowed_methods [2] but i only support [2]" |
||
348 | 46 | Kotresh Hiremath Ravishankar | |
349 | 41 | Patrick Donnelly | h3. 2021 September 14 (QE) |
350 | |||
351 | https://pulpito.ceph.com/yuriw-2021-09-10_15:01:12-fs-pacific-distro-basic-smithi/ |
||
352 | 1 | Patrick Donnelly | http://pulpito.front.sepia.ceph.com/yuriw-2021-09-13_14:09:59-fs-pacific-distro-basic-smithi/ |
353 | 43 | Ramana Raja | https://pulpito.ceph.com/yuriw-2021-09-14_15:09:45-fs-pacific-distro-basic-smithi/ |
354 | 41 | Patrick Donnelly | |
355 | * https://tracker.ceph.com/issues/52606 |
||
356 | qa: test_dirfrag_limit |
||
357 | * https://tracker.ceph.com/issues/52607 |
||
358 | qa: "mon.a (mon.0) 1022 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY)" |
||
359 | 44 | Ramana Raja | * https://tracker.ceph.com/issues/51705 |
360 | qa: tasks.cephfs.fuse_mount:mount command failed |
||
361 | 41 | Patrick Donnelly | |
362 | 40 | Ramana Raja | h3. 2021 Sep 7 |
363 | |||
364 | https://pulpito.ceph.com/yuriw-2021-09-07_17:38:38-fs-wip-yuri-testing-2021-09-07-0905-pacific-distro-basic-smithi/ |
||
365 | https://pulpito.ceph.com/yuriw-2021-09-07_23:57:28-fs-wip-yuri-testing-2021-09-07-0905-pacific-distro-basic-smithi/ |
||
366 | |||
367 | |||
368 | * https://tracker.ceph.com/issues/52396 |
||
369 | qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable) |
||
370 | * https://tracker.ceph.com/issues/51705 |
||
371 | qa: tasks.cephfs.fuse_mount:mount command failed |
||
372 | 4 | Patrick Donnelly | |
373 | 34 | Ramana Raja | h3. 2021 Aug 30 |
374 | |||
375 | https://pulpito.ceph.com/?branch=wip-yuri8-testing-2021-08-30-0930-pacific |
||
376 | |||
377 | * https://tracker.ceph.com/issues/45434 |
||
378 | qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed |
||
379 | * https://tracker.ceph.com/issues/51705 |
||
380 | qa: tasks.cephfs.fuse_mount:mount command failed |
||
381 | * https://tracker.ceph.com/issues/52396 |
||
382 | qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable) |
||
383 | 35 | Ramana Raja | * https://tracker.ceph.com/issues/52487 |
384 | 36 | Ramana Raja | qa: Test failure: test_deep_split (tasks.cephfs.test_fragment.TestFragmentation) |
385 | * https://tracker.ceph.com/issues/51267 |
||
386 | 37 | Ramana Raja | Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) |
387 | * https://tracker.ceph.com/issues/48772 |
||
388 | qa: pjd: not ok 9, 44, 80 |
||
389 | 35 | Ramana Raja | |
390 | 34 | Ramana Raja | |
391 | 31 | Ramana Raja | h3. 2021 Aug 23 |
392 | |||
393 | https://pulpito.ceph.com/yuriw-2021-08-23_19:33:26-fs-wip-yuri4-testing-2021-08-23-0812-pacific-distro-basic-smithi/ |
||
394 | |||
395 | * https://tracker.ceph.com/issues/45434 |
||
396 | qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed |
||
397 | * https://tracker.ceph.com/issues/51705 |
||
398 | qa: tasks.cephfs.fuse_mount:mount command failed |
||
399 | 32 | Ramana Raja | * https://tracker.ceph.com/issues/52396 |
400 | 33 | Ramana Raja | qa: ERROR: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable) |
401 | * https://tracker.ceph.com/issues/52397 |
||
402 | qa: test_acls (tasks.cephfs.test_acls.TestACLs) failed |
||
403 | |||
404 | 31 | Ramana Raja | |
405 | 29 | Ramana Raja | h3. 2021 Aug 11 |
406 | |||
407 | https://pulpito.ceph.com/yuriw-2021-08-11_14:28:23-fs-wip-yuri8-testing-2021-08-09-0844-pacific-distro-basic-smithi/ |
||
408 | https://pulpito.ceph.com/yuriw-2021-08-12_15:32:35-fs-wip-yuri8-testing-2021-08-09-0844-pacific-distro-basic-smithi/ |
||
409 | |||
410 | * https://tracker.ceph.com/issues/45434 |
||
411 | qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed |
||
412 | * https://tracker.ceph.com/issues/51705 |
||
413 | qa: tasks.cephfs.fuse_mount:mount command failed |
||
414 | 30 | Ramana Raja | * https://tracker.ceph.com/issues/50222 |
415 | osd: 5.2s0 deep-scrub : stat mismatch |
||
416 | 29 | Ramana Raja | |
417 | 19 | Jos Collin | h3. 2021 July 15 |
418 | 20 | Jos Collin | |
419 | 19 | Jos Collin | https://pulpito.ceph.com/yuriw-2021-07-13_17:37:59-fs-wip-yuri-testing-2021-07-13-0812-pacific-distro-basic-smithi/ |
420 | |||
421 | 26 | Jos Collin | * https://tracker.ceph.com/issues/45434 |
422 | qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed |
||
423 | * https://tracker.ceph.com/issues/51705 |
||
424 | qa: tasks.cephfs.fuse_mount:mount command failed |
||
425 | 27 | Jos Collin | * https://tracker.ceph.com/issues/51183 |
426 | Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) |
||
427 | 28 | Jos Collin | * https://tracker.ceph.com/issues/50528 |
428 | qa: fs:thrash: pjd suite not ok 80 |
||
429 | * https://tracker.ceph.com/issues/51706 |
||
430 | qa: osd deep-scrub stat mismatch |
||
431 | 26 | Jos Collin | |
432 | 19 | Jos Collin | h3. 2021 July 13 |
433 | 20 | Jos Collin | |
434 | 19 | Jos Collin | https://pulpito.ceph.com/yuriw-2021-07-08_23:33:26-fs-wip-yuri2-testing-2021-07-08-1142-pacific-distro-basic-smithi/ |
435 | |||
436 | 21 | Jos Collin | * https://tracker.ceph.com/issues/51704 |
437 | Test failure: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth) |
||
438 | 23 | Jos Collin | * https://tracker.ceph.com/issues/45434 |
439 | qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed |
||
440 | 24 | Jos Collin | * https://tracker.ceph.com/issues/51705 |
441 | 25 | Jos Collin | qa: tasks.cephfs.fuse_mount:mount command failed |
442 | * https://tracker.ceph.com/issues/48640 |
||
443 | 24 | Jos Collin | qa: snapshot mismatch during mds thrashing |
444 | 21 | Jos Collin | |
445 | 18 | Patrick Donnelly | h3. 2021 June 29 (Integration Branch) |
446 | |||
447 | https://pulpito.ceph.com/yuriw-2021-06-29_00:15:33-fs-wip-yuri3-testing-2021-06-28-1259-pacific-distro-basic-smithi/ |
||
448 | |||
449 | Some failures caused by incomplete backup of: https://github.com/ceph/ceph/pull/42065 |
||
450 | Some package failures caused by nautilus missing package, like: https://pulpito.ceph.com/yuriw-2021-06-29_00:15:33-fs-wip-yuri3-testing-2021-06-28-1259-pacific-distro-basic-smithi/6241300/ |
||
451 | |||
452 | * https://tracker.ceph.com/issues/45434 |
||
453 | qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed |
||
454 | * https://tracker.ceph.com/issues/50260 |
||
455 | pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty" |
||
456 | * https://tracker.ceph.com/issues/51183 |
||
457 | qa: FileNotFoundError: [Errno 2] No such file or directory: '/sys/kernel/debug/ceph/3fab6bea-f243-47a4-a956-8c03a62b61b5.client4721/mds_sessions' |
||
458 | |||
459 | |||
460 | 16 | Jeff Layton | h3. 2021 June 28 |
461 | |||
462 | https://pulpito.ceph.com/yuriw-2021-06-29_00:14:14-fs-wip-yuri-testing-2021-06-28-1259-pacific-distro-basic-smithi/ |
||
463 | |||
464 | * https://tracker.ceph.com/issues/45434 |
||
465 | qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed |
||
466 | * https://tracker.ceph.com/issues/51440 |
||
467 | 17 | Jeff Layton | fallocate fails with EACCES |
468 | 16 | Jeff Layton | * https://tracker.ceph.com/issues/51264 |
469 | TestVolumeClient failure |
||
470 | * https://tracker.ceph.com/issues/51266 |
||
471 | test cleanup failure |
||
472 | * https://tracker.ceph.com/issues/51183 |
||
473 | Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) |
||
474 | |||
475 | 12 | Jeff Layton | h3. 2021 June 14 |
476 | |||
477 | https://pulpito.ceph.com/yuriw-2021-06-14_16:21:07-fs-wip-yuri2-testing-2021-06-14-0717-pacific-distro-basic-smithi/ |
||
478 | |||
479 | * https://tracker.ceph.com/issues/45434 |
||
480 | qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed |
||
481 | * https://bugzilla.redhat.com/show_bug.cgi?id=1973276 |
||
482 | Could not reconnect to ubuntu@smithi076.front.sepia.ceph.com |
||
483 | * https://tracker.ceph.com/issues/51263 |
||
484 | pjdfstest rename test 10.t failed with EACCES |
||
485 | * https://tracker.ceph.com/issues/51264 |
||
486 | TestVolumeClient failure |
||
487 | 13 | Jeff Layton | * https://tracker.ceph.com/issues/51266 |
488 | Command failed on smithi204 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' |
||
489 | 14 | Jeff Layton | * https://tracker.ceph.com/issues/50279 |
490 | qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c" |
||
491 | 15 | Jeff Layton | * https://tracker.ceph.com/issues/51267 |
492 | Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status 1 |
||
493 | 11 | Patrick Donnelly | |
494 | h3. 2021 June 07 (Integration Branch) |
||
495 | |||
496 | http://pulpito.front.sepia.ceph.com/?branch=wip-yuri4-testing-2021-06-07-0955-pacific |
||
497 | |||
498 | * https://tracker.ceph.com/issues/45434 |
||
499 | qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed |
||
500 | * https://tracker.ceph.com/issues/50279 |
||
501 | qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c" |
||
502 | * https://tracker.ceph.com/issues/48773 |
||
503 | qa: scrub does not complete |
||
504 | * https://tracker.ceph.com/issues/51170 |
||
505 | pacific: qa: AttributeError: 'RemoteProcess' object has no attribute 'split' |
||
506 | * https://tracker.ceph.com/issues/48203 (stock kernel update required) |
||
507 | qa: quota failure |
||
508 | |||
509 | |||
510 | 6 | Patrick Donnelly | h3. 2021 Apr 28 (QE pre-release) |
511 | |||
512 | https://pulpito.ceph.com/yuriw-2021-04-28_19:27:30-fs-pacific-distro-basic-smithi/ |
||
513 | https://pulpito.ceph.com/yuriw-2021-04-30_13:46:58-fs-pacific-distro-basic-smithi/ |
||
514 | |||
515 | * https://tracker.ceph.com/issues/45434 |
||
516 | qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed |
||
517 | * https://tracker.ceph.com/issues/50258 |
||
518 | pacific: qa: "run() got an unexpected keyword argument 'stdin_data'" |
||
519 | * https://tracker.ceph.com/issues/50260 |
||
520 | pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty" |
||
521 | * https://tracker.ceph.com/issues/49962 |
||
522 | 'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes |
||
523 | * https://tracker.ceph.com/issues/50016 |
||
524 | qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes" |
||
525 | * https://tracker.ceph.com/issues/48203 (stock kernel update required) |
||
526 | qa: quota failure |
||
527 | * https://tracker.ceph.com/issues/50528 |
||
528 | pacific: qa: fs:thrash: pjd suite not ok 20 |
||
529 | |||
530 | |||
531 | 3 | Patrick Donnelly | h3. 2021 Apr 22 (Integration Branch) |
532 | |||
533 | https://pulpito.ceph.com/yuriw-2021-04-22_16:46:11-fs-wip-yuri5-testing-2021-04-20-0819-pacific-distro-basic-smithi/ |
||
534 | |||
535 | * https://tracker.ceph.com/issues/50527 |
||
536 | pacific: qa: untar_snap_rm timeout during osd thrashing (ceph-fuse) |
||
537 | * https://tracker.ceph.com/issues/50528 |
||
538 | pacific: qa: fs:thrash: pjd suite not ok 20 |
||
539 | * https://tracker.ceph.com/issues/49500 (fixed in another integration run) |
||
540 | qa: "Assertion `cb_done' failed." |
||
541 | * https://tracker.ceph.com/issues/49500 |
||
542 | qa: "Assertion `cb_done' failed." |
||
543 | * https://tracker.ceph.com/issues/48203 (stock kernel update required) |
||
544 | qa: quota failure |
||
545 | * https://tracker.ceph.com/issues/45434 |
||
546 | qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed |
||
547 | * https://tracker.ceph.com/issues/50279 |
||
548 | qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c" |
||
549 | * https://tracker.ceph.com/issues/50258 |
||
550 | pacific: qa: "run() got an unexpected keyword argument 'stdin_data'" |
||
551 | * https://tracker.ceph.com/issues/49962 |
||
552 | 'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes |
||
553 | * https://tracker.ceph.com/issues/49962 |
||
554 | 'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes |
||
555 | * https://tracker.ceph.com/issues/50530 |
||
556 | pacific: client: abort after MDS blocklist |
||
557 | |||
558 | 2 | Patrick Donnelly | |
559 | h3. 2021 Apr 21 (Integration Branch) |
||
560 | |||
561 | https://pulpito.ceph.com/yuriw-2021-04-21_19:25:00-fs-wip-yuri3-testing-2021-04-21-0937-pacific-distro-basic-smithi/ |
||
562 | |||
563 | * https://tracker.ceph.com/issues/45434 |
||
564 | qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed |
||
565 | * https://tracker.ceph.com/issues/50250 |
||
566 | mds: "log [WRN] : Scrub error on inode 0x10000004506 (/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.a log and `damage ls` output for details" |
||
567 | * https://tracker.ceph.com/issues/50258 |
||
568 | pacific: qa: "run() got an unexpected keyword argument 'stdin_data'" |
||
569 | * https://tracker.ceph.com/issues/48203 (stock kernel update required) |
||
570 | qa: quota failure |
||
571 | * https://tracker.ceph.com/issues/50016 |
||
572 | qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes" |
||
573 | * https://tracker.ceph.com/issues/50495 |
||
574 | pacific: client: shutdown race fails with status 141 |
||
575 | |||
576 | |||
577 | 1 | Patrick Donnelly | h3. 2021 Apr 07 (Integration Branch) |
578 | |||
579 | https://pulpito.ceph.com/yuriw-2021-04-07_17:37:43-fs-wip-yuri-testing-2021-04-07-0905-pacific-distro-basic-smithi/ |
||
580 | |||
581 | * https://tracker.ceph.com/issues/45434 |
||
582 | qa: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) failed |
||
583 | * https://tracker.ceph.com/issues/48805 |
||
584 | mds: "cluster [WRN] Scrub error on inode 0x1000000039d (/client.0/tmp/blogbench-1.0/src/blogtest_in) see mds.a log and `damage ls` output for details" |
||
585 | * https://tracker.ceph.com/issues/49500 |
||
586 | qa: "Assertion `cb_done' failed." |
||
587 | * https://tracker.ceph.com/issues/50258 (new) |
||
588 | pacific: qa: "run() got an unexpected keyword argument 'stdin_data'" |
||
589 | * https://tracker.ceph.com/issues/49962 |
||
590 | 'sudo ceph --cluster ceph osd crush tunables default' fails due to valgrind: Unknown option: --exit-on-first-error=yes |
||
591 | * https://tracker.ceph.com/issues/48203 (stock kernel update required) |
||
592 | qa: quota failure |
||
593 | * https://tracker.ceph.com/issues/50260 |
||
594 | pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty" |
||
595 | * https://tracker.ceph.com/issues/50016 |
||
596 | qa: test_damage: "RuntimeError: 2 mutations had unexpected outcomes" |