https://tracker.ceph.com/
https://tracker.ceph.com/favicon.ico
2020-05-19T01:50:18Z
Ceph
CephFS - Bug #45593: qa: removing network bridge appears to cause dropped packets
https://tracker.ceph.com/issues/45593?journal_id=166095
2020-05-19T01:50:18Z
Xiubo Li
xiubli@redhat.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li></ul><p>It seems not the removing NAT rule's issue, this began very early and last for minutes already:</p>
<pre><code class="c syntaxhl"><span class="CodeRay"><span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.797</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/oauthlib/openid/connect/core/grant_types/
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.797</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/oauthlib/openid/connect/core/grant_types/__init__.py
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.797</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/oauthlib/openid/connect/core/grant_types/authorization_code.py
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.797</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/oauthlib/openid/connect/core/grant_types/base.py
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.924</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/oauthlib/openid/connect/core/grant_types/dispatchers.py
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.924</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/oauthlib/openid/connect/core/grant_types/exceptions.py
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.924</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/oauthlib/openid/connect/core/grant_types/hybrid.py
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.924</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/oauthlib/openid/connect/core/grant_types/implicit.py
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.925</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/oauthlib/openid/connect/core/grant_types/__pycache__/
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.925</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/oauthlib/openid/connect/core/grant_types/__pycache__/__init__.cpython-<span class="integer">3</span><span class="float">8</span>.pyc
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.925</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/oauthlib/openid/connect/core/grant_types/__pycache__/authorization_code.cpython-<span class="integer">3</span><span class="float">8</span>.pyc
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.925</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/oauthlib/openid/connect/core/grant_types/__pycache__/base.cpython-<span class="integer">3</span><span class="float">8</span>.pyc
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.925</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/oauthlib/openid/connect/core/grant_types/__pycache__/dispatchers.cpython-<span class="integer">3</span><span class="float">8</span>.pyc
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.926</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/oauthlib/openid/connect/core/grant_types/__pycache__/exceptions.cpython-<span class="integer">3</span><span class="float">8</span>.pyc
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.926</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/oauthlib/openid/connect/core/grant_types/__pycache__/hybrid.cpython-<span class="integer">3</span><span class="float">8</span>.pyc
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.926</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/oauthlib/openid/connect/core/grant_types/__pycache__/implicit.cpython-<span class="integer">3</span><span class="float">8</span>.pyc
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.926</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/olefile-<span class="float">0</span><span class="float">.46</span>.egg-info/
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.926</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/olefile-<span class="float">0</span><span class="float">.46</span>.egg-info/PKG-INFO
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.927</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/olefile-<span class="float">0</span><span class="float">.46</span>.egg-info/dependency_links.txt
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.927</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/olefile-<span class="float">0</span><span class="float">.46</span>.egg-info/top_level.txt
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.927</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/olefile/
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.927</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/olefile/__init__.py
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">9</span><span class="float">.927</span> INFO:tasks.workunit.client<span class="float">.0</span>.smithi114.stdout:multiple_rsync_payload<span class="float">.115994</span>/python3/dist-packages/olefile/olefile.py
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.287</span> INFO:tasks.ceph.osd<span class="float">.1</span>.smithi114.stderr:<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.276</span>+<span class="octal">0000</span> <span class="float">7f</span><span class="integer">473</span>bcfe700 -<span class="integer">1</span> osd<span class="float">.1</span> <span class="integer">28</span> heartbeat_check: no reply from <span class="integer">17</span><span class="float">2</span><span class="float">.21</span><span class="float">.15</span><span class="float">.38</span>:<span class="integer">6820</span> osd<span class="float">.4</span> since back <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">8</span><span class="float">.623014</span>+<span class="octal">0000</span> front <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">8</span><span class="float">.623561</span>+<span class="octal">0000</span> (oldest deadline <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">3</span><span class="float">.922886</span>+<span class="octal">0000</span>)
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.288</span> INFO:tasks.ceph.osd<span class="float">.1</span>.smithi114.stderr:<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.276</span>+<span class="octal">0000</span> <span class="float">7f</span><span class="integer">473</span>bcfe700 -<span class="integer">1</span> osd<span class="float">.1</span> <span class="integer">28</span> heartbeat_check: no reply from <span class="integer">17</span><span class="float">2</span><span class="float">.21</span><span class="float">.15</span><span class="float">.38</span>:<span class="integer">6812</span> osd<span class="float">.5</span> since back <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">8</span><span class="float">.623455</span>+<span class="octal">0000</span> front <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">8</span><span class="float">.623480</span>+<span class="octal">0000</span> (oldest deadline <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">3</span><span class="float">.922886</span>+<span class="octal">0000</span>)
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.288</span> INFO:tasks.ceph.osd<span class="float">.1</span>.smithi114.stderr:<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.276</span>+<span class="octal">0000</span> <span class="float">7f</span><span class="integer">473</span>bcfe700 -<span class="integer">1</span> osd<span class="float">.1</span> <span class="integer">28</span> heartbeat_check: no reply from <span class="integer">17</span><span class="float">2</span><span class="float">.21</span><span class="float">.15</span><span class="float">.38</span>:<span class="integer">6804</span> osd<span class="float">.6</span> since back <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">8</span><span class="float">.627164</span>+<span class="octal">0000</span> front <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">8</span><span class="float">.627247</span>+<span class="octal">0000</span> (oldest deadline <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">3</span><span class="float">.922886</span>+<span class="octal">0000</span>)
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.289</span> INFO:tasks.ceph.osd<span class="float">.1</span>.smithi114.stderr:<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.276</span>+<span class="octal">0000</span> <span class="float">7f</span><span class="integer">473</span>bcfe700 -<span class="integer">1</span> osd<span class="float">.1</span> <span class="integer">28</span> heartbeat_check: no reply from <span class="integer">17</span><span class="float">2</span><span class="float">.21</span><span class="float">.15</span><span class="float">.38</span>:<span class="integer">6828</span> osd<span class="float">.7</span> since back <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">8</span><span class="float">.623311</span>+<span class="octal">0000</span> front <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">8</span><span class="float">.623426</span>+<span class="octal">0000</span> (oldest deadline <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">3</span><span class="float">.922886</span>+<span class="octal">0000</span>)
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.293</span> INFO:tasks.ceph.osd<span class="float">.3</span>.smithi114.stderr:<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.280</span>+<span class="octal">0000</span> <span class="float">7f</span><span class="integer">7</span>ae1214700 -<span class="integer">1</span> osd<span class="float">.3</span> <span class="integer">28</span> heartbeat_check: no reply from <span class="integer">17</span><span class="float">2</span><span class="float">.21</span><span class="float">.15</span><span class="float">.38</span>:<span class="integer">6820</span> osd<span class="float">.4</span> since back <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">6</span><span class="float">.468398</span>+<span class="octal">0000</span> front <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">6</span><span class="float">.469517</span>+<span class="octal">0000</span> (oldest deadline <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">2</span><span class="float">.367345</span>+<span class="octal">0000</span>)
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.293</span> INFO:tasks.ceph.osd<span class="float">.3</span>.smithi114.stderr:<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.280</span>+<span class="octal">0000</span> <span class="float">7f</span><span class="integer">7</span>ae1214700 -<span class="integer">1</span> osd<span class="float">.3</span> <span class="integer">28</span> heartbeat_check: no reply from <span class="integer">17</span><span class="float">2</span><span class="float">.21</span><span class="float">.15</span><span class="float">.38</span>:<span class="integer">6812</span> osd<span class="float">.5</span> since back <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">6</span><span class="float">.467510</span>+<span class="octal">0000</span> front <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">6</span><span class="float">.467465</span>+<span class="octal">0000</span> (oldest deadline <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">2</span><span class="float">.367345</span>+<span class="octal">0000</span>)
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.294</span> INFO:tasks.ceph.osd<span class="float">.3</span>.smithi114.stderr:<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.280</span>+<span class="octal">0000</span> <span class="float">7f</span><span class="integer">7</span>ae1214700 -<span class="integer">1</span> osd<span class="float">.3</span> <span class="integer">28</span> heartbeat_check: no reply from <span class="integer">17</span><span class="float">2</span><span class="float">.21</span><span class="float">.15</span><span class="float">.38</span>:<span class="integer">6804</span> osd<span class="float">.6</span> since back <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">6</span><span class="float">.468007</span>+<span class="octal">0000</span> front <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">6</span><span class="float">.468556</span>+<span class="octal">0000</span> (oldest deadline <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">2</span><span class="float">.367345</span>+<span class="octal">0000</span>)
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.294</span> INFO:tasks.ceph.osd<span class="float">.3</span>.smithi114.stderr:<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.280</span>+<span class="octal">0000</span> <span class="float">7f</span><span class="integer">7</span>ae1214700 -<span class="integer">1</span> osd<span class="float">.3</span> <span class="integer">28</span> heartbeat_check: no reply from <span class="integer">17</span><span class="float">2</span><span class="float">.21</span><span class="float">.15</span><span class="float">.38</span>:<span class="integer">6828</span> osd<span class="float">.7</span> since back <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">6</span><span class="float">.468678</span>+<span class="octal">0000</span> front <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">6</span><span class="float">.468054</span>+<span class="octal">0000</span> (oldest deadline <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">2</span><span class="float">.367345</span>+<span class="octal">0000</span>)
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.297</span> INFO:tasks.ceph.osd<span class="float">.2</span>.smithi114.stderr:<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">4</span><span class="float">.280</span>+<span class="octal">0000</span> <span class="float">7f</span><span class="integer">72</span>cece0700 -<span class="integer">1</span> osd<span class="float">.2</span> <span class="integer">28</span> heartbeat_check: no reply from <span class="integer">17</span><span class="float">2</span><span class="float">.21</span><span class="float">.15</span><span class="float">.38</span>:<span class="integer">6820</span> osd<span class="float">.4</span> since back <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">8</span><span class="float">.062805</span>+<span class="octal">0000</span> front <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">2</span><span class="float">8</span><span class="float">.062526</span>+<span class="octal">0000</span> (oldest deadline <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">38</span>:<span class="integer">5</span><span class="float">3</span><span class="float">.362437</span>+<span class="octal">0000</span>)
...
</span></code></pre>
<p>It should be the smith38 node was down or something else:</p>
<pre><code class="c syntaxhl"><span class="CodeRay">front or back, first ping sent <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T12:<span class="integer">59</span>:<span class="integer">5</span><span class="float">5</span><span class="float">.310340</span>+<span class="octal">0000</span> (oldest deadline <span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T13:<span class="octal">00</span>:<span class="integer">1</span><span class="float">5</span><span class="float">.310340</span>+<span class="octal">0000</span>)
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T13:<span class="octal">00</span>:<span class="integer">4</span><span class="float">4</span><span class="float">.439</span> INFO:tasks.ceph.osd<span class="float">.3</span>.smithi114.stderr:<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T13:<span class="octal">00</span>:<span class="integer">4</span><span class="float">4</span><span class="float">.434</span>+<span class="octal">0000</span> <span class="float">7f</span><span class="integer">7</span>ae1214700 -<span class="integer">1</span> osd<span class="float">.3</span> <span class="integer">28</span> get_health_metrics reporting <span class="integer">1</span> slow ops, oldest is osd_op(client<span class="float">.4160</span><span class="float">.0</span>:<span class="integer">29</span> <span class="float">2</span><span class="float">.2</span> <span class="integer">2</span>:<span class="float">4</span>e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=<span class="integer">16</span>b] snapc <span class="integer">0</span>=[] ondisk+read+known_if_redirected e28)
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T13:<span class="octal">00</span>:<span class="integer">4</span><span class="float">4</span><span class="float">.446</span> DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port <span class="integer">22</span> on <span class="integer">17</span><span class="float">2</span><span class="float">.21</span><span class="float">.15</span><span class="float">.38</span>
<span class="integer">2020</span>-<span class="octal">05</span>-<span class="integer">16</span>T13:<span class="octal">00</span>:<span class="integer">4</span><span class="float">4</span><span class="float">.447</span> DEBUG:tasks.ceph:Missed logrotate, node <span class="char">'s</span>mithi038<span class="char">' </span>is offline
</span></code></pre>
<p>As I remembered before the unsharing netns patches I have also hit this, just thought it should be the network issue and didn't pay any attention to it.</p>
CephFS - Bug #45593: qa: removing network bridge appears to cause dropped packets
https://tracker.ceph.com/issues/45593?journal_id=167146
2020-06-02T04:31:28Z
Xiubo Li
xiubli@redhat.com
<ul></ul><p>The netns was only set up on simithi114, but connections to smiithi038 test node was lost(node 'smithi038' is offline) from the deploy node and smithi114 at the same time.</p>
CephFS - Bug #45593: qa: removing network bridge appears to cause dropped packets
https://tracker.ceph.com/issues/45593?journal_id=168708
2020-06-22T11:30:49Z
Xiubo Li
xiubli@redhat.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Rejected</i></li></ul><p>This is not the ceph qa test suite's bug, the root cause is the node itself get lost.</p>