Project

General

Profile

Actions
Copy link

Bug #55516

closed

qa: fs suite tests failing with "json.decoder.JSONDecodeError: Extra data: line 2 column 82 (char 82)"

Added by Venky Shankar almost 2 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Jos Collin
Category:
Testing
Target version:
Ceph - v18.0.0
% Done:

0%

Source:
Tags:
Backport:
quincy, pacific
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Labels (FS):
qa, qa-failure
Pull request ID:
46161
Crash signature (v1):
Crash signature (v2):

Description

Started seeing these failures recently:

- https://pulpito.ceph.com/vshankar-2022-05-02_09:11:25-fs-wip-vshankar-testing1-20220428-204527-testing-default-smithi/6818428/
- https://pulpito.ceph.com/vshankar-2022-05-02_09:11:25-fs-wip-vshankar-testing1-20220428-204527-testing-default-smithi/6818427/
- https://pulpito.ceph.com/vshankar-2022-05-02_09:11:25-fs-wip-vshankar-testing1-20220428-204527-testing-default-smithi/6818441/

Related issues 3 (0 open — 3 closed)

Related to RADOS - Feature #53050: Support blocklisting a CIDR rangeResolvedGreg Farnum

Actions
Copied to CephFS - Backport #55621: quincy: qa: fs suite tests failing with "json.decoder.JSONDecodeError: Extra data: line 2 column 82 (char 82)"ResolvedGreg FarnumActions
Copied to CephFS - Backport #55916: pacific: qa: fs suite tests failing with "json.decoder.JSONDecodeError: Extra data: line 2 column 82 (char 82)"ResolvedGreg FarnumActions
Actions
Copy link
 #1

Updated by Venky Shankar almost 2 years ago

From logs for a sample test:

2022-05-02T09:39:42.719 INFO:tasks.cephfs_test_runner:======================================================================
2022-05-02T09:39:42.719 INFO:tasks.cephfs_test_runner:ERROR: test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc)
2022-05-02T09:39:42.719 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2022-05-02T09:39:42.719 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2022-05-02T09:39:42.720 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_b82312f270e887c49dafe642c4ee39a73bc7d8fa/qa/tasks/cephfs/test_misc.py", line 188, in test_cap_revoke_nonresponder
2022-05-02T09:39:42.720 INFO:tasks.cephfs_test_runner:    self.mount_a.get_global_addr()))
2022-05-02T09:39:42.720 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_b82312f270e887c49dafe642c4ee39a73bc7d8fa/qa/tasks/cephfs/filesystem.py", line 274, in is_addr_blocklisted
2022-05-02T09:39:42.720 INFO:tasks.cephfs_test_runner:    stdout=StringIO()).stdout.getvalue())
2022-05-02T09:39:42.720 INFO:tasks.cephfs_test_runner:  File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
2022-05-02T09:39:42.720 INFO:tasks.cephfs_test_runner:    return _default_decoder.decode(s)
2022-05-02T09:39:42.720 INFO:tasks.cephfs_test_runner:  File "/usr/lib/python3.6/json/decoder.py", line 342, in decode
2022-05-02T09:39:42.720 INFO:tasks.cephfs_test_runner:    raise JSONDecodeError("Extra data", s, end)
2022-05-02T09:39:42.720 INFO:tasks.cephfs_test_runner:json.decoder.JSONDecodeError: Extra data: line 2 column 82 (char 82)

Seems to be happening when decoding `osd blocklist ls' json dump.

Actions
Copy link
 #2

Updated by Venky Shankar almost 2 years ago

Another instance: https://pulpito.ceph.com/vshankar-2022-05-02_16:58:59-fs-wip-vshankar-testing1-20220502-201957-testing-default-smithi/6818686/

2022-05-02T17:26:25.496 INFO:teuthology.orchestra.run.smithi003.stderr:2022-05-02T17:26:25.488+0000 7fcee7492700  1 -- 172.21.15.3:0/2304122625 shutdown_connections
2022-05-02T17:26:25.497 INFO:teuthology.orchestra.run.smithi003.stderr:2022-05-02T17:26:25.488+0000 7fcee7492700  1 -- 172.21.15.3:0/2304122625 wait complete.
2022-05-02T17:26:25.524 INFO:tasks.cephfs_test_runner:test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) ... ERROR
2022-05-02T17:26:25.524 INFO:tasks.cephfs_test_runner:
2022-05-02T17:26:25.524 INFO:tasks.cephfs_test_runner:======================================================================
2022-05-02T17:26:25.524 INFO:tasks.cephfs_test_runner:ERROR: test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc)
2022-05-02T17:26:25.524 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2022-05-02T17:26:25.524 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2022-05-02T17:26:25.525 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_af5caccc3a270ee8c7fe4530fa943e48e8028552/qa/tasks/cephfs/test_misc.py", line 188, in test_cap_revoke_nonresponder
2022-05-02T17:26:25.525 INFO:tasks.cephfs_test_runner:    self.mount_a.get_global_addr()))
2022-05-02T17:26:25.525 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_af5caccc3a270ee8c7fe4530fa943e48e8028552/qa/tasks/cephfs/filesystem.py", line 274, in is_addr_blocklisted
2022-05-02T17:26:25.525 INFO:tasks.cephfs_test_runner:    stdout=StringIO()).stdout.getvalue())
2022-05-02T17:26:25.525 INFO:tasks.cephfs_test_runner:  File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
2022-05-02T17:26:25.525 INFO:tasks.cephfs_test_runner:    return _default_decoder.decode(s)
2022-05-02T17:26:25.525 INFO:tasks.cephfs_test_runner:  File "/usr/lib/python3.6/json/decoder.py", line 342, in decode
2022-05-02T17:26:25.525 INFO:tasks.cephfs_test_runner:    raise JSONDecodeError("Extra data", s, end)
2022-05-02T17:26:25.525 INFO:tasks.cephfs_test_runner:json.decoder.JSONDecodeError: Extra data: line 2 column 81 (char 81)
Actions
Copy link
 #3

Updated by Jos Collin almost 2 years ago

  • Assignee changed from Rishabh Dave to Jos Collin
Actions
Copy link
 #4

Updated by Venky Shankar almost 2 years ago

Jos figured this failure is due to range based (CIDR) blocklisting changes.

Actions
Copy link
 #5

Updated by Jos Collin almost 2 years ago

  • Status changed from New to Fix Under Review
  • Pull request ID set to 46161
Actions
Copy link
 #6

Updated by Venky Shankar almost 2 years ago

  • Backport set to quincy
Actions
Copy link
 #7

Updated by Venky Shankar almost 2 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions
Copy link
 #8

Updated by Jos Collin almost 2 years ago

  • Copied to Backport #55621: quincy: qa: fs suite tests failing with "json.decoder.JSONDecodeError: Extra data: line 2 column 82 (char 82)" added
Actions
Copy link
 #9

Updated by Jos Collin almost 2 years ago

Actions
Copy link
 #10

Updated by Jos Collin almost 2 years ago

  • Backport changed from quincy to quincy, pacific
Actions
Copy link
 #11

Updated by Backport Bot almost 2 years ago

  • Copied to Backport #55916: pacific: qa: fs suite tests failing with "json.decoder.JSONDecodeError: Extra data: line 2 column 82 (char 82)" added
Actions
Copy link
 #12

Updated by Jos Collin almost 2 years ago

  • Status changed from Pending Backport to Resolved
Actions
Copy link

Also available in: Atom PDF