Project

General

Profile

Bug #55808

task/test_nfs: KeyError: 'events'

Added by Laura Flores 8 months ago. Updated 6 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
backport_processed
Backport:
quincy
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

/a/yuriw-2022-05-27_21:59:17-rados-wip-yuri-testing-2022-05-27-0934-distro-default-smithi/6851013

2022-05-27T22:51:21.158 INFO:tasks.cephfs_test_runner:======================================================================
2022-05-27T22:51:21.158 INFO:tasks.cephfs_test_runner:ERROR: test_update_export (tasks.cephfs.test_nfs.TestNFS)
2022-05-27T22:51:21.158 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2022-05-27T22:51:21.159 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2022-05-27T22:51:21.159 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_a847be1e97ec9c953b2352ff329516e826d4d048/qa/tasks/cephfs/test_nfs.py", line 642, in test_update_export
2022-05-27T22:51:21.159 INFO:tasks.cephfs_test_runner:    if not self._check_nfs_cluster_event('restart'):
2022-05-27T22:51:21.159 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_a847be1e97ec9c953b2352ff329516e826d4d048/qa/tasks/cephfs/test_nfs.py", line 91, in _check_nfs_cluster_event
2022-05-27T22:51:21.159 INFO:tasks.cephfs_test_runner:    for event in daemons_details[0]['events']:
2022-05-27T22:51:21.159 INFO:tasks.cephfs_test_runner:KeyError: 'events'
2022-05-27T22:51:21.159 INFO:tasks.cephfs_test_runner:
2022-05-27T22:51:21.159 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------


Related issues

Copied to Orchestrator - Backport #56584: quincy: task/test_nfs: KeyError: 'events' Resolved

History

#1 Updated by Laura Flores 8 months ago

  • Project changed from Orchestrator to CephFS
  • Component(FS) mgr/nfs added

#2 Updated by Laura Flores 8 months ago

/a/yuriw-2022-06-03_14:09:08-rados-wip-yuri7-testing-2022-06-02-1633-distro-default-smithi/6862503

#3 Updated by Ramana Raja 8 months ago

  • Assignee set to Ramana Raja

#4 Updated by Laura Flores 8 months ago

  • Related to Bug #56000: task/test_nfs: ERROR: Daemon not found: mds.a.smithi060.ujwxef. See `cephadm ls` added

#5 Updated by Ramana Raja 8 months ago

In the logs,

2022-06-03T15:24:59.294 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph orch ps --service_name=nfs.test --format=json

'events' key is missing in the output of `ceph orch ps --service_name=nfs.test --format=json` command.

2022-06-03T15:24:59.670 INFO:teuthology.orchestra.run.smithi036.stdout:[{"container_id": "320b4a67e293", "container_image_digests": ["quay.ceph.io/ceph-ci/ceph@sha256:76c5b9629cc5219226e93c4f515f3c20ae34e55b1ec935b2d8316b0939ee88a2"], "container_image_id": "cde988ec5559676e7b593f63c6eed00fd55715b7ec061d379984cfb9176b68d7", "container_image_name": "quay.ceph.io/ceph-ci/ceph@sha256:76c5b9629cc5219226e93c4f515f3c20ae34e55b1ec935b2d8316b0939ee88a2", "cpu_percentage": "0.20%", "created": "2022-06-03T15:23:03.782604Z", "daemon_id": "test.0.0.smithi036.xvvoix", "daemon_name": "nfs.test.0.0.smithi036.xvvoix", "daemon_type": "nfs", "hostname": "smithi036", "is_active": false, "last_refresh": "2022-06-03T15:24:51.711974Z", "memory_usage": 44449136, "ports": [2049], "rank": 0, "rank_generation": 0, "service_name": "nfs.test", "started": "2022-06-03T15:23:03.886913Z", "status": 1, "status_desc": "running", "version": "3.5"}]

2022-06-03T15:24:59.672 INFO:tasks.cephfs.test_nfs:daemons details [{'container_id': '320b4a67e293', 'container_image_digests': ['quay.ceph.io/ceph-ci/ceph@sha256:76c5b9629cc5219226e93c4f515f3c20ae34e55b1ec935b2d8316b0939ee88a2'], 'container_image_id': 'cde988ec5559676e7b593f63c6eed00fd55715b7ec061d379984cfb9176b68d7', 'container_image_name': 'quay.ceph.io/ceph-ci/ceph@sha256:76c5b9629cc5219226e93c4f515f3c20ae34e55b1ec935b2d8316b0939ee88a2', 'cpu_percentage': '0.20%', 'created': '2022-06-03T15:23:03.782604Z', 'daemon_id': 'test.0.0.smithi036.xvvoix', 'daemon_name': 'nfs.test.0.0.smithi036.xvvoix', 'daemon_type': 'nfs', 'hostname': 'smithi036', 'is_active': False, 'last_refresh': '2022-06-03T15:24:51.711974Z', 'memory_usage': 44449136, 'ports': [2049], 'rank': 0, 'rank_generation': 0, 'service_name': 'nfs.test', 'started': '2022-06-03T15:23:03.886913Z', 'status': 1, 'status_desc': 'running', 'version': '3.5'}]

I'm not sure why "events" key is not in the output of `ceph orch ps --format=json` command. I see it locally with ceph main branch using vstart cluster.

#7 Updated by Laura Flores 8 months ago

@Ramana we can re-file this as an Orchestrator bug if you believe it's a problem with cephadm.

#8 Updated by Ramana Raja 8 months ago

  • Project changed from CephFS to Orchestrator

This most likely is a cephadm bug, where `ceph orch ps ceph --service_name=nfs.<nfs cluster name> --format=json` does not output "events" when adm agent is on. Changing this to an orchestrator bug.

#9 Updated by Ramana Raja 8 months ago

  • Assignee deleted (Ramana Raja)

#10 Updated by Laura Flores 8 months ago

  • Backport set to quincy

/a/yuriw-2022-06-16_16:41:04-rados-wip-yuri6-testing-2022-06-16-0651-quincy-distro-default-smithi/6882321$

#11 Updated by Adam King 7 months ago

  • Status changed from New to Pending Backport
  • Pull request ID set to 46897

#12 Updated by Backport Bot 7 months ago

  • Copied to Backport #56584: quincy: task/test_nfs: KeyError: 'events' added

#13 Updated by Laura Flores 6 months ago

  • Related to deleted (Bug #56000: task/test_nfs: ERROR: Daemon not found: mds.a.smithi060.ujwxef. See `cephadm ls`)

#14 Updated by Laura Flores 6 months ago

  • Duplicated by Bug #56000: task/test_nfs: ERROR: Daemon not found: mds.a.smithi060.ujwxef. See `cephadm ls` added

#15 Updated by Backport Bot 6 months ago

  • Tags set to backport_processed

#16 Updated by Adam King 6 months ago

  • Status changed from Pending Backport to Resolved

#17 Updated by Laura Flores about 2 months ago

  • Duplicated by deleted (Bug #56000: task/test_nfs: ERROR: Daemon not found: mds.a.smithi060.ujwxef. See `cephadm ls`)

Also available in: Atom PDF