Project

General

Profile

Actions
Copy link

Bug #7704

closed

"[ERR] scrub mismatch" in upgrade:dumpling-x:parallel-firefly---basic-plana suite

Added by Yuri Weinstein about 10 years ago. Updated about 10 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Sage Weil
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Logs are in http://qa-proxy.ceph.com/teuthology/teuthology-2014-03-12_17:27:09-upgrade:dumpling-x:parallel-firefly---basic-plana/127868/

2014-03-12T19:30:56.771 WARNING:teuthology.task.ceph:Found errors (ERR|WRN|SEC) in cluster log

archive_path: /var/lib/teuthworker/archive/teuthology-2014-03-12_17:27:09-upgrade:dumpling-x:parallel-firefly---basic-plana/127868
description: upgrade/dumpling-x/parallel/{0-cluster/start.yaml 1-dumpling-install/dumpling.yaml
  2-workload/test_rbd_api.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-final-upgrade/client.yaml
  5-final-workload/rados_mon_thrash.yaml distros/ubuntu_12.04.yaml}
email: null
job_id: '127868'
last_in_suite: false
machine_type: plana
name: teuthology-2014-03-12_17:27:09-upgrade:dumpling-x:parallel-firefly---basic-plana
nuke-on-error: true
os_type: ubuntu
os_version: '12.04'
overrides:
  admin_socket:
    branch: firefly
  ceph:
    conf:
      mon:
        debug mon: 20
        debug ms: 1
        debug paxos: 20
        mon warn on legacy crush tunables: false
      osd:
        debug filestore: 20
        debug ms: 1
        debug osd: 20
    log-whitelist:
    - slow request
    sha1: b5d2df4a9247cbf7831dd82369fa4e04b3db1b9c
  ceph-deploy:
    branch:
      dev: firefly
    conf:
      client:
        log file: /var/log/ceph/ceph-$name.$pid.log
      mon:
        debug mon: 1
        debug ms: 20
        debug paxos: 20
        osd default pool size: 2
  install:
    ceph:
      sha1: b5d2df4a9247cbf7831dd82369fa4e04b3db1b9c
  s3tests:
    branch: master
  workunit:
    sha1: b5d2df4a9247cbf7831dd82369fa4e04b3db1b9c
owner: scheduled_teuthology@teuthology
roles:
- - mon.a
  - mds.a
  - osd.0
  - osd.1
- - mon.b
  - mon.c
  - osd.2
  - osd.3
- - client.0
  - client.1
targets:
  ubuntu@plana14.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCmXdWDw+LAohI86VIYUCTd8SKOj8HkKuVD9pBeNrwiLH11CT6NmuPSPvUhDpCa+IXHmvGXsdjgcKft4tt2OdvrFrJQszRne8iOtV6Zvjr2oU3A3zLGDifFmdMYw687qk7Yjt6qBRU9oyzxD6HtWxZqZwtLFpmmH1fozw7ZBLsaOx80dcVSlr3YY2HtqhFHGVj2Mikay8AmCDvkTygn9BsA4MBL0jkkAqF5fOydJYSWDScFBcJKEAH5gI/7LzWcoOFAdo14/NDxNIvsryBhsqUDc/qvVh+wrxhWVOpfQC0o6JONYFGKnZyxgcHdAzB+1QzgwPhTCQsSXNL5Lm/1Y5R7
  ubuntu@plana15.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaIHfpiG2pSr6O+YmgouHs73FZXANKSNcyDiNBfKZiBzcIHV+DovmZU8CT/p5RlztQTmUguZnYQ3uNGECu85BsVNaTDsdGgRP73VFDc5j6hf5c/7kIbdr4EJivKhjtQO0gRbJEqinT6/KD00LIfZ8NPmaJoCwg1D6rm3+vW+xWzMSzJmiMU5vQCUCWiQardS1u1vOHXKelXFftEFdFifOsA+KZD5zkcq2HeNQvHqryVxLFW31YfCizj+vO//IsL+gDH2+g2fieIYPYfuzjnx41Ve6lLvJfwnVl61Tk++cFbVrUWTp3Y6N2X42S6Frj2g6nieEWg2AlBrY7rDIoOVDh
  ubuntu@plana21.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUcR+fVqP6cDG9ax0r2ZiO1kX3BPmYU6jD3j44V9oejFMr7C3zCi82gsGcmbFEKz3JGYK5wWYrn6ueBxC1lH1FQO8qafvOuF5M87BU/l+AFAn8mqt6ohJZd5G+ze5+oxzDDHBeLBl0Mth7siHfODtTv7L8sOh/uPZRPUM0GH3wrftpu/adv1yvNdbgJGPoKSZNmDlvBfhvvBIBY4X8/5F8DfA8zEB1dQE96bIFaf56yoh7wDiumM56q4h4Hm7X9X+gLwYtw1BnLSqqqgGHhcQEwRhhXTP99iHswNJ0GDlqXpCcEgCwBqP1y7YbBS3p7SLsd4q4uG0+D9QV5Q9pfBIp
tasks:
- internal.lock_machines:
  - 3
  - plana
- internal.save_config: null
- internal.check_lock: null
- internal.connect: null
- internal.check_conflict: null
- internal.check_ceph_data: null
- internal.vm_setup: null
- internal.base: null
- internal.archive: null
- internal.coredump: null
- internal.sudo: null
- internal.syslog: null
- internal.timer: null
- chef: null
- clock.check: null
- install:
    branch: dumpling
- ceph:
    fs: xfs
- parallel:
  - workload
  - upgrade-sequence
- install.upgrade:
    client.0: null
- mon_thrash:
    revive_delay: 20
    thrash_delay: 1
- workunit:
    clients:
      client.1:
      - rados/test.sh
teuthology_branch: firefly
upgrade-sequence:
  sequential:
  - install.upgrade:
      mon.a: null
      mon.b: null
  - ceph.restart:
      daemons:
      - mon.a
      wait-for-healthy: false
      wait-for-osds-up: true
  - sleep:
      duration: 60
  - ceph.restart:
      daemons:
      - mon.b
      wait-for-healthy: false
      wait-for-osds-up: true
  - sleep:
      duration: 60
  - ceph.restart:
    - mon.c
  - sleep:
      duration: 60
  - ceph.restart:
    - osd.0
  - sleep:
      duration: 60
  - ceph.restart:
    - osd.1
  - sleep:
      duration: 60
  - ceph.restart:
    - osd.2
  - sleep:
      duration: 60
  - ceph.restart:
    - osd.3
  - sleep:
      duration: 60
  - ceph.restart:
    - mds.a
verbose: true
worker_log: /var/lib/teuthworker/archive/worker_logs/worker.plana.11160
workload:
  sequential:
  - workunit:
      branch: dumpling
      clients:
        client.0:
        - rbd/test_librbd.sh

description: upgrade/dumpling-x/parallel/{0-cluster/start.yaml 1-dumpling-install/dumpling.yaml
  2-workload/test_rbd_api.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-final-upgrade/client.yaml
  5-final-workload/rados_mon_thrash.yaml distros/ubuntu_12.04.yaml}
duration: 1735.6043090820312
failure_reason: '"2014-03-12 19:16:46.592710 mon.0 10.214.131.25:6789/0 17 : [ERR]
  scrub mismatch" in cluster log'
flavor: basic
owner: scheduled_teuthology@teuthology
success: false
Actions
Copy link
 #2

Updated by Sage Weil about 10 years ago

  • Status changed from New to In Progress
  • Assignee set to Sage Weil
  • Priority changed from High to Urgent
  • Source changed from other to Q/A

I think this is a manifestation of the full osdmap encoding features, which aren't specified for dumpling and can thus vary. Need to confirm and, if so, whitelist.

Actions
Copy link
 #3

Updated by Sage Weil about 10 years ago

  • Status changed from In Progress to Resolved
Actions
Copy link

Also available in: Atom PDF