Project

General

Profile

Actions
Copy link

Bug #45423

open

api_tier_pp: [ FAILED ] LibRadosTwoPoolsPP.HitSetWrite

Added by Brad Hubbard about 4 years ago. Updated almost 2 years ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Myoungwon Oh
Category:
Tiering
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
backport_processed
Backport:
octopus, pacific
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
Ceph - v16.0.0
ceph-qa-suite:
Component(RADOS):
Pull request ID:
42564
Crash signature (v1):
Crash signature (v2):

Description

/a/yuriw-2020-05-05_15:20:13-rados-wip-yuri8-testing-2020-05-04-2117-octopus-distro-basic-smithi/5024839

2020-05-06T01:50:57.014 INFO:tasks.workunit.client.0.smithi184.stdout:              api_tier_pp: pg_num = 32
2020-05-06T01:50:57.014 INFO:tasks.workunit.client.0.smithi184.stdout:              api_tier_pp: checking for 186:6cac518f:::0:head
2020-05-06T01:50:57.014 INFO:tasks.workunit.client.0.smithi184.stdout:              api_tier_pp: checking for 186:02547ec2:::1:head
2020-05-06T01:50:57.014 INFO:tasks.workunit.client.0.smithi184.stdout:              api_tier_pp: checking for 186:f905c69b:::2:head
2020-05-06T01:50:57.014 INFO:tasks.workunit.client.0.smithi184.stdout:              api_tier_pp: checking for 186:cfc208b3:::3:head
2020-05-06T01:50:57.014 INFO:tasks.workunit.client.0.smithi184.stdout:              api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.1-267-gbdabc7742f5/rpm/el8/BUILD/ceph-15.2.1-267-gbdabc7742f5/src/test/librados/tier_cxx.cc:2521: Failure
2020-05-06T01:50:57.015 INFO:tasks.workunit.client.0.smithi184.stdout:              api_tier_pp: Value of: found
2020-05-06T01:50:57.015 INFO:tasks.workunit.client.0.smithi184.stdout:              api_tier_pp:   Actual: false
2020-05-06T01:50:57.015 INFO:tasks.workunit.client.0.smithi184.stdout:              api_tier_pp: Expected: true
2020-05-06T01:50:57.015 INFO:tasks.workunit.client.0.smithi184.stdout:              api_tier_pp: [  FAILED  ] LibRadosTwoPoolsPP.HitSetWrite (15983 ms)

Related issues 3 (1 open — 2 closed)

Related to RADOS - Bug #40119: api_tier_pp hung causing a dead jobNew

Actions
Copied to RADOS - Backport #52077: octopus: api_tier_pp: [ FAILED ] LibRadosTwoPoolsPP.HitSetWriteResolvedLaura FloresActions
Copied to RADOS - Backport #52078: pacific: api_tier_pp: [ FAILED ] LibRadosTwoPoolsPP.HitSetWriteResolvedLaura FloresActions
Actions
Copy link
 #1

Updated by Neha Ojha almost 4 years ago

  • Priority changed from High to Normal
Actions
Copy link
 #2

Updated by Brad Hubbard almost 4 years ago

  • Priority changed from Normal to High
  • Backport set to octopus

/ceph/teuthology-archive/yuriw-2020-07-13_23:00:15-rados-wip-yuri8-testing-2020-07-13-1946-octopus-distro-basic-smithi/5224000

Actions
Copy link
 #3

Updated by Neha Ojha over 3 years ago

/a/teuthology-2020-08-26_07:01:02-rados-master-distro-basic-smithi/5377120/

Actions
Copy link
 #4

Updated by Kefu Chai over 3 years ago

/a/kchai-2020-10-10_09:47:31-rados-wip-kefu-testing-2020-10-09-1210-distro-basic-smithi/5512894

Actions
Copy link
 #5

Updated by Neha Ojha over 3 years ago

  • Category set to Tiering
  • Priority changed from High to Normal
Actions
Copy link
 #6

Updated by Sage Weil about 3 years ago

/a/sage-2021-03-03_16:41:22-rados-wip-sage2-testing-2021-03-03-0744-pacific-distro-basic-smithi/5930113

Actions
Copy link
 #7

Updated by Deepika Upadhyay about 3 years ago 

2021-03-11T17:41:42.373 INFO:teuthology.orchestra.run.smithi129.stderr:2021-03-11T17:41:42.374+0000 7f9LibRadosTwoPoolsPP.HitSetWrite374fdb700  1 -- 172.21.15.129:0/211496500 shutdown_connections
2021-03-11T17:41:42.373 INFO:teuthology.orchestra.run.smithi129.stderr:2021-03-11T17:41:42.374+0000 7f9374fdb700  1 --2- 172.21.15.129:0/211496500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93701309e0 0x7f937009eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop
2021-03-11T17:41:42.374 INFO:teuthology.orchestra.run.smithi129.stderr:2021-03-11T17:41:42.374+0000 7f9374fdb700  1 --2- 172.21.15.129:0/211496500 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f93700a1bd0 0x7f937009f410 unknown :-1 s=CLOSED pgs=4061 cs=0 l=1 rev1=1 rx=0 tx=0).stop
2021-03-11T17:41:42.374 INFO:teuthology.orchestra.run.smithi129.stderr:2021-03-11T17:41:42.374+0000 7f9374fdb700  1 --2- 172.21.15.129:0/211496500 >> v2:172.21.15.129:6800/13629 conn(0x7f9350057fc0 0x7f935005a480 unknown :-1 s=CLOSED pgs=4905 cs=0 l=1 rev1=1 rx=0 tx=0).stop
2021-03-11T17:41:42.374 INFO:teuthology.orchestra.run.smithi129.stderr:2021-03-11T17:41:42.374+0000 7f9374fdb700  1 --2- 172.21.15.129:0/211496500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f93700a7710 0x7f937009e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop
2021-03-11T17:41:42.375 INFO:teuthology.orchestra.run.smithi129.stderr:2021-03-11T17:41:42.374+0000 7f9374fdb700  1 -- 172.21.15.129:0/211496500 >> 172.21.15.129:0/211496500 conn(0x7f93700aac90 msgr2=0x7f937006c790 unknown :-1 s=STATE_NONE l=0).mark_down
2021-03-11T17:41:42.375 INFO:teuthology.orchestra.run.smithi129.stderr:2021-03-11T17:41:42.374+0000 7f9374fdb700  1 -- 172.21.15.129:0/211496500 shutdown_connections
2021-03-11T17:41:42.375 INFO:teuthology.orchestra.run.smithi129.stderr:2021-03-11T17:41:42.374+0000 7f9374fdb700  1 -- 172.21.15.129:0/211496500 wait complete.
2021-03-11T17:41:42.375 INFO:teuthology.orchestra.run.smithi129.stderr:2021-03-11T17:41:42.374+0000 7f9374fdb700  1 librados: shutdown
2021-03-11T17:41:42.376 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.6 to 1 (10000)
2021-03-11T17:41:42.378 INFO:tasks.workunit.client.0.smithi129.stdout:              api_tier_pp: [       OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (5021 ms)
2021-03-11T17:41:42.378 INFO:tasks.workunit.client.0.smithi129.stdout:              api_tier_pp: [----------] 21 tests from LibRadosTwoPoolsECPP (252290 ms total)
2021-03-11T17:41:42.378 INFO:tasks.workunit.client.0.smithi129.stdout:              api_tier_pp:
2021-03-11T17:41:42.379 INFO:tasks.workunit.client.0.smithi129.stdout:              api_tier_pp: [----------] Global test environment tear-down
2021-03-11T17:41:42.379 INFO:tasks.workunit.client.0.smithi129.stdout:              api_tier_pp: [==========] 59 tests from 4 test suites ran. (680211 ms total)
2021-03-11T17:41:42.379 INFO:tasks.workunit.client.0.smithi129.stdout:              api_tier_pp: [  PASSED  ] 58 tests.
2021-03-11T17:41:42.379 INFO:tasks.workunit.client.0.smithi129.stdout:              api_tier_pp: [  FAILED  ] 1 test, listed below:
2021-03-11T17:41:42.380 INFO:tasks.workunit.client.0.smithi129.stdout:              api_tier_pp: [  FAILED  ] LibRadosTwoPoolsPP.HitSetWrite
2021-03-11T17:41:42.380 INFO:tasks.workunit.client.0.smithi129.stdout:              api_tier_pp:
2021-03-11T17:41:42.380 INFO:tasks.workunit.client.0.smithi129.stdout:              api_tier_pp:  1 FAILED TEST

/ceph/teuthology-archive/yuriw-2021-03-09_20:27:38-rados-wip-yuri2-testing-2021-03-09-1006-octopus-distro-basic-smithi/5950496/teuthology.log

Actions
Copy link
 #8

Updated by Sage Weil about 3 years ago

/a/sage-2021-03-20_15:11:51-rados-wip-sage2-testing-2021-03-20-0832-pacific-distro-basic-smithi/5983920

Actions
Copy link
 #9

Updated by Neha Ojha about 3 years ago

/a/sage-2021-04-06_20:05:38-rados-wip-sage-testing-2021-04-06-1403-distro-basic-smithi/6025407

Actions
Copy link
 #10

Updated by Sage Weil about 3 years ago

/a/sage-2021-05-05_15:58:13-rados-wip-sage-testing-2021-05-04-1814-distro-basic-smithi/6099487

Actions
Copy link
 #11

Updated by Brad Hubbard about 3 years ago

  • Related to Bug #40119: api_tier_pp hung causing a dead job added
Actions
Copy link
 #12

Updated by Brad Hubbard about 3 years ago

Possibly related to 40119

Actions
Copy link
 #13

Updated by Kefu Chai almost 3 years ago

/a/kchai-2021-05-28_13:33:45-rados-wip-kefu-testing-2021-05-28-1806-distro-basic-smithi/6140866

Actions
Copy link
 #14

Updated by Deepika Upadhyay almost 3 years ago

/ceph/teuthology-archive/yuriw-2021-06-14_19:20:57-rados-wip-yuri6-testing-2021-06-14-1106-octopus-distro-basic-smithi/6172250/teuthology.log

Actions
Copy link
 #15

Updated by Sridhar Seshasayee almost 3 years ago

/a/sseshasa-2021-07-14_10:37:09-rados-wip-sseshasa-testing-2021-07-14-1320-distro-basic-smithi/6270208

Actions
Copy link
 #16

Updated by Neha Ojha almost 3 years ago

  • Assignee set to Myoungwon Oh

Hi Myoungwon Oh, this is tiering test has been causing a lot of failures, do you think you can help investigate?

Actions
Copy link
 #17

Updated by Deepika Upadhyay almost 3 years ago

"/ceph/teuthology-archive/yuriw-2021-07-23_17:36:03-rados-wip-yuri6-testing-2021-07-23-0834-octopus-distro-basic-smithi/6288961/teuthology.log

Actions
Copy link
 #18

Updated by Deepika Upadhyay almost 3 years ago

/ceph/teuthology-archive/yuriw-2021-07-26_16:32:00-rados-wip-yuri5-testing-2021-07-26-0803-octopus-distro-basic-smithi/6294133/teuthology.log

Actions
Copy link
 #19

Updated by Myoungwon Oh almost 3 years ago

OK, I need to time to look at this.

Actions
Copy link
 #20

Updated by Neha Ojha almost 3 years ago

Myoungwon Oh wrote:

OK, I need to time to look at this.

sure, no worries, thanks!

Actions
Copy link
 #22

Updated by Neha Ojha almost 3 years ago

  • Status changed from New to Fix Under Review
  • Backport changed from octopus to octopus, pacific
  • Pull request ID set to 42564
Actions
Copy link
 #23

Updated by Kefu Chai almost 3 years ago

/a//kchai-2021-08-03_15:40:41-rados-wip-kefu-testing-2021-08-03-2117-distro-basic-smithi/6309402

Actions
Copy link
 #24

Updated by Kefu Chai almost 3 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions
Copy link
 #25

Updated by Backport Bot almost 3 years ago

  • Copied to Backport #52077: octopus: api_tier_pp: [ FAILED ] LibRadosTwoPoolsPP.HitSetWrite added
Actions
Copy link
 #26

Updated by Backport Bot almost 3 years ago

  • Copied to Backport #52078: pacific: api_tier_pp: [ FAILED ] LibRadosTwoPoolsPP.HitSetWrite added
Actions
Copy link
 #27

Updated by Laura Flores about 2 years ago

Seems like this may have come back:
/a/dgalloway-2022-03-09_02:34:58-rados-wip-45272-distro-basic-smithi/6727572

Actions
Copy link
 #28

Updated by Laura Flores about 2 years ago

  • Related to Bug #50042: rados/test.sh: api_watch_notify failures added
Actions
Copy link
 #29

Updated by Neha Ojha about 2 years ago

  • Related to deleted (Bug #50042: rados/test.sh: api_watch_notify failures)
Actions
Copy link
 #30

Updated by Kamoltat (Junior) Sirivadhna about 2 years ago

/a/yuriw-2022-03-04_21:56:41-rados-wip-yuri4-testing-2022-03-03-1448-distro-default-smithi/6721464

Actions
Copy link
 #31

Updated by Backport Bot almost 2 years ago

  • Tags set to backport_processed
Actions
Copy link

Also available in: Atom PDF