Tools » teuthology

reference tracker ticket: http://tracker.ceph.com/issues/20251

Test case:
setup rgw multisite cluster with master and secondary zones.

1. On master, create ~20 buckets to generate some metadata sync load.
2. On secondary, use systemctl to restart radosgw a few times, maybe every ~60sec or so.

test result: ensure there is no crash in RGWMetaSyncShardCR::incremental_sync completion.

note: the fix for this ticket was shipped as cutomer hot fix in ceph version 10.2.7-28.0.hotfix.bz1476888.el7cp

we already have test scripts to be run on primary and secondary setup to run, all we need is include the test as part of rgw qa suite.

on master zone - createbuckets.py [run the script in master zone]

import sys
from time import sleep
import boto
import boto.s3.connection
access_key = '7ZYL6IDWBC1C1K3WBI6E'
secret_key = 'loL0logoEpqsKMynNlreeNDCpxkTvQS2ay2sXzw1'

conn = boto.connect_s3(
aws_access_key_id = access_key,
aws_secret_access_key = secret_key,
host = 'magna057.ceph.redhat.com',
is_secure=False, port=80,
calling_format = boto.s3.connection.OrdinaryCallingFormat(),
)

count = 0
while True:
count += 1
bname = 'bzthursbucketnumber%d' % count
bucket = conn.create_bucket(bname)
sleep(1)

on secondary zone - rgwstart.sh & [run the script in background, so it restarts rgw every 1 minute]

#! /bin/bash -fv
while true; do
systemctl restart ceph-radosgw@rgw.`hostname -s`
sleep 60
done

leave the test scripts running for 10 hours and ensure there is no segfault in rgw log [/var/log/ceph/ceph-rgw.<hostname>.log]