Project

General

Profile

Actions
Copy link

Bug #11781

closed

multiple_rsync failure with fuse client

Added by John Spray almost 9 years ago. Updated almost 9 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

The second rsync run is transferring some data, so either timestamps are going wrong, dentries are going missing, or the infrastructure is changing clocks beneath us.

http://pulpito.ceph.com/teuthology-2015-05-15_23:04:01-fs-master-testing-basic-multi/895391/

Actions
Copy link
 #1

Updated by John Spray almost 9 years ago

  • Project changed from devops to CephFS
  • Category set to 45

Wrong project, sorry.

Actions
Copy link
 #2

Updated by Greg Farnum almost 9 years ago

Possibly a dup of #9884? :/ I'm not sure how accurate that "too many files" is as the actual bug cause, though...

Actions
Copy link
 #3

Updated by John Spray almost 9 years ago

#9884 was the case where it timed out b/c there were just so many files, whereas in this instance it's actually failing outright.

Actions
Copy link
 #4

Updated by Zheng Yan almost 9 years ago

  • Status changed from New to Rejected
2015-05-17T06:48:31.565 INFO:tasks.workunit.client.0.plana96.stdout:sent 6,261,568,409 bytes  received 1,496,889 bytes  1,529,255.35 bytes/sec
2015-05-17T06:48:31.565 INFO:tasks.workunit.client.0.plana96.stdout:total size is 6,254,714,661  speedup is 1.00
2015-05-17T06:48:31.571 INFO:tasks.workunit.client.0.plana96.stderr:+ echo we should get 4 here if no additional files are transfered
2015-05-17T06:48:31.572 INFO:tasks.workunit.client.0.plana96.stdout:we should get 4 here if no additional files are transfered
2015-05-17T06:48:31.572 INFO:tasks.workunit.client.0.plana96.stderr:+ sudo rsync -auv --exclude local/ /usr/ usr.1
2015-05-17T06:48:31.572 INFO:tasks.workunit.client.0.plana96.stderr:+ tee /tmp/18207
2015-05-17T06:48:31.614 INFO:tasks.workunit.client.0.plana96.stdout:sending incremental file list
2015-05-17T06:50:15.753 INFO:tasks.workunit.client.0.plana96.stdout:share/doc/
2015-05-17T06:58:08.209 INFO:tasks.workunit.client.0.plana96.stdout:
2015-05-17T06:58:08.209 INFO:tasks.workunit.client.0.plana96.stdout:sent 2,426,491 bytes  received 11,316 bytes  4,221.31 bytes/sec
2015-05-17T06:58:08.209 INFO:tasks.workunit.client.0.plana96.stdout:total size is 6,254,711,077  speedup is 2,565.71
2015-05-17T06:58:08.213 INFO:tasks.workunit.client.0.plana96.stderr:+ hexdump -C /tmp/18207
2015-05-17T06:58:08.225 INFO:tasks.workunit.client.0.plana96.stdout:00000000  73 65 6e 64 69 6e 67 20  69 6e 63 72 65 6d 65 6e  |sending incremen|
2015-05-17T06:58:08.225 INFO:tasks.workunit.client.0.plana96.stdout:00000010  74 61 6c 20 66 69 6c 65  20 6c 69 73 74 0a 73 68  |tal file list.sh|
2015-05-17T06:58:08.225 INFO:tasks.workunit.client.0.plana96.stdout:00000020  61 72 65 2f 64 6f 63 2f  0a 0a 73 65 6e 74 20 32  |are/doc/..sent 2|
2015-05-17T06:58:08.226 INFO:tasks.workunit.client.0.plana96.stdout:00000030  2c 34 32 36 2c 34 39 31  20 62 79 74 65 73 20 20  |,426,491 bytes  |
2015-05-17T06:58:08.226 INFO:tasks.workunit.client.0.plana96.stdout:00000040  72 65 63 65 69 76 65 64  20 31 31 2c 33 31 36 20  |received 11,316 |
2015-05-17T06:58:08.226 INFO:tasks.workunit.client.0.plana96.stdout:00000050  62 79 74 65 73 20 20 34  2c 32 32 31 2e 33 31 20  |bytes  4,221.31 |
2015-05-17T06:58:08.226 INFO:tasks.workunit.client.0.plana96.stdout:00000060  62 79 74 65 73 2f 73 65  63 0a 74 6f 74 61 6c 20  |bytes/sec.total |
2015-05-17T06:58:08.227 INFO:tasks.workunit.client.0.plana96.stdout:00000070  73 69 7a 65 20 69 73 20  36 2c 32 35 34 2c 37 31  |size is 6,254,71|
2015-05-17T06:58:08.227 INFO:tasks.workunit.client.0.plana96.stdout:00000080  31 2c 30 37 37 20 20 73  70 65 65 64 75 70 20 69  |1,077  speedup i|
2015-05-17T06:58:08.227 INFO:tasks.workunit.client.0.plana96.stdout:00000090  73 20 32 2c 35 36 35 2e  37 31 0a                 |s 2,565.71.|
2015-05-17T06:58:08.227 INFO:tasks.workunit.client.0.plana96.stdout:0000009b

log for updating share/doc's mtime.

2015-05-17 04:51:01.413849 7f3f24ff1700  3 client.4125 ll_setattr 10000010209.head mask 18
2015-05-17 04:51:01.413864 7f3f24ff1700 10 client.4125 mark_caps_dirty 10000010209.head(ref=3 ll_ref=2123 cap_refs={} open={} mode=40755 size=0/0 mtime=2015-05-17 04:08:08.209612 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE parents=0x7f3ea000c9b0 0x7f3f1ab80a50) - -> Fx
2015-05-17 04:51:01.413879 7f3f24ff1700 10 client.4125 fill_stat on 10000010209 snap/devhead mode 040755 mtime 2015-05-17 04:08:08.209612 ctime 2015-05-17 04:51:01.413864


2015-05-17 06:50:09.324997 7f3f257f2700 10 client.4125 send_request client_request(unknown.0:631558 setattr mtime=2015-05-17 06:48:08.028113 atime=2015-05-17 06:50:09.317534 #10000010209 2015-05-17 06:50:09.324974) v2 to mds.0

you can see that /usr/share/doc's mtime changed (by someone else).

Actions
Copy link
 #5

Updated by John Spray almost 9 years ago

Hmm, maybe using /usr/ as a source of workload files is something we just need to stop doing, there's no particular reason to expect it to be constant

Actions
Copy link
 #6

Updated by Greg Farnum almost 9 years ago

#11807. I'm not actually sure what good options there are for replacement, but it won't get lost.

Actions
Copy link

Also available in: Atom PDF