Project

General

Profile

Actions

Bug #22536

closed

client:_rmdir() uses a deleted memory structure(Dentry) leading a core

Added by Ivan Guan over 6 years ago. Updated about 6 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
% Done:

0%

Source:
Tags:
Backport:
luminous,jewel
Regression:
No
Severity:
1 - critical
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Client
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Version: ceph-10.2.2

Bug description:
"::rmdir()" acquires the Dentry structure "by get_or_create(dir, name, &de)" and when do "_do_lookup" may release the client_lock which the dentry memory very likely be deleted by others.

Log:
2017-12-19 23:11:48.525406 7efd862ea700 3 client.6924 ll_rmdir 100002d1047.head d3d
2017-12-19 23:11:48.525409 7efd862ea700 3 client.6924 rmdir(100002d1047 d3d uid 0 gid 0)
//get_or_create acquires de from dir dentries :0x7efda8cfe6f0,and then will lookup indoe,bug make_request will release Client_lock
2017-12-19 23:11:48.525449 7efd6e2a5700 3 client.6924 ll_forget 100002d065b 1
2017-12-19 23:11:48.525458 7efd69a9c700 3 client.6924 ll_forget 100002d1047 1
2017-12-19 23:11:48.525539 7efd77abf700 3 client.6924 ll_lookup 0x7efdb90e8d00 f20 -> 0 (100002d0b10)
2017-12-19 23:11:48.525564 7efd77abf700 3 client.6924 ll_forget 100002d0971 1
2017-12-19 23:11:48.525570 7efd7e4d5700 3 client.6924 ll_getattr 100002d0b10.head
2017-12-19 23:11:48.525574 7efd7e4d5700 3 client.6924 ll_getattr 100002d0b10.head = 0
2017-12-19 23:11:48.525612 7efd7e4d5700 3 client.6924 ll_forget 100002d0b10 1
2017-12-19 23:11:48.525622 7efd83ae5700 3 client.6924 ll_open 100002d0b10.head 32769
2017-12-19 23:11:48.525652 7efd83ae5700 0 Ivan dentry.put on 0x7efda8cfdce0 f5e now 0
2017-12-19 23:11:48.525654 7efd83ae5700 0 Ivan dentry.put on 0x7efda8cfe6f0 d3d now 0
//the bad guy put the ref to zero and the dentry will be deleted.
2017-12-19 23:11:48.525660 7efd83ae5700 0 Ivan dentry.put on 0x7efdacbc5f10 f20 now 0
2017-12-19 23:11:48.525663 7efd83ae5700 3 client.6924 ll_open 100002d0b10.head 32769 = 0 (0x7efdb280cf00)
2017-12-19 23:11:48.525672 7efd83ae5700 3 client.6924 ll_forget 100002d0b10 1
2017-12-19 23:11:48.525685 7efd702a9700 3 client.6924 ll_getattr 100002d0b10.head
2017-12-19 23:11:48.525687 7efd702a9700 3 client.6924 ll_getattr 100002d0b10.head = 0
2017-12-19 23:11:48.525693 7efd702a9700 3 client.6924 ll_forget 100002d0b10 11
2017-12-19 23:11:48.525862 7efd54a72700 3 client.6924 ll_getxattr 100002d0b10.head security.capability size 0
2017-12-19 23:11:48.525876 7efd54a72700 3 client.6924 _getxattr(100002d0b10, "security.capability", 0) = -61
2017-12-19 23:11:48.525889 7efd54a72700 3 client.6924 ll_forget 100002d0b10 1
2017-12-19 23:11:48.526149 7efd76abd700 3 client.6924 ll_write 0x7efdb280cf00 100002d0b10 960693~125837
2017-12-19 23:11:48.526878 7efd8b5f7700 0 Ivan dentry.get on 0x7efdaa5ec5b0 d3d now 2
2017-12-19 23:11:48.860657 7efd862ea700 -1 client/Dentry.h: In function 'void Dentry::get()' thread 7efd862ea700 time 2017-12-19 23:11:48.559681
//there will use a deleted dentry memory structure
client/Dentry.h: 55: FAILED assert(ref > 0)
ceph version attr-v1-file-share-op-code-55-gef21802 (ef21802a2ab88f8f64cab3edc9fe0a65c483281e)
1: (ceph::
_ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7efd949f67cb]
2: (Dentry::get()+0x2ca) [0x7efd9487d14a]
3: (Client::_rmdir(Inode*, char const*, int, int)+0x4c7) [0x7efd948681a7]
4: (Client::ll_rmdir(Inode*, char const*, int, int)+0xc5) [0x7efd948687e5]
5: (()+0x199754) [0x7efd947fc754]
6: (()+0x16beb) [0x7efd94218beb]
7: (()+0x13481) [0x7efd94215481]
8: (()+0x7e25) [0x7efd933fce25]
9: (clone()+0x6d) [0x7efd922e434d]


Related issues 2 (0 open2 closed)

Copied to CephFS - Backport #22699: luminous: client:_rmdir() uses a deleted memory structure(Dentry) leading a coreResolvedZheng YanActions
Copied to CephFS - Backport #22700: jewel: client:_rmdir() uses a deleted memory structure(Dentry) leading a coreResolvedPrashant DActions
Actions #2

Updated by Patrick Donnelly over 6 years ago

  • Subject changed from ceph-fuse ::_rmdir() uses a deleted memorey sturcture(Dentry) leading a core to client:_rmdir() uses a deleted memory structure(Dentry) leading a core
  • Status changed from New to Fix Under Review
  • Backport set to luminous,jewel
  • Component(FS) Client added
Actions #3

Updated by Patrick Donnelly over 6 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions #4

Updated by Nathan Cutler over 6 years ago

  • Copied to Backport #22699: luminous: client:_rmdir() uses a deleted memory structure(Dentry) leading a core added
Actions #5

Updated by Nathan Cutler over 6 years ago

  • Copied to Backport #22700: jewel: client:_rmdir() uses a deleted memory structure(Dentry) leading a core added
Actions #6

Updated by Nathan Cutler about 6 years ago

  • Status changed from Pending Backport to Resolved
Actions

Also available in: Atom PDF