Project

General

Profile

Bug #608

mds: MDCache::create_system_inode()

Added by Wido den Hollander over 13 years ago. Updated over 7 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
Severity:
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

On a small test cluster I saw that my MDS was not coming up after a fresh mkcephfs, this is what the log showed:

2010-11-23 16:08:47.923106 7fa332d77710 mds0.6 mds has 1 queued contexts
2010-11-23 16:08:47.923115 7fa332d77710 mds0.6 0x299cab0
2010-11-23 16:08:47.923127 7fa332d77710 mds0.cache open_root
2010-11-23 16:08:47.923140 7fa332d77710 mds0.cache.dir(1) fetch on [dir 1 / [2,head] auth v=1 cv=0/0 dir_auth=0 state=1073741824 f(v0 1=0+1) n(v0 1=0+1) hs=0+0,ss=0+0 | subtree 0x29f5000]
2010-11-23 16:08:47.923157 7fa332d77710 2010-11-23 16:08:47.923160 mds0.cache.dir(1) add_waiter 2 0x299cb00 on [dir 1 / [2,head] auth v=1 cv=0/0 dir_auth=0 state=1073741824 f(v0 1=0+1) n(v0 1=0+1) hs=0+0,ss=0+0 | subtree waiter 0x29f5000]
2010-11-23 16:08:47.923175 7fa332d77710 mds0.cache.dir(1) auth_pin by 0x29f5000 on [dir 1 / [2,head] auth v=1 cv=0/0 dir_auth=0 ap=1+0+0 state=1073741824 f(v0 1=0+1) n(v0 1=0+1) hs=0+0,ss=0+0 | subtree waiter authpin 0x29f5000] count now 1 + 0
2010-11-23 16:08:47.923913 7fa332d77710 mds0.cache.dir(1) _fetched 545 bytes for [dir 1 / [2,head] auth v=1 cv=0/0 dir_auth=0 ap=1+0+0 state=1073741952 f(v0 1=0+1) n(v0 1=0+1) hs=0+0,ss=0+0 | subtree waiter authpin 0x29f5000] want_dn=
2010-11-23 16:08:47.923937 7fa332d77710 mds0.cache.dir(1) _fetched version 1, 545 bytes, 1 keys
2010-11-23 16:08:47.923947 7fa332d77710  mds0.cache.snaprealm(1 seq 1 0x29d9000) have_past_parents_open [1,head]
2010-11-23 16:08:47.923957 7fa332d77710  mds0.cache.snaprealm(1 seq 1 0x29d9000) build_snap_set [0,head] on snaprealm(1 seq 1 lc 0 cr 0 cps 1 snaps={} 0x29d9000)
2010-11-23 16:08:47.923976 7fa332d77710  mds0.cache.snaprealm(1 seq 1 0x29d9000) build_snap_trace my_snaps []
2010-11-23 16:08:47.923987 7fa332d77710  mds0.cache.snaprealm(1 seq 1 0x29d9000) check_cache rebuilt  seq 1 cached_seq 1 cached_last_created 0 cached_last_destroyed 0)
2010-11-23 16:08:47.924000 7fa332d77710 mds0.cache.dir(1) lookup (head, '.ceph')
2010-11-23 16:08:47.924015 7fa332d77710 mds0.cache.dir(1) add_primary_dentry [dentry #1/.ceph [2,head] auth (dversion lock) pv=0 v=1 inode=0x29e1240 0x2a063b0]
2010-11-23 16:08:47.924030 7fa332d77710 mds0.cache.dir(1) _fetched  got [dentry #1/.ceph [2,head] auth (dversion lock) pv=0 v=1 inode=0x29e1240 0x2a063b0] [inode 2 [...2,head] /.ceph/ auth v1 f() n() (iversion lock) 0x29e1240]
2010-11-23 16:08:47.924049 7fa332d77710 mds0.cache.dir(1) auth_unpin by 0x29f5000 on [dir 1 / [2,head] auth v=1 cv=0/0 dir_auth=0 state=1073741826|complete f(v0 1=0+1) n(v0 1=0+1) hs=1+0,ss=0+0 | child subtree waiter 0x29f5000] count now 0 + 0
2010-11-23 16:08:47.924066 7fa332d77710 mds0.cache.dir(1) finish_waiting mask 2 result 0 on [dir 1 / [2,head] auth v=1 cv=0/0 dir_auth=0 state=1073741826|complete f(v0 1=0+1) n(v0 1=0+1) hs=1+0,ss=0+0 | child subtree waiter 0x29f5000]
2010-11-23 16:08:47.924082 7fa332d77710 2010-11-23 16:08:47.924085 mds0.cache.dir(1) take_waiting mask 2 took 0x299cb00 tag 2 on [dir 1 / [2,head] auth v=1 cv=0/0 dir_auth=0 state=1073741826|complete f(v0 1=0+1) n(v0 1=0+1) hs=1+0,ss=0+0 | child subtree waiter 0x29f5000]
2010-11-23 16:08:47.924103 7fa332d77710 mds0.6 mds has 1 queued contexts
2010-11-23 16:08:47.924112 7fa332d77710 mds0.6 0x299cb00
2010-11-23 16:08:47.924121 7fa332d77710 mds0.cache open_root
2010-11-23 16:08:47.924130 7fa332d77710 mds0.cache populate_mydir [dir 100 ~mds0/ [2,head] auth v=1 cv=0/0 dir_auth=0 state=1073741826|complete f(v0 2=1+1) n(v0 2=1+1) hs=2+0,ss=0+0 | child subtree 0x29f5618]
2010-11-23 16:08:47.924181 7fa332d77710 mds0.cache.dir(100) lookup (head, 'stray0')
2010-11-23 16:08:47.924192 7fa332d77710 mds0.cache creating system inode with ino:600
mds/MDCache.cc: In function 'void MDCache::add_inode(CInode*)':
mds/MDCache.cc:189: FAILED assert(inode_map.count(in->vino()) == 0)
 ceph version 0.24~rc (commit:c327c6a2064f3a14f0221b53dc0c87d744d0e977)
 1: /usr/bin/cmds() [0x50f17d]
 2: (MDCache::create_system_inode(inodeno_t, int)+0x202) [0x51afe2]
 3: (MDCache::populate_mydir()+0x1e8) [0x546298]
 4: (MDS::_dispatch(Message*)+0x6e2) [0x4a7062]
 5: (MDS::ms_dispatch(Message*)+0x6d) [0x4ab3fd]
 6: (SimpleMessenger::dispatch_entry()+0x759) [0x4810e9]
 7: (SimpleMessenger::DispatchThread::entry()+0x1c) [0x478edc]
 8: (Thread::_entry_func(void*)+0xa) [0x48d78a]
 9: (()+0x69ca) [0x7fa33572d9ca]
 10: (clone()+0x6d) [0x7fa33449370d]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
*** Caught signal (ABRT) ***
 ceph version 0.24~rc (commit:c327c6a2064f3a14f0221b53dc0c87d744d0e977)
 1: (sigabrt_handler(int)+0x7d) [0x6ddf4d]
 2: (()+0x33af0) [0x7fa3343e0af0]
 3: (gsignal()+0x35) [0x7fa3343e0a75]
 4: (abort()+0x180) [0x7fa3343e45c0]
 5: (__gnu_cxx::__verbose_terminate_handler()+0x115) [0x7fa334c968e5]
 6: (()+0xcad16) [0x7fa334c94d16]
 7: (()+0xcad43) [0x7fa334c94d43]
 8: (()+0xcae3e) [0x7fa334c94e3e]
 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x448) [0x6c9bd8]
 10: /usr/bin/cmds() [0x50f17d]
 11: (MDCache::create_system_inode(inodeno_t, int)+0x202) [0x51afe2]
 12: (MDCache::populate_mydir()+0x1e8) [0x546298]
 13: (MDS::_dispatch(Message*)+0x6e2) [0x4a7062]
 14: (MDS::ms_dispatch(Message*)+0x6d) [0x4ab3fd]
 15: (SimpleMessenger::dispatch_entry()+0x759) [0x4810e9]
 16: (SimpleMessenger::DispatchThread::entry()+0x1c) [0x478edc]
 17: (Thread::_entry_func(void*)+0xa) [0x48d78a]
 18: (()+0x69ca) [0x7fa33572d9ca]
 19: (clone()+0x6d) [0x7fa33449370d]

I've ran cdebugpack and uploaded the data to logger.ceph.widodh.nl:/srv/ceph/issues/cmds_create_system_inode

If you want to check out the machine itself, it's noisy.ceph.widodh.nl, you can access it through the logger machine.

History

#1 Updated by Sage Weil over 13 years ago

  • Status changed from New to Resolved
  • Assignee set to Sage Weil

this should be fixed by commit:fc212548aea1d7f001b56ba096a79ba54b8a92c3

Thanks!

#2 Updated by Sage Weil over 13 years ago

  • Target version set to v0.24

#3 Updated by John Spray over 7 years ago

  • Project changed from Ceph to CephFS
  • Category deleted (1)
  • Target version deleted (v0.24)

Bulk updating project=ceph category=mds bugs so that I can remove the MDS category from the Ceph project to avoid confusion.

Also available in: Atom PDF