Project

General

Profile

Bug #58395 ยป crash log.txt

zhikuo du, 01/09/2023 08:58 AM

 

6837902 -51> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 MDSContext::complete: 18C_MDS_RetryRequest
6837903 -50> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.server dispatch_client_request client_request(client.408135:34341420 create #0x30004bf4c4f/hydra_nameserver 2022-12-31T22:58:02.294645+0800 RETRY=74 caller_uid=0, caller_gid=0{}) v2
6837904 -49> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.server open w/ O_CREAT on #0x30004bf4c4f/hydra_nameserver
6837905 -48> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.server rdlock_path_xlock_dentry request(client.408135:34341420 nref=2 cr=0x55eb0d1cc480) #0x30004bf4c4f/hydra_nameserver
6837906 -47> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.cache traverse: opening base ino 0x30004bf4c4f snap head
6837907 -46> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker try_rdlock_snap_layout request(client.408135:34341420 nref=3 cr=0x55eb0d1cc480) [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75, 1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75,11=75,12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock) (iversion lock) caps={408135=pAsLsXs/p@0},l=408135 | lock=2 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300]
6837908 -45> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 12 mds.2.cache traverse: path seg depth 0 'hydra_nameserver' snapid head
6837909 -44> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 20 mds.2.cache.dir(0x30004bf4c4f) lookup (head, 'hydra_nameserver')
6837910 -43> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 20 mds.2.cache.dir(0x30004bf4c4f) hit -> (hydra_nameserver,head)
6837911 -42> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker acquire_locks request(client.408135:34341420 nref=3 cr=0x55eb0d1cc480)
6837912 -41> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 20 mds.2.locker must wrlock (ifile lock) [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75,11=75, 12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock) (iversion lock) caps={408135=pAsLsXs/p@0},l=408135 | lock=2 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300]
6837913 -40> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 20 mds.2.locker must wrlock (inest mix) [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75,11=75, 12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock) (iversion lock) caps={408135=pAsLsXs/p@0},l=408135 | lock=2 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300]
6837914 -39> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 20 mds.2.locker must rdlock (iauth sync) [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75,11=75, 12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock) (iversion lock) caps={408135=pAsLsXs/p@0},l=408135 | lock=2 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300]
6837915 -38> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 20 mds.2.locker must xlock (dn sync) [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dversion lock) pv=0 v=171188706 ap=1 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=0 authpin=1 0x55eb31037600]
6837916 -37> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 20 mds.2.locker must wrlock (dversion lock) [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dversion lock) pv=0 v=171188706 ap=1 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=0 authpin=1 0x55eb31037600]
6837917 -36> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker must authpin [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75,11=75,12=75,13=75, 14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock) (iversion lock) caps={408135=pAsLsXs/p@0},l=408135 | lock=2 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300]
6837918 -35> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker must authpin [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dversion lock) pv=0 v=171188706 ap=1 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=0 authpin=1 0x55eb31037600]
6837919 -34> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker already auth_pinned [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75,11=75,12=75, 13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock) (iversion lock) caps={408135=pAsLsXs/p@0},l=408135 | lock=2 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300]
6837920 -33> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker already auth_pinned [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dversion lock) pv=0 v=171188706 ap=1 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=0 authpin=1 0x55eb31037600]
6837921 -32> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker wrlock_start (ifile lock) on [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75,11=75, 12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock) (iversion lock) caps={408135=pAsLsXs/p@0},l=408135 | lock=2 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300]
6837922 -31> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker got wrlock on (ifile lock w=1) [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75, 11=75,12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock w=1) (iversion lock) caps={408135=pAsLsXs/p@ 0},l=408135 | lock=3 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300]
6837923 -30> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker wrlock_start (inest mix) on [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75,11=75, 12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock w=1) (iversion lock) caps={408135=pAsLsXs/p@0}, l=408135 | lock=3 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300]
6837924 -29> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker got wrlock on (inest mix w=1) [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75, 11=75,12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix w=1) (ipolicy sync r=1) (ifile lock w=1) (iversion lock) caps={408135=pAsLsXs/ p@0},l=408135 | lock=4 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300]
6837925 -28> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.locker rdlock_start on (iauth sync) on [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75, 11=75,12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix w=1) (ipolicy sync r=1) (ifile lock w=1) (iversion lock) caps={408135=pAsLsXs/ p@0},l=408135 | lock=4 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300]
6837926 -27> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker got rdlock on (iauth sync r=1) [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75, 11=75,12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (iauth sync r=1) (isnap sync r=1) (inest mix w=1) (ipolicy sync r=1) (ifile lock w=1) (iversion lock) caps={408135=pAsLsXs/p@0},l=408135 | lock=5 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300]
6837927 -26> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.locker xlock_start on (dn sync) on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dversion lock) pv=0 v=171188706 ap=1 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=0 authpin=1 0x55eb31037600]
6837928 -25> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.locker simple_lock on (dn sync) on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dversion lock) pv=0 v=171188706 ap=1 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=0 authpin=1 0x55eb31037600]
6837929 -24> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.locker simple_xlock on (dn lock) on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn lock) (dversion lock) pv=0 v=171188706 ap=1 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=0 authpin=1 0x55eb31037600]
6837930 -23> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.cache.den(0x30004bf4c4f hydra_nameserver) auth_pin by 0x55eb310376c0 on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn lock) (dversion lock) pv=0 v=171188706 ap=2 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=0 authpin=1 0x55eb31037600] now 2
6837931 -22> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 15 mds.2.cache.dir(0x30004bf4c4f) adjust_nested_auth_pins 1 on [dir 0x30004bf4c4f /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ [2,head] auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75, 10=75,11=75,12=75,13=75,14=75,15=75} v=171188707 cv=0/0 ap=0+2 state=1610612737|complete f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 182=182+0) hs=361+0,ss=0+0 dirty=62 | child=1 replicated=1 dirty=1 waiter=0 authpin=0 0x55eb0f092a00] by 0x55eb310376c0 count now 0/2
6837932 -21> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker got xlock on (dn xlock x=1 by 0x55eb1ba9a400) [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn xlock x=1 by 0x55eb1ba9a400) (dversion lock) pv=0 v=171188706 ap=2 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=1 authpin=1 0x55eb31037600]
6837933 -20> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.locker local_wrlock_start on (dversion lock) on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn xlock x=1 by 0x55eb1ba9a400) (dversion lock) pv=0 v=171188706 ap=2 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=1 authpin=1 0x55eb31037600]
6837934 -19> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker got wrlock on (dversion lock w=1 last_client=408135) [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn xlock x=1 by 0x55eb1ba9a400) (dversion lock w=1 last_client=408135) pv=0 v=171188706 ap=2 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=2 authpin=1 0x55eb31037600]
6837935 -18> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.cache linking in remote in [inode 0x30004be7edb [...2,head] ~mds4/stray1/30004be7edb rep@4.1 v172464532 snaprealm=0x55eb2dc0a500 s=2203120 n(v0 rc2022-12-31T22:44:39.135649+0800 b2203120 1=1+0) (iversion lock) 0x55eb2ce3bc00]
6837936 -17> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.cache.snaprealm(0x30004be7edb seq 1 0x55eb2dc0a500) have_past_parents_open [1,head]
6837937 -16> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.cache path_traverse finish on snapid head
6837938 -15> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker xlock_downgrade on (dn xlock x=1 by 0x55eb1ba9a400) [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn xlock x=1 by 0x55eb1ba9a400) (dversion lock w=1 last_client=408135) pv=0 v=171188706 ap=2 ino=0x30004be7edb state=1073741824 | ptrwaiter=0 request=1 lock=2 authpin=1 0x55eb31037600]
6837939 -14> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker xlock_finish on (dn xlockdone r=1 x=1) [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn xlockdone r=1 x=1) (dversion lock w=1 last_client=408135) pv=0 v=171188706 ap=2 ino=0x30004be7edb state=1073741824 | ptrwaiter=0 request=1 lock=3 authpin=1 0x55eb31037600]
6837940 -13> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker eval_gather (dn xlockdone r=1) on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn xlockdone r=1) (dversion lock w=1 last_client=408135) pv=0 v=171188706 ap=2 ino=0x30004be7edb state=1073741824 | ptrwaiter=0 request=1 lock=2 authpin=1 0x55eb31037600]
6837941 -12> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.locker eval_gather finished gather on (dn xlockdone r=1) on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn xlockdone r=1) (dversion lock w=1 last_client=408135) pv=0 v=171188706 ap=2 ino=0x30004be7edb state=1073741824 | ptrwaiter=0 request=1 lock=2 authpin=1 0x55eb31037600]
6837942 -11> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.cache.den(0x30004bf4c4f hydra_nameserver) auth_unpin by 0x55eb310376c0 on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn sync r=1) (dversion lock w=1 last_client=408135) pv=0 v=171188706 ap=1 ino=0x30004be7edb state=1073741824 | ptrwaiter=0 request=1 lock=2 authpin=1 0x55eb31037600] now 1
6837943 -10> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 15 mds.2.cache.dir(0x30004bf4c4f) adjust_nested_auth_pins -1 on [dir 0x30004bf4c4f /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ [2,head] auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75, 9=75,10=75,11=75,12=75,13=75,14=75,15=75} v=171188707 cv=0/0 ap=0+1 state=1610612737|complete f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 182=182+0) hs=361+0,ss=0+0 dirty=62 | child=1 replicated=1 dirty=1 waiter=0 authpin=0 0x55eb0f092a00] by 0x55eb310376c0 count now 0/1
6837944 -9> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker simple_eval (dn sync r=1) on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn sync r=1) (dversion lock w=1 last_client=408135) pv=0 v=171188706 ap=1 ino=0x30004be7edb state=1073741824 | ptrwaiter=0 request=1 lock=2 authpin=1 0x55eb31037600]

6837945 -8> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker acquire_locks request(client.408135:34341420 nref=2 cr=0x55eb0d1cc480)
6837946 -7> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 20 mds.2.locker must rdlock (isnap sync) [inode 0x30004be7edb [...2,head] ~mds4/stray1/30004be7edb rep@4.1 v172464532 snaprealm=0x55eb2dc0a500 s=2203120 n(v0 rc2022-12-31T22:44:39.135649+0800 b2203120 1=1+ 0) (iversion lock) | remoteparent=1 0x55eb2ce3bc00]
6837947 -6> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.locker rdlock_start on (isnap sync) on [inode 0x30004be7edb [...2,head] ~mds4/stray1/30004be7edb rep@4.1 v172464532 snaprealm=0x55eb2dc0a500 s=2203120 n(v0 rc2022-12-31T22:44:39.135649+0800 b2203120 1=1+0) (iversion lock) | remoteparent=1 0x55eb2ce3bc00]
6837948 -5> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker got rdlock on (isnap sync r=1) [inode 0x30004be7edb [...2,head] ~mds4/stray1/30004be7edb rep@4.1 v172464532 snaprealm=0x55eb2dc0a500 s=2203120 n(v0 rc2022-12-31T22:44:39.135649+0800 b2203120 1=1+0) (iversion lock) | lock=1 remoteparent=1 0x55eb2ce3bc00]

6837949 -4> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.server open on #0x30004bf4c4f/hydra_nameserver

6837950 -3> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.server rdlock_path_pin_ref request(client.408135:34341420 nref=2 cr=0x55eb0d1cc480) #0x30004bf4c4f/hydra_nameserver
6837951 -2> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.server open flags = 577, filemode = 2, need_auth = 1
6837952 -1> 2023-01-01T09:47:23.651+0800 7f2d7a4e0700 -1 /root/src/ceph/src/mds/Server.cc: In function 'void Server::handle_client_open(MDRequestRef&)' thread 7f2d7a4e0700 time 2023-01-01T09:47:23.649572+0800
6837953 /root/src/ceph/src/mds/Server.cc: 4164: FAILED ceph_assert(cur->is_auth())
6837954
6837955 ceph version 15.2.16 (d46a73d6d0a67a79558054a3a5a72cb561724974) octopus (stable)
6837956 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14c) [0x7f2d81fd9bc9]
6837957 2: (()+0x269d91) [0x7f2d81fd9d91]
6837958 3: (Server::handle_client_open(boost::intrusive_ptr<MDRequestImpl>&)+0x1768) [0x55eb0ab92978]
6837959 4: (Server::handle_client_openc(boost::intrusive_ptr<MDRequestImpl>&)+0x61b) [0x55eb0ab9302b]
6837960 5: (Server::dispatch_client_request(boost::intrusive_ptr<MDRequestImpl>&)+0xd24) [0x55eb0abbaaa4]
6837961 6: (MDCache::dispatch_request(boost::intrusive_ptr<MDRequestImpl>&)+0x2e) [0x55eb0ac4af1e]
6837962 7: (MDSContext::complete(int)+0x4f) [0x55eb0addb01f]
6837963 8: (MDSCacheObject::finish_waiting(unsigned long, int)+0xbe) [0x55eb0adf618e]
6837964 9: (Locker::eval_gather(SimpleLock*, bool, bool*, std::vector<MDSContext*, std::allocator<MDSContext*> >*)+0x1259) [0x55eb0acce9d9]
6837965 10: (Locker::handle_file_lock(ScatterLock*, boost::intrusive_ptr<MLock const> const&)+0xe85) [0x55eb0ace01f5]
6837966 11: (Locker::handle_lock(boost::intrusive_ptr<MLock const> const&)+0x1a3) [0x55eb0ace0b23]
6837967 12: (Locker::dispatch(boost::intrusive_ptr<Message const> const&)+0xbf) [0x55eb0ace0d4f]
6837968 13: (MDSRank::handle_message(boost::intrusive_ptr<Message const> const&)+0x6e4) [0x55eb0ab3cf54]
6837969 14: (MDSRank::_dispatch(boost::intrusive_ptr<Message const> const&, bool)+0x7a3) [0x55eb0ab3f503]
6837970 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr<Message const> const&)+0x43) [0x55eb0ab3fa23]
6837971 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr<Message> const&)+0x118) [0x55eb0ab2dc28]
6837972 17: (DispatchQueue::entry()+0x11ba) [0x7f2d821e9f0a]
6837973 18: (DispatchQueue::DispatchThread::entry()+0xd) [0x7f2d8228786d]
6837974 19: (()+0x7ea5) [0x7f2d80ea9ea5]
6837975 20: (clone()+0x6d) [0x7f2d7fb569fd]
6837976
6837977 0> 2023-01-01T09:47:23.651+0800 7f2d7a4e0700 -1 *** Caught signal (Aborted) **
6837978 in thread 7f2d7a4e0700 thread_name:ms_dispatch
6837979
6837980 ceph version 15.2.16 (d46a73d6d0a67a79558054a3a5a72cb561724974) octopus (stable)
6837981 1: (()+0xf630) [0x7f2d80eb1630]
6837982 2: (gsignal()+0x37) [0x7f2d7fa8e3d7]
6837983 3: (abort()+0x148) [0x7f2d7fa8fac8]
6837984 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x19b) [0x7f2d81fd9c18]
6837985 5: (()+0x269d91) [0x7f2d81fd9d91]
6837986 6: (Server::handle_client_open(boost::intrusive_ptr<MDRequestImpl>&)+0x1768) [0x55eb0ab92978]
6837987 7: (Server::handle_client_openc(boost::intrusive_ptr<MDRequestImpl>&)+0x61b) [0x55eb0ab9302b]
6837988 8: (Server::dispatch_client_request(boost::intrusive_ptr<MDRequestImpl>&)+0xd24) [0x55eb0abbaaa4]
6837989 9: (MDCache::dispatch_request(boost::intrusive_ptr<MDRequestImpl>&)+0x2e) [0x55eb0ac4af1e]
6837990 10: (MDSContext::complete(int)+0x4f) [0x55eb0addb01f]
6837991 11: (MDSCacheObject::finish_waiting(unsigned long, int)+0xbe) [0x55eb0adf618e]
6837992 12: (Locker::eval_gather(SimpleLock*, bool, bool*, std::vector<MDSContext*, std::allocator<MDSContext*> >*)+0x1259) [0x55eb0acce9d9]
6837993 13: (Locker::handle_file_lock(ScatterLock*, boost::intrusive_ptr<MLock const> const&)+0xe85) [0x55eb0ace01f5]
6837994 14: (Locker::handle_lock(boost::intrusive_ptr<MLock const> const&)+0x1a3) [0x55eb0ace0b23]
6837995 15: (Locker::dispatch(boost::intrusive_ptr<Message const> const&)+0xbf) [0x55eb0ace0d4f]
6837996 16: (MDSRank::handle_message(boost::intrusive_ptr<Message const> const&)+0x6e4) [0x55eb0ab3cf54]
6837997 17: (MDSRank::_dispatch(boost::intrusive_ptr<Message const> const&, bool)+0x7a3) [0x55eb0ab3f503]
6837998 18: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr<Message const> const&)+0x43) [0x55eb0ab3fa23]
6837999 19: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr<Message> const&)+0x118) [0x55eb0ab2dc28]
6838000 20: (DispatchQueue::entry()+0x11ba) [0x7f2d821e9f0a]
6838001 21: (DispatchQueue::DispatchThread::entry()+0xd) [0x7f2d8228786d]
6838002 22: (()+0x7ea5) [0x7f2d80ea9ea5]
6838003 23: (clone()+0x6d) [0x7f2d7fb569fd]
6838004 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
6838005
6838006 --- logging levels ---
6838007 0/ 5 none
6838008 0/ 1 lockdep
6838009 0/ 1 context
6838010 1/ 1 crush
6838011 20/20 mds
6838012 1/ 5 mds_balancer
6838013 1/ 5 mds_locker
6838014 1/ 5 mds_log
6838015 1/ 5 mds_log_expire
6838016 1/ 5 mds_migrator
6838017 0/ 1 buffer
6838018 0/ 1 timer
6838019 0/ 1 filer
6838020 0/ 1 striper
6838021 0/ 1 objecter
6838022 0/ 5 rados
6838023 0/ 5 rbd
6838024 0/ 5 rbd_mirror
6838025 0/ 5 rbd_replay
6838026 0/ 5 rbd_rwl
6838027 0/ 5 journaler
6838028 0/ 5 objectcacher
6838029 0/ 5 immutable_obj_cache
6838030 0/ 5 client
6838031 1/ 5 osd
6838032 0/ 5 optracker
6838033 0/ 5 objclass
6838034 1/ 3 filestore
6838035 1/ 3 journal
6838036 0/ 0 ms
6838037 1/ 5 mon
6838038 0/10 monc
6838039 1/ 5 paxos
6838040 0/ 5 tp
6838041 1/ 5 auth
6838042 1/ 5 crypto
6838043 1/ 1 finisher
6838044 1/ 1 reserver
6838045 1/ 5 heartbeatmap
6838046 1/ 5 perfcounter
6838047 1/ 5 rgw
6838048 1/ 5 rgw_sync
6838049 1/10 civetweb
6838050 1/ 5 javaclient
6838051 1/ 5 asok
6838052 1/ 1 throttle
6838053 0/ 0 refs
6838054 1/ 5 compressor
6838055 1/ 5 bluestore
6838056 1/ 5 bluefs
6838057 1/ 3 bdev
6838058 1/ 5 kstore
6838059 4/ 5 rocksdb
6838060 4/ 5 leveldb
6838061 4/ 5 memdb
6838062 1/ 5 fuse
6838063 2/ 5 mgr
6838064 1/ 5 mgrc
6838065 1/ 5 dpdk
6838066 1/ 5 eventtrace
6838067 1/ 5 prioritycache
6838068 0/ 5 test
6838069 -2/-2 (syslog threshold)
6838070 -1/-1 (stderr threshold)
6838071 --- pthread ID / name mapping for recent threads ---
6838072 7f2d72cd1700 / md_submit
6838073 7f2d73cd3700 / MR_Finisher
6838074 7f2d74cd5700 / PQ_Finisher
6838075 7f2d7a4e0700 / ms_dispatch
6838076 max_recent 10000
6838077 max_new 1000
6838078 log_file /var/log/ceph/ceph-mds.gdqc_SG11_57_211.log
6838079 --- end dump of recent events ---
    (1-1/1)