6837902 -51> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 MDSContext::complete: 18C_MDS_RetryRequest 6837903 -50> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.server dispatch_client_request client_request(client.408135:34341420 create #0x30004bf4c4f/hydra_nameserver 2022-12-31T22:58:02.294645+0800 RETRY=74 caller_uid=0, caller_gid=0{}) v2 6837904 -49> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.server open w/ O_CREAT on #0x30004bf4c4f/hydra_nameserver 6837905 -48> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.server rdlock_path_xlock_dentry request(client.408135:34341420 nref=2 cr=0x55eb0d1cc480) #0x30004bf4c4f/hydra_nameserver 6837906 -47> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.cache traverse: opening base ino 0x30004bf4c4f snap head 6837907 -46> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker try_rdlock_snap_layout request(client.408135:34341420 nref=3 cr=0x55eb0d1cc480) [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75, 1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75,11=75,12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock) (iversion lock) caps={408135=pAsLsXs/p@0},l=408135 | lock=2 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300] 6837908 -45> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 12 mds.2.cache traverse: path seg depth 0 'hydra_nameserver' snapid head 6837909 -44> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 20 mds.2.cache.dir(0x30004bf4c4f) lookup (head, 'hydra_nameserver') 6837910 -43> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 20 mds.2.cache.dir(0x30004bf4c4f) hit -> (hydra_nameserver,head) 6837911 -42> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker acquire_locks request(client.408135:34341420 nref=3 cr=0x55eb0d1cc480) 6837912 -41> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 20 mds.2.locker must wrlock (ifile lock) [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75,11=75, 12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock) (iversion lock) caps={408135=pAsLsXs/p@0},l=408135 | lock=2 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300] 6837913 -40> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 20 mds.2.locker must wrlock (inest mix) [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75,11=75, 12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock) (iversion lock) caps={408135=pAsLsXs/p@0},l=408135 | lock=2 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300] 6837914 -39> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 20 mds.2.locker must rdlock (iauth sync) [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75,11=75, 12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock) (iversion lock) caps={408135=pAsLsXs/p@0},l=408135 | lock=2 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300] 6837915 -38> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 20 mds.2.locker must xlock (dn sync) [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dversion lock) pv=0 v=171188706 ap=1 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=0 authpin=1 0x55eb31037600] 6837916 -37> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 20 mds.2.locker must wrlock (dversion lock) [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dversion lock) pv=0 v=171188706 ap=1 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=0 authpin=1 0x55eb31037600] 6837917 -36> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker must authpin [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75,11=75,12=75,13=75, 14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock) (iversion lock) caps={408135=pAsLsXs/p@0},l=408135 | lock=2 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300] 6837918 -35> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker must authpin [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dversion lock) pv=0 v=171188706 ap=1 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=0 authpin=1 0x55eb31037600] 6837919 -34> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker already auth_pinned [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75,11=75,12=75, 13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock) (iversion lock) caps={408135=pAsLsXs/p@0},l=408135 | lock=2 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300] 6837920 -33> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker already auth_pinned [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dversion lock) pv=0 v=171188706 ap=1 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=0 authpin=1 0x55eb31037600] 6837921 -32> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker wrlock_start (ifile lock) on [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75,11=75, 12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock) (iversion lock) caps={408135=pAsLsXs/p@0},l=408135 | lock=2 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300] 6837922 -31> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker got wrlock on (ifile lock w=1) [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75, 11=75,12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock w=1) (iversion lock) caps={408135=pAsLsXs/p@ 0},l=408135 | lock=3 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300] 6837923 -30> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker wrlock_start (inest mix) on [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75,11=75, 12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix) (ipolicy sync r=1) (ifile lock w=1) (iversion lock) caps={408135=pAsLsXs/p@0}, l=408135 | lock=3 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300] 6837924 -29> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker got wrlock on (inest mix w=1) [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75, 11=75,12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix w=1) (ipolicy sync r=1) (ifile lock w=1) (iversion lock) caps={408135=pAsLsXs/ p@0},l=408135 | lock=4 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300] 6837925 -28> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.locker rdlock_start on (iauth sync) on [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75, 11=75,12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (isnap sync r=1) (inest mix w=1) (ipolicy sync r=1) (ifile lock w=1) (iversion lock) caps={408135=pAsLsXs/ p@0},l=408135 | lock=4 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300] 6837926 -27> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker got rdlock on (iauth sync r=1) [inode 0x30004bf4c4f [...2,head] /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75,10=75, 11=75,12=75,13=75,14=75,15=75} v54278 ap=1 f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 183=182+1) (iauth sync r=1) (isnap sync r=1) (inest mix w=1) (ipolicy sync r=1) (ifile lock w=1) (iversion lock) caps={408135=pAsLsXs/p@0},l=408135 | lock=5 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 0x55eb0f09c300] 6837927 -26> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.locker xlock_start on (dn sync) on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dversion lock) pv=0 v=171188706 ap=1 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=0 authpin=1 0x55eb31037600] 6837928 -25> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.locker simple_lock on (dn sync) on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dversion lock) pv=0 v=171188706 ap=1 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=0 authpin=1 0x55eb31037600] 6837929 -24> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.locker simple_xlock on (dn lock) on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn lock) (dversion lock) pv=0 v=171188706 ap=1 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=0 authpin=1 0x55eb31037600] 6837930 -23> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.cache.den(0x30004bf4c4f hydra_nameserver) auth_pin by 0x55eb310376c0 on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn lock) (dversion lock) pv=0 v=171188706 ap=2 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=0 authpin=1 0x55eb31037600] now 2 6837931 -22> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 15 mds.2.cache.dir(0x30004bf4c4f) adjust_nested_auth_pins 1 on [dir 0x30004bf4c4f /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ [2,head] auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75,9=75, 10=75,11=75,12=75,13=75,14=75,15=75} v=171188707 cv=0/0 ap=0+2 state=1610612737|complete f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 182=182+0) hs=361+0,ss=0+0 dirty=62 | child=1 replicated=1 dirty=1 waiter=0 authpin=0 0x55eb0f092a00] by 0x55eb310376c0 count now 0/2 6837932 -21> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker got xlock on (dn xlock x=1 by 0x55eb1ba9a400) [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn xlock x=1 by 0x55eb1ba9a400) (dversion lock) pv=0 v=171188706 ap=2 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=1 authpin=1 0x55eb31037600] 6837933 -20> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.locker local_wrlock_start on (dversion lock) on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn xlock x=1 by 0x55eb1ba9a400) (dversion lock) pv=0 v=171188706 ap=2 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=1 authpin=1 0x55eb31037600] 6837934 -19> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker got wrlock on (dversion lock w=1 last_client=408135) [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn xlock x=1 by 0x55eb1ba9a400) (dversion lock w=1 last_client=408135) pv=0 v=171188706 ap=2 ino=(nil) state=1073741824 | ptrwaiter=0 request=1 lock=2 authpin=1 0x55eb31037600] 6837935 -18> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.cache linking in remote in [inode 0x30004be7edb [...2,head] ~mds4/stray1/30004be7edb rep@4.1 v172464532 snaprealm=0x55eb2dc0a500 s=2203120 n(v0 rc2022-12-31T22:44:39.135649+0800 b2203120 1=1+0) (iversion lock) 0x55eb2ce3bc00] 6837936 -17> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.cache.snaprealm(0x30004be7edb seq 1 0x55eb2dc0a500) have_past_parents_open [1,head] 6837937 -16> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.cache path_traverse finish on snapid head 6837938 -15> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker xlock_downgrade on (dn xlock x=1 by 0x55eb1ba9a400) [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn xlock x=1 by 0x55eb1ba9a400) (dversion lock w=1 last_client=408135) pv=0 v=171188706 ap=2 ino=0x30004be7edb state=1073741824 | ptrwaiter=0 request=1 lock=2 authpin=1 0x55eb31037600] 6837939 -14> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker xlock_finish on (dn xlockdone r=1 x=1) [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn xlockdone r=1 x=1) (dversion lock w=1 last_client=408135) pv=0 v=171188706 ap=2 ino=0x30004be7edb state=1073741824 | ptrwaiter=0 request=1 lock=3 authpin=1 0x55eb31037600] 6837940 -13> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker eval_gather (dn xlockdone r=1) on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn xlockdone r=1) (dversion lock w=1 last_client=408135) pv=0 v=171188706 ap=2 ino=0x30004be7edb state=1073741824 | ptrwaiter=0 request=1 lock=2 authpin=1 0x55eb31037600] 6837941 -12> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.locker eval_gather finished gather on (dn xlockdone r=1) on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn xlockdone r=1) (dversion lock w=1 last_client=408135) pv=0 v=171188706 ap=2 ino=0x30004be7edb state=1073741824 | ptrwaiter=0 request=1 lock=2 authpin=1 0x55eb31037600] 6837942 -11> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.cache.den(0x30004bf4c4f hydra_nameserver) auth_unpin by 0x55eb310376c0 on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn sync r=1) (dversion lock w=1 last_client=408135) pv=0 v=171188706 ap=1 ino=0x30004be7edb state=1073741824 | ptrwaiter=0 request=1 lock=2 authpin=1 0x55eb31037600] now 1 6837943 -10> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 15 mds.2.cache.dir(0x30004bf4c4f) adjust_nested_auth_pins -1 on [dir 0x30004bf4c4f /data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/ [2,head] auth{0=75,1=75,3=75,4=75,5=75,6=75,7=75,8=75, 9=75,10=75,11=75,12=75,13=75,14=75,15=75} v=171188707 cv=0/0 ap=0+1 state=1610612737|complete f(v0 m2022-12-31T22:57:56.849645+0800 361=361+0) n(v0 rc2022-12-31T22:57:56.849645+0800 b12196376 182=182+0) hs=361+0,ss=0+0 dirty=62 | child=1 replicated=1 dirty=1 waiter=0 authpin=0 0x55eb0f092a00] by 0x55eb310376c0 count now 0/1 6837944 -9> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker simple_eval (dn sync r=1) on [dentry #0x1/data/9a2/77a/e075fbba8f8f7a0b064bb93625/env/.local/anaconda3/bin/hydra_nameserver [2,head] auth REMOTE(reg) (dn sync r=1) (dversion lock w=1 last_client=408135) pv=0 v=171188706 ap=1 ino=0x30004be7edb state=1073741824 | ptrwaiter=0 request=1 lock=2 authpin=1 0x55eb31037600] 6837945 -8> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker acquire_locks request(client.408135:34341420 nref=2 cr=0x55eb0d1cc480) 6837946 -7> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 20 mds.2.locker must rdlock (isnap sync) [inode 0x30004be7edb [...2,head] ~mds4/stray1/30004be7edb rep@4.1 v172464532 snaprealm=0x55eb2dc0a500 s=2203120 n(v0 rc2022-12-31T22:44:39.135649+0800 b2203120 1=1+ 0) (iversion lock) | remoteparent=1 0x55eb2ce3bc00] 6837947 -6> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.locker rdlock_start on (isnap sync) on [inode 0x30004be7edb [...2,head] ~mds4/stray1/30004be7edb rep@4.1 v172464532 snaprealm=0x55eb2dc0a500 s=2203120 n(v0 rc2022-12-31T22:44:39.135649+0800 b2203120 1=1+0) (iversion lock) | remoteparent=1 0x55eb2ce3bc00] 6837948 -5> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.locker got rdlock on (isnap sync r=1) [inode 0x30004be7edb [...2,head] ~mds4/stray1/30004be7edb rep@4.1 v172464532 snaprealm=0x55eb2dc0a500 s=2203120 n(v0 rc2022-12-31T22:44:39.135649+0800 b2203120 1=1+0) (iversion lock) | lock=1 remoteparent=1 0x55eb2ce3bc00] 6837949 -4> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 7 mds.2.server open on #0x30004bf4c4f/hydra_nameserver 6837950 -3> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.server rdlock_path_pin_ref request(client.408135:34341420 nref=2 cr=0x55eb0d1cc480) #0x30004bf4c4f/hydra_nameserver 6837951 -2> 2023-01-01T09:47:23.647+0800 7f2d7a4e0700 10 mds.2.server open flags = 577, filemode = 2, need_auth = 1 6837952 -1> 2023-01-01T09:47:23.651+0800 7f2d7a4e0700 -1 /root/src/ceph/src/mds/Server.cc: In function 'void Server::handle_client_open(MDRequestRef&)' thread 7f2d7a4e0700 time 2023-01-01T09:47:23.649572+0800 6837953 /root/src/ceph/src/mds/Server.cc: 4164: FAILED ceph_assert(cur->is_auth()) 6837954 6837955 ceph version 15.2.16 (d46a73d6d0a67a79558054a3a5a72cb561724974) octopus (stable) 6837956 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14c) [0x7f2d81fd9bc9] 6837957 2: (()+0x269d91) [0x7f2d81fd9d91] 6837958 3: (Server::handle_client_open(boost::intrusive_ptr&)+0x1768) [0x55eb0ab92978] 6837959 4: (Server::handle_client_openc(boost::intrusive_ptr&)+0x61b) [0x55eb0ab9302b] 6837960 5: (Server::dispatch_client_request(boost::intrusive_ptr&)+0xd24) [0x55eb0abbaaa4] 6837961 6: (MDCache::dispatch_request(boost::intrusive_ptr&)+0x2e) [0x55eb0ac4af1e] 6837962 7: (MDSContext::complete(int)+0x4f) [0x55eb0addb01f] 6837963 8: (MDSCacheObject::finish_waiting(unsigned long, int)+0xbe) [0x55eb0adf618e] 6837964 9: (Locker::eval_gather(SimpleLock*, bool, bool*, std::vector >*)+0x1259) [0x55eb0acce9d9] 6837965 10: (Locker::handle_file_lock(ScatterLock*, boost::intrusive_ptr const&)+0xe85) [0x55eb0ace01f5] 6837966 11: (Locker::handle_lock(boost::intrusive_ptr const&)+0x1a3) [0x55eb0ace0b23] 6837967 12: (Locker::dispatch(boost::intrusive_ptr const&)+0xbf) [0x55eb0ace0d4f] 6837968 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x6e4) [0x55eb0ab3cf54] 6837969 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x7a3) [0x55eb0ab3f503] 6837970 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x43) [0x55eb0ab3fa23] 6837971 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x118) [0x55eb0ab2dc28] 6837972 17: (DispatchQueue::entry()+0x11ba) [0x7f2d821e9f0a] 6837973 18: (DispatchQueue::DispatchThread::entry()+0xd) [0x7f2d8228786d] 6837974 19: (()+0x7ea5) [0x7f2d80ea9ea5] 6837975 20: (clone()+0x6d) [0x7f2d7fb569fd] 6837976 6837977 0> 2023-01-01T09:47:23.651+0800 7f2d7a4e0700 -1 *** Caught signal (Aborted) ** 6837978 in thread 7f2d7a4e0700 thread_name:ms_dispatch 6837979 6837980 ceph version 15.2.16 (d46a73d6d0a67a79558054a3a5a72cb561724974) octopus (stable) 6837981 1: (()+0xf630) [0x7f2d80eb1630] 6837982 2: (gsignal()+0x37) [0x7f2d7fa8e3d7] 6837983 3: (abort()+0x148) [0x7f2d7fa8fac8] 6837984 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x19b) [0x7f2d81fd9c18] 6837985 5: (()+0x269d91) [0x7f2d81fd9d91] 6837986 6: (Server::handle_client_open(boost::intrusive_ptr&)+0x1768) [0x55eb0ab92978] 6837987 7: (Server::handle_client_openc(boost::intrusive_ptr&)+0x61b) [0x55eb0ab9302b] 6837988 8: (Server::dispatch_client_request(boost::intrusive_ptr&)+0xd24) [0x55eb0abbaaa4] 6837989 9: (MDCache::dispatch_request(boost::intrusive_ptr&)+0x2e) [0x55eb0ac4af1e] 6837990 10: (MDSContext::complete(int)+0x4f) [0x55eb0addb01f] 6837991 11: (MDSCacheObject::finish_waiting(unsigned long, int)+0xbe) [0x55eb0adf618e] 6837992 12: (Locker::eval_gather(SimpleLock*, bool, bool*, std::vector >*)+0x1259) [0x55eb0acce9d9] 6837993 13: (Locker::handle_file_lock(ScatterLock*, boost::intrusive_ptr const&)+0xe85) [0x55eb0ace01f5] 6837994 14: (Locker::handle_lock(boost::intrusive_ptr const&)+0x1a3) [0x55eb0ace0b23] 6837995 15: (Locker::dispatch(boost::intrusive_ptr const&)+0xbf) [0x55eb0ace0d4f] 6837996 16: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x6e4) [0x55eb0ab3cf54] 6837997 17: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x7a3) [0x55eb0ab3f503] 6837998 18: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x43) [0x55eb0ab3fa23] 6837999 19: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x118) [0x55eb0ab2dc28] 6838000 20: (DispatchQueue::entry()+0x11ba) [0x7f2d821e9f0a] 6838001 21: (DispatchQueue::DispatchThread::entry()+0xd) [0x7f2d8228786d] 6838002 22: (()+0x7ea5) [0x7f2d80ea9ea5] 6838003 23: (clone()+0x6d) [0x7f2d7fb569fd] 6838004 NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 6838005 6838006 --- logging levels --- 6838007 0/ 5 none 6838008 0/ 1 lockdep 6838009 0/ 1 context 6838010 1/ 1 crush 6838011 20/20 mds 6838012 1/ 5 mds_balancer 6838013 1/ 5 mds_locker 6838014 1/ 5 mds_log 6838015 1/ 5 mds_log_expire 6838016 1/ 5 mds_migrator 6838017 0/ 1 buffer 6838018 0/ 1 timer 6838019 0/ 1 filer 6838020 0/ 1 striper 6838021 0/ 1 objecter 6838022 0/ 5 rados 6838023 0/ 5 rbd 6838024 0/ 5 rbd_mirror 6838025 0/ 5 rbd_replay 6838026 0/ 5 rbd_rwl 6838027 0/ 5 journaler 6838028 0/ 5 objectcacher 6838029 0/ 5 immutable_obj_cache 6838030 0/ 5 client 6838031 1/ 5 osd 6838032 0/ 5 optracker 6838033 0/ 5 objclass 6838034 1/ 3 filestore 6838035 1/ 3 journal 6838036 0/ 0 ms 6838037 1/ 5 mon 6838038 0/10 monc 6838039 1/ 5 paxos 6838040 0/ 5 tp 6838041 1/ 5 auth 6838042 1/ 5 crypto 6838043 1/ 1 finisher 6838044 1/ 1 reserver 6838045 1/ 5 heartbeatmap 6838046 1/ 5 perfcounter 6838047 1/ 5 rgw 6838048 1/ 5 rgw_sync 6838049 1/10 civetweb 6838050 1/ 5 javaclient 6838051 1/ 5 asok 6838052 1/ 1 throttle 6838053 0/ 0 refs 6838054 1/ 5 compressor 6838055 1/ 5 bluestore 6838056 1/ 5 bluefs 6838057 1/ 3 bdev 6838058 1/ 5 kstore 6838059 4/ 5 rocksdb 6838060 4/ 5 leveldb 6838061 4/ 5 memdb 6838062 1/ 5 fuse 6838063 2/ 5 mgr 6838064 1/ 5 mgrc 6838065 1/ 5 dpdk 6838066 1/ 5 eventtrace 6838067 1/ 5 prioritycache 6838068 0/ 5 test 6838069 -2/-2 (syslog threshold) 6838070 -1/-1 (stderr threshold) 6838071 --- pthread ID / name mapping for recent threads --- 6838072 7f2d72cd1700 / md_submit 6838073 7f2d73cd3700 / MR_Finisher 6838074 7f2d74cd5700 / PQ_Finisher 6838075 7f2d7a4e0700 / ms_dispatch 6838076 max_recent 10000 6838077 max_new 1000 6838078 log_file /var/log/ceph/ceph-mds.gdqc_SG11_57_211.log 6838079 --- end dump of recent events ---