Project

General

Profile

Feature #601

mds: order directory commits after rename

Added by Sage Weil about 12 years ago. Updated over 6 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Correctness/Safety
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Reviewed:
Affected Versions:
Component(FS):
Labels (FS):
Pull request ID:

Description

When we rename something between directories, we should try to commit the target directory before the source directory. That way, if all goes to hell, we have at worst 2 links to the file instead of 0.

This probably means adding some xlist<CDir*> commit_after list to the CDirs and make commit wait until the others have committed first.

History

#1 Updated by Sage Weil about 12 years ago

  • Parent task changed from #547 to #86

#2 Updated by Sage Weil about 12 years ago

  • Tracker changed from Tasks to Feature

#3 Updated by Sage Weil about 12 years ago

  • Source changed from 0 to 3

#4 Updated by Sage Weil about 12 years ago

  • Tracker changed from Tasks to Feature
  • Target version changed from 12 to v0.25
  • Parent task deleted (#86)

#5 Updated by Sage Weil about 12 years ago

  • translation missing: en.field_position set to 5

#6 Updated by Sage Weil about 12 years ago

  • translation missing: en.field_position deleted (5)
  • translation missing: en.field_position set to 2

#7 Updated by Sage Weil about 12 years ago

  • Assignee set to Sage Weil

#8 Updated by Sage Weil about 12 years ago

What if another file is renamed in the other direction? Do a partial commit first? Tricky.

#9 Updated by Sage Weil about 12 years ago

  • translation missing: en.field_position deleted (3)
  • translation missing: en.field_position set to 1
  • translation missing: en.field_position changed from 1 to 525

#10 Updated by Sage Weil about 12 years ago

  • Target version changed from v0.25 to v0.26

#11 Updated by Greg Farnum almost 12 years ago

  • Assignee changed from Sage Weil to Greg Farnum

I'm going to take a look at this while waiting for data collection to run as I work on #698.

#12 Updated by Sage Weil almost 12 years ago

I think this is going to be much ahrder than I originally imagined. For example:

- mv /a/foo /b/foo
- mv /b/bar /a/bar

Which do we write first? We could:
- force dir /b flush in between the renames?
- flush them later, but in multiple stages.... e.g. flush /b/foo, /a/*, /b/bar removal

#13 Updated by Greg Farnum almost 12 years ago

I haven't gotten too far into it yet, but my current line of attack is to see how feasible it is to place "holds" on dentries (linked to their parent CDir) which requires that that dentry not be written to disk until another dentry has been written. Then, when a CDir commit occurs, if any dentries have holds, request the blocking dentry get written to disk. Then attempt to write the whole CDir, but if you can't (ie, you have a hold) just commit the blocking dentry. So:
1) mv a/foo -> b/foo
2) a/foo (null dentry, right?) gets hold linked to b/foo
3) mv b/bar -> a/bar
4) b/bar gets hold linked to a/bar
5) commit a comes in
6) a asks b to commit "foo"
7) b commits just "foo" because there is a hold on "bar"
8) a commits
9) commit b comes in
10) b asks a to commit "bar"
11) a commits full dir
12) b commits

There are some obvious options for optimization here, but I think that if this is feasible to get working then any optimizations will be simple add-ons. I haven't dug into the commit code much more than I have previously but I think this should work?

#14 Updated by Greg Farnum almost 12 years ago

Err, I guess actually that should read:
8) a commits
9) hold on b/bar gets removed
10) commit b comes in
11) b commits

Or alternatively, depending on data structures in place:
8) a commits
9) commit b comes in
10) b asks a to commit "bar"
11) a says "already committed that dentry at that version!"
12) b commits

#15 Updated by Sage Weil almost 12 years ago

  • translation missing: en.field_position deleted (526)
  • translation missing: en.field_position set to 1
  • translation missing: en.field_position changed from 1 to 527

#16 Updated by Sage Weil almost 12 years ago

  • translation missing: en.field_position deleted (531)
  • translation missing: en.field_position set to 539

#17 Updated by Sage Weil almost 12 years ago

  • Target version changed from v0.26 to 12

#18 Updated by Sage Weil almost 12 years ago

  • translation missing: en.field_position deleted (585)
  • translation missing: en.field_position set to 602

#19 Updated by Sage Weil over 11 years ago

  • Target version changed from 12 to 19

#20 Updated by Sage Weil over 11 years ago

  • Target version deleted (19)

#21 Updated by Sage Weil over 10 years ago

  • Project changed from Ceph to CephFS
  • Category deleted (1)

#22 Updated by Sage Weil over 10 years ago

  • Assignee deleted (Greg Farnum)

#23 Updated by Greg Farnum over 6 years ago

  • Category set to Correctness/Safety

Also available in: Atom PDF