Project

General

Profile

Bug #10834

SAMBA VFS module: Timestamps revert back to 01-01-1970

Added by Dennis Kramer over 4 years ago. Updated 6 months ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
-
Target version:
Start date:
02/11/2015
Due date:
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Labels (FS):
Samba/CIFS
Pull request ID:

Description

I've used the ceph_vfs.c upstream module from samba v4.1.16 for our cephfs cluster, but i've noticed a bug where timestamps aren't being updated correctly.
Modified files are being reset to the beginning of Unix time.

It looks like this bug only manifest itself in applications like MS Office where extra metadata is added to files. If I for example modify a text file in notepad the timestamps are working fine, but when I modify a .docx (or .xls for that matter) in MS Office, the timestamp is getting a reset to 1-1-1970.

I've compiled samba from samba-latest source (4.1.16) on Ubuntu 14.04LTS
My CEPH cluster is running version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578) on Ubuntu 14.04LTS

patch View (962 Bytes) Zheng Yan, 02/13/2015 12:20 PM

patch-4.1.6 (979 Bytes) Zheng Yan, 02/13/2015 01:50 PM

patch-fake-birth (2.23 KB) Zheng Yan, 02/14/2015 02:39 AM

0001-vfs_ceph-fix-ntimes_fn-callback.patch View (1.49 KB) Zheng Yan, 02/25/2015 03:51 AM

0002-vfs_ceph-remove-cephwrap_init_stat_ex_from_stat.patch View (2.63 KB) Zheng Yan, 02/25/2015 03:51 AM

0000-pre.patch View (721 Bytes) Zheng Yan, 02/25/2015 11:07 AM

History

#1 Updated by Zheng Yan over 4 years ago

  • File patch added

could you try the attached patch

#2 Updated by Zheng Yan over 4 years ago

  • File deleted (patch)

#3 Updated by Zheng Yan over 4 years ago

#4 Updated by Dennis Kramer over 4 years ago

It won't patch on samba 4.1.16 (latest) source. Did you grab a different version?

#5 Updated by Zheng Yan over 4 years ago

it's for samba 4.3. please try the new one

#6 Updated by Dennis Kramer over 4 years ago

Tried it. Patched successful. But not entirely fixed though, it seems that the timestamp "created timestamp" also changes unintentionally (tested on ms office .docx files). Plain textfiles (edited with notepad) is still working as intended (e.g. only modified/access time changes, not the creation time).

#7 Updated by Zheng Yan over 4 years ago

ceph inode does not store birth time. ceph vfs module assigns 'modified time' to 'birth time'

#8 Updated by Greg Farnum over 4 years ago

I wonder how other Linux filesystems keep track of creation times. Are they just storing them in an xattr or something?

Reporting an mtime as a creation time is fairly misleading, I think I'd rather we just set it to the Unix epoch or something to make clear that it's not really correct...which is apparently what we're actually doing if it reverts to 1970.

#9 Updated by Zheng Yan over 4 years ago

most linux filesystems (extN,xfs,btrfs) do not have birth time. When dealing with linux local filesystem, samba fakes a birth time. please try the new patch (together with previous patch)

#10 Updated by Dennis Kramer over 4 years ago

It doesn't build after patching:

[3740/3767] Linking default/source3/modules/libvfs-ceph.so
default/source3/modules/vfs_ceph_80.o: In function `cephwrap_stat':
vfs_ceph.c:(.text+0x1f91): undefined reference to `lp_fake_directory_create_times'
default/source3/modules/vfs_ceph_80.o: In function `cephwrap_fstat':
vfs_ceph.c:(.text+0x22b2): undefined reference to `lp_fake_directory_create_times'
default/source3/modules/vfs_ceph_80.o: In function `cephwrap_lstat':
vfs_ceph.c:(.text+0x24fd): undefined reference to `lp_fake_directory_create_times'
collect2: error: ld returned 1 exit status
Waf: Leaving directory `/usr/src/samba-4.1.16/bin'
Build failed:  -> task failed (err #1): 
        {task: cc_link vfs_ceph_80.o -> libvfs-ceph.so}
make: *** [all] Error 1

#11 Updated by Dennis Kramer over 4 years ago

Hi Zheng, I was wondering if there is any progress with the above?

#12 Updated by Greg Farnum over 4 years ago

He's on vacation for Chinese New Year, and the rest of us are traveling. This'll probably require some work to get it going properly; have patience! :)

#13 Updated by Dennis Kramer over 4 years ago

I will :-)
Thanks for the reply Greg!

#14 Updated by Zheng Yan over 4 years ago

try replacing "lp_fake_directory_create_times" in the patch with "lp_fake_dir_create_times"

#15 Updated by Dennis Kramer over 4 years ago

Replaced it and it builds, but the problem still exists.
The created time == modified time after editing.

#16 Updated by Zheng Yan over 4 years ago

please check how samba behave when using local file system as backbend. I think this patch make ceph backend behave the same as local filesystem

#17 Updated by Dennis Kramer over 4 years ago

When I use the local filesystem or even cephfs with the kernel module (instead of ceph vfs) everything is working fine. Timestamps are shown correctly, for both created time and modified time. This patch didn't work for me when using the ceph vfs samba module, specifically on MS Office files (.docx).

#18 Updated by Zheng Yan over 4 years ago

  • File 0002-vfs_ceph-remove-cephwrap_init_stat_ex_from_stat.patch added

please try the new patches

#19 Updated by Zheng Yan over 4 years ago

  • File deleted (0002-vfs_ceph-remove-cephwrap_init_stat_ex_from_stat.patch)

#21 Updated by Dennis Kramer over 4 years ago

It won't patch. Hunk #1 failures.
Is it for SAMBA 4.1.16? I see a couple of discrepancies in the patches.

#22 Updated by Zheng Yan over 4 years ago

they are for 4.3. please apply the prerequisite patch first.

#23 Updated by Dennis Kramer over 4 years ago

Are the previous patches also needed?

#24 Updated by Dennis Kramer over 4 years ago

Awesome Zheng! It seems to be working correctly now.
I've only used the patches 0000-, 0001- and 0002-

Thank you.

#25 Updated by Dennis Kramer over 4 years ago

After some more testing, I've noticed it still doesn't work perfectly.
My setup is as follows for test purposes:
I have two samba servers:
- samba1 has a cephfs kernel mount, which is exported in SAMBA (e.g. /mnt/cephfs/test)
- samba2 has the ceph_vfs module from samba, which is configure in SAMBA to point to the same share in cephfs (e.g. /test). This server has implemented above patches.

Both samba servers are being accessed from a Windows client.

Scenario 1:
1. samba1 is being accessed by the client and creates a .docx file. Timestamps are correctly shown.
2. client modifies the file (after 1 minute). Timestamps are correctly shown: Creation time is untouched, and the modified and accessed time is correctly adjusted.

Scenario 2:
1. samba2 is being accessed by the client and creates a .docx file. Timestamps are correctly shown.
2. client modifies the file (after 1 minute). Timestamps are correctly shown: Creation time is untouched, and the modified and accessed time is correctly adjusted.

Scenario 3:
1. samba2 is being accessed by the client and creates a .docx file. Timestamps are correctly shown.
2. client modifies the file (after 1 minute). Timestamps are correctly shown: Creation time is untouched, and the modified and accessed time is correctly adjusted.
3. client modifies the same file again (after 1 minute), but this time the client uses the server samba1 instead of samba2. Timestamps are now incorrectly shown: Creation time has the same timestamp as the modified time. This is shown on the file properties on server samba2, however when I look at the same file properties on server samba1 the timestamps are correct. Creation time seems untouched, and only the modified and accessed time is correctly adjusted. So both servers are showing different creation times on the same file in the situation when the file is modified on samba1 and read on samba2.

#26 Updated by Dennis Kramer over 4 years ago

Zheng, I've compiled your patches against Samba v4.2.0rc5 and it seems to work now for scenario 3 aswell. So far so good :-)

#27 Updated by Dennis Kramer over 4 years ago

Sorry, I was wrong.
Even with Samba v4.2.0rc5 the problem exists for "scenario 3" in my comment above.

#28 Updated by Dennis Kramer over 4 years ago

Okay, I've noticed that when I restart the samba process the timestamps are shown correctly as expected, however when I edit the file again on samba1 (when using the same test-setup as described above) it shows the incorrect timestamps again on samba2.

#29 Updated by Zheng Yan over 4 years ago

do both of your samba servers have "store dos attributes" option enabled.

#30 Updated by Dennis Kramer over 4 years ago

yes they do.

#31 Updated by Dennis Kramer over 4 years ago

My smb.conf is as follows:

[global]
workgroup = DOMAIN
realm = FQDN
netbios name = samba2
server string = Ubuntu - samba2
kerberos method = secrets and keytab
client signing = yes
client use spnego = yes
security = ADS
os level = 0
lm announce = no
preferred master = no
local master = no
domain master = no
encrypt passwords = true
log level = 0 vfs:0
syslog = 2
log file = /var/log/samba/samba.log
max log size = 10000000
bind interfaces only = yes
interfaces = 127.0.0.1 172.17.135.111
client max protocol = SMB3
server max protocol = SMB3
acl allow execute always = True
store dos attributes = yes
dfree cache time = 60
dfree command = /usr/bin/dfree

  1. Tuneables
    socket options = TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE
    strict sync = no
    sync always = no
    dead time = 15
    large readwrite = yes
    strict locking = no
    dns proxy= no
    max xmit = 1048576
    read raw = yes
    write raw = yes
    use sendfile = no
    oplocks = 0
    level2 oplocks = 0
    getwd cache = yes
  2. TEST TUNABLES
    strict allocate = Yes
    aio read size = 16384
    aio write size = 16384
    smb2 leases = yes
  1. ADS IDMAPPER <-> UID
    winbind separator = +
    idmap backend = tdb
    idmap uid = 50000 - 99999
    idmap gid = 50000 - 99999
    idmap config DOMAIN : backend = ad
    idmap config DOMAIN : schema_mode = rfc2307
    idmap config DOMAIN : readonly = yes
    idmap config DOMAIN : range = 50000 - 99999
    idmap cache time = 120
    idmap negative cache time = 20
    winbind nss info = rfc2307
    winbind expand groups = 2
    winbind nested groups = yes
    winbind use default domain = no
    winbind enum users = yes
    winbind enum groups = yes
    winbind refresh tickets = yes
    winbind offline logon = false
    obey pam restrictions = yes
    client ntlmv2 auth = yes
  1. DISABLE CUPS
    load printers = no
    printing = bsd
    printcap name = /dev/null
    disable spoolss = yes

#================ Share Definitions ===================
[TEMPUS]
comment = TEMPUS SHARE
browseable = yes
read only = no
nt acl support = yes
create mask = 0664
directory mask = 2775
valid users = @BOFH
force group = BOFH
vfs objects = ceph recycle
path = /FIG
recycle:repository = .recycle
recycle:keeptree = yes
recycle:versions = yes
recycle:maxsize = 20971520
recylce:touch = yes
recylce:exclude = .tmp|.temp|*.o|*.obj
recycle:directory_mode = 2775

#32 Updated by Zheng Yan over 4 years ago

sound like a libcephfs bug. could you please try the newest development version of ceph (just need to update ceph on machine that run samba server, no need to update ceph on machine run osd/mds/mon)

#33 Updated by Greg Farnum about 3 years ago

  • Status changed from New to Closed

Closing in favor of #16679, since this is really about birthtime and we're adding a real one.

#34 Updated by Patrick Donnelly 6 months ago

  • Category deleted (43)
  • Labels (FS) Samba/CIFS added

Also available in: Atom PDF