Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

casenorm/mixed_create_failure broken on FreeBSD (false positive heretofor) – silent data loss (no link to O_CREATed inode) inode leak #13215

Closed
nabijaczleweli opened this issue Mar 14, 2022 · 1 comment · Fixed by #16138
Labels
Type: Defect Incorrect behavior (e.g. crash, hang)

Comments

@nabijaczleweli
Copy link
Contributor

nabijaczleweli commented Mar 14, 2022

System information

Type Version/Name
Distribution FreeBSD freebsd 13.0-RELEASE FreeBSD 13.0-RELEASE #0 releng/13.0-n244733-ea31abc261f: Fri Apr 9 04:24:09 UTC 2021 [email protected]:/usr/obj/usr/src/amd64.a md64/sys/GENERIC amd64
OpenZFS Version HEAD (1282274 userspace delta)

Describe the problem you're observing

On HEAD, as reference, see http://build.zfsonlinux.org/builders/FreeBSD main amd64 (TEST)/builds/3262/steps/shell_4/logs/log:

20:50:08.70 NOTE: cmd: touch /mnt/testdir/file/ABCDEFGHIJKL ret: 0 out=
20:50:08.70 NOTE: Test rename \"sample_name\" rename
20:50:08.71 /usr/local/share/zfs/zfs-tests/tests/functional/casenorm/mixed_create_failure.ksh[133]: test_ops[125]: log_err: not found [No such file or directory]
20:50:08.71 NOTE: Testing create of dir
20:50:08.71 NOTE: The op : mkdir

When I fixed that, I got the loop also running to completion and

04:52:12.63 NOTE: cmd: touch /mnt/testroot/testdir/file/ABCDEFGHIJKL ret: 0 out=
04:52:12.63 NOTE: Test rename "sample_name" rename
04:52:12.64 SUCCESS: touch /mnt/testroot/testdir/file/tmp_obj
04:52:12.65 touch /mnt/testroot/testdir/file/tmp_obj failed: usage: mv [-f | -i | -n] [-hv] source target
04:52:12.65        mv [-f | -i | -n] [-v] source ... directory

(coming from the empty $save_name – i.e. the touch never failed).

Additional testing outside the ZTS confirms this – this test doesn't fail on FreeBSD, but!!!!!! The file doesn't get created, either!:

% ls -l /testpool/fs/ABCDEFGHIJKL
ls: /testpool/fs/ABCDEFGHIJKL: No such file or directory
% echo dupa > /testpool/fs/ABCDEFGHIJKL
% ls -l /testpool/fs/ABCDEFGHIJKL
ls: /testpool/fs/ABCDEFGHIJKL: No such file or directory

% ktrace touch /testpool/fs/ABCDEFGHIJKL
% kdump | tail -14
 88833 touch    CALL  mprotect(0x203000,0x1000,0x1<PROT_READ>)
 88833 touch    RET   mprotect 0
 88833 touch    CALL  fstatat(AT_FDCWD,0x7fffffffee0e,0x7fffffffe950,0)
 88833 touch    NAMI  "/testpool/fs/ABCDEFGHIJKL"
 88833 touch    RET   fstatat -1 errno 2 No such file or directory
 88833 touch    CALL  openat(AT_FDCWD,0x7fffffffee0e,0x201<O_WRONLY|O_CREAT>,0666<S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH>)
 88833 touch    NAMI  "/testpool/fs/ABCDEFGHIJKL"
 88833 touch    RET   openat 3
 88833 touch    CALL  fstat(0x3,0x7fffffffe950)
 88833 touch    STRU  struct stat {dev=11961901315426196474, ino=14881, mode=0100644, nlink=1, uid=1001, gid=0, rdev=18446744073709551615, atime=1647269330.748562000, mtime=1647269330.748562000, ctime=1647269330.748562000, birthtime=1647269330.748562000, size=0, blksize=131072, blocks=1, flags=0x800 }
 88833 touch    RET   fstat 0
 88833 touch    CALL  close(0x3)
 88833 touch    RET   close 0
 88833 touch    CALL  exit(0)
% ls -l /testpool/fs/ABCDEFGHIJKL
ls: /testpool/fs/ABCDEFGHIJKL: No such file or directory

Whichever error prevented the creation of the link (since inodes do get allocated) is not bubbled down to userspace.

For comparison, under Linux (where this works), the strace looks like this:

close(3)                                = 0
openat(AT_FDCWD, "/testpool/fs/abCDEFGHijKL", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = -1 ENOSPC (No space left on device)
utimensat(AT_FDCWD, "/testpool/fs/abCDEFGHijKL", NULL, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
@nabijaczleweli nabijaczleweli added the Type: Defect Incorrect behavior (e.g. crash, hang) label Mar 14, 2022
@nabijaczleweli nabijaczleweli changed the title casenorm/mixed_create_failure broken on FreeBSD (false positive heretofor) – data loss casenorm/mixed_create_failure broken on FreeBSD (false positive heretofor) – silent data loss (no link to creat()ed inode) Mar 14, 2022
@nabijaczleweli nabijaczleweli changed the title casenorm/mixed_create_failure broken on FreeBSD (false positive heretofor) – silent data loss (no link to creat()ed inode) casenorm/mixed_create_failure broken on FreeBSD (false positive heretofor) – silent data loss (no link to O_CREATed inode) Mar 14, 2022
@nabijaczleweli
Copy link
Contributor Author

nabijaczleweli commented Mar 14, 2022

Haha, it also leaks the inode:

$ df -i /testpool/fs/
Filesystem  1K-blocks Used    Avail Capacity iused    ifree %iused  Mounted on
testpool/fs  40140633 2447 40138186     0%   32442 80276373    0%   /testpool/fs
$ while :; do echo > /testpool/fs/ABCDEFGHIJKL; done
^C
$ cat /testpool/fs/ABCDEFGHIJKL
cat: /testpool/fs/ABCDEFGHIJKL: No such file or directory
$ df -i /testpool/fs/
Filesystem  1K-blocks Used    Avail Capacity iused    ifree %iused  Mounted on
testpool/fs  40140632 3605 40137027     0%   50116 80274054    0%   /testpool/fs

And a scrub hasn't fixed it :)

@nabijaczleweli nabijaczleweli changed the title casenorm/mixed_create_failure broken on FreeBSD (false positive heretofor) – silent data loss (no link to O_CREATed inode) casenorm/mixed_create_failure broken on FreeBSD (false positive heretofor) – silent data loss (no link to O_CREATed inode) inode leak Mar 14, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 23, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 23, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 23, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 24, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 24, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 25, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 25, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 25, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 25, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 26, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 26, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 27, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 27, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 27, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 27, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 27, 2022
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue Mar 27, 2022
behlendorf pushed a commit that referenced this issue Apr 2, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Issue #13215
Closes #13259
behlendorf pushed a commit that referenced this issue Apr 2, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Issue: #13215
Closes #13259
behlendorf pushed a commit that referenced this issue Apr 2, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Closes #13259
BrainSlayer pushed a commit to BrainSlayer/zfs that referenced this issue Apr 11, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Issue openzfs#13215
Closes openzfs#13259
BrainSlayer pushed a commit to BrainSlayer/zfs that referenced this issue Apr 11, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Issue: openzfs#13215
Closes openzfs#13259
andrewc12 pushed a commit to andrewc12/openzfs that referenced this issue Sep 23, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Issue openzfs#13215
Closes openzfs#13259
andrewc12 pushed a commit to andrewc12/openzfs that referenced this issue Sep 23, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Issue: openzfs#13215
Closes openzfs#13259
andrewc12 pushed a commit to andrewc12/openzfs that referenced this issue Sep 23, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Closes openzfs#13259
andrewc12 pushed a commit to andrewc12/openzfs that referenced this issue Sep 23, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Issue openzfs#13215
Closes openzfs#13259
andrewc12 pushed a commit to andrewc12/openzfs that referenced this issue Sep 23, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Issue: openzfs#13215
Closes openzfs#13259
andrewc12 pushed a commit to andrewc12/openzfs that referenced this issue Sep 23, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Closes openzfs#13259
andrewc12 pushed a commit to andrewc12/openzfs that referenced this issue Sep 23, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Issue openzfs#13215
Closes openzfs#13259
andrewc12 pushed a commit to andrewc12/openzfs that referenced this issue Sep 23, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Issue: openzfs#13215
Closes openzfs#13259
andrewc12 pushed a commit to andrewc12/openzfs that referenced this issue Sep 23, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Closes openzfs#13259
andrewc12 pushed a commit to andrewc12/openzfs that referenced this issue Sep 23, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Issue openzfs#13215
Closes openzfs#13259
andrewc12 pushed a commit to andrewc12/openzfs that referenced this issue Sep 23, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Issue: openzfs#13215
Closes openzfs#13259
andrewc12 pushed a commit to andrewc12/openzfs that referenced this issue Sep 23, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Closes openzfs#13259
andrewc12 pushed a commit to andrewc12/openzfs that referenced this issue Sep 23, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Issue openzfs#13215
Closes openzfs#13259
andrewc12 pushed a commit to andrewc12/openzfs that referenced this issue Sep 23, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Issue: openzfs#13215
Closes openzfs#13259
andrewc12 pushed a commit to andrewc12/openzfs that referenced this issue Sep 23, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Closes openzfs#13259
andrewc12 pushed a commit to andrewc12/openzfs that referenced this issue Sep 23, 2022
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: John Kennedy <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Ahelenia Ziemiańska <[email protected]>
Closes openzfs#13259
amotin added a commit to amotin/zfs that referenced this issue Apr 26, 2024
Originally Solaris didn't expect errors there, but they may happen
if we fail to add entry into ZAP.  Linux fixed it in openzfs#7421, but it
was never fully ported to FreeBSD.

Signed-off-by:	Alexander Motin <[email protected]>
Sponsored-By:	iXsystems, Inc.
Closes openzfs#13215
amotin added a commit to amotin/zfs that referenced this issue Apr 26, 2024
Originally Solaris didn't expect errors there, but they may happen
if we fail to add entry into ZAP.  Linux fixed it in openzfs#7421, but it
was never fully ported to FreeBSD.

Signed-off-by:	Alexander Motin <[email protected]>
Sponsored-By:	iXsystems, Inc.
Closes openzfs#13215
amotin added a commit to amotin/zfs that referenced this issue Apr 26, 2024
Originally Solaris didn't expect errors there, but they may happen
if we fail to add entry into ZAP.  Linux fixed it in openzfs#7421, but it
was never fully ported to FreeBSD.

Signed-off-by:	Alexander Motin <[email protected]>
Sponsored-By:	iXsystems, Inc.
Closes openzfs#13215
amotin added a commit to amotin/zfs that referenced this issue Apr 27, 2024
Originally Solaris didn't expect errors there, but they may happen
if we fail to add entry into ZAP.  Linux fixed it in openzfs#7421, but it
was never fully ported to FreeBSD.

Signed-off-by:	Alexander Motin <[email protected]>
Sponsored-By:	iXsystems, Inc.
Closes openzfs#13215
amotin added a commit to amotin/zfs that referenced this issue May 23, 2024
Originally Solaris didn't expect errors there, but they may happen
if we fail to add entry into ZAP.  Linux fixed it in openzfs#7421, but it
was never fully ported to FreeBSD.

Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Alexander Motin <[email protected]>
Sponsored-By: iXsystems, Inc.
Closes openzfs#13215
Closes openzfs#16138
ixhamza pushed a commit to truenas/zfs that referenced this issue May 23, 2024
Originally Solaris didn't expect errors there, but they may happen
if we fail to add entry into ZAP.  Linux fixed it in openzfs#7421, but it
was never fully ported to FreeBSD.

Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Alexander Motin <[email protected]>
Sponsored-By: iXsystems, Inc.
Closes openzfs#13215
Closes openzfs#16138
ixhamza pushed a commit to truenas/zfs that referenced this issue May 23, 2024
Originally Solaris didn't expect errors there, but they may happen
if we fail to add entry into ZAP.  Linux fixed it in openzfs#7421, but it
was never fully ported to FreeBSD.

Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Alexander Motin <[email protected]>
Sponsored-By: iXsystems, Inc.
Closes openzfs#13215
Closes openzfs#16138
behlendorf pushed a commit that referenced this issue May 29, 2024
Originally Solaris didn't expect errors there, but they may happen
if we fail to add entry into ZAP.  Linux fixed it in #7421, but it
was never fully ported to FreeBSD.

Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Alexander Motin <[email protected]>
Sponsored-By: iXsystems, Inc.
Closes #13215
Closes #16138
lundman pushed a commit to openzfsonwindows/openzfs that referenced this issue Sep 4, 2024
Originally Solaris didn't expect errors there, but they may happen
if we fail to add entry into ZAP.  Linux fixed it in openzfs#7421, but it
was never fully ported to FreeBSD.

Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Alexander Motin <[email protected]>
Sponsored-By: iXsystems, Inc.
Closes openzfs#13215
Closes openzfs#16138
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Defect Incorrect behavior (e.g. crash, hang)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant