Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use patched fuser crate #8997

Draft
wants to merge 19 commits into
base: releases/3.2
Choose a base branch
from

Conversation

FirelightFlagboy
Copy link
Contributor

@FirelightFlagboy FirelightFlagboy commented Nov 26, 2024

TODOs

  • Increase block size to 512 Ko, and use the value every where (statfs & file attr) (5f8a64d)

To inspect

  • Compare the return value of opendir on the root folder
  • Compare the return value of readdir on the root folder
  • Compare the return value of releasedir on the root folder
  • Does added file have a correct date (form crtime, ctime, mtime, atime) ?
  • Does modified file have a correct date (form crtime, ctime, mtime, atime) ?
  • Compare statfs return value
  • Compare getattr
    • getattr for root folder
    • getattr for added file
    • getattr for temp added file (._<file>)
    • getattr for modified file
    • getattr for temp modified file (._<file>)
  • Compare create
    • create for added file
    • create for temp added file (._<file>)
    • create for modified file
    • create for temp modified file (._<file>)
  • Compare open
    • open for added file
    • open for temp added file (._<file>)
    • open for modified file
    • open for temp modified file (._<file>)
  • Compare write
    • write for added file
    • write for temp added file (._<file>)
    • write for modified file
    • write for temp modified file (._<file>)
  • Compare read
    • read for added file
    • read for temp added file (._<file>)
    • read for modified file
    • read for temp modified file (._<file>)
  • Compare flush
    • flush for added file
    • flush for temp added file (._<file>)
    • flush for modified file
    • flush for temp modified file (._<file>)
  • Compare close
    • close for added file
    • close for temp added file (._<file>)
    • close for modified file
    • close for temp modified file (._<file>)
  • fuse-memfs impl allow for ._<file> to be created but not parsec-cloud impl, why is that?

getattr

Compare the return value of getattr on root folder

Attr returned by memfs implementation
FileAttr {
	ino: 1,
	size: 0,
	blocks: 0,
	atime: SystemTime {
		tv_sec: 1736506310,
		tv_nsec: 465954000
	},
	mtime: SystemTime {
		tv_sec: 1736506310,
		tv_nsec: 465954000
	},
	ctime: SystemTime {
		tv_sec: 1736506310,
		tv_nsec: 465954000
	},
	crtime: SystemTime {
		tv_sec: 1736506310,
		tv_nsec: 465954000
	},
	kind: Directory,
	perm: 493,
	nlink: 1,
	uid: 505,
	gid: 20,
	rdev: 0,
	blksize: 1048576,
	flags: 0
}
Attr returned by parsec implementation
FileAttr {
	ino: 1,
	size: 0,
	blocks: 0,
	atime: SystemTime {
		tv_sec: 947548800,
		tv_nsec: 0
	},
	mtime: SystemTime {
		tv_sec: 947548800,
		tv_nsec: 0
	},
	ctime: SystemTime {
		tv_sec: 947548800,
		tv_nsec: 0
	},
	crtime: SystemTime {
		tv_sec: 947548800,
		tv_nsec: 0
	},
	kind: Directory,
	perm: 448,
	nlink: 1,
	uid: 505,
	gid: 20,
	rdev: 0,
	blksize: 524288,
	flags: 0
}
Diff between the returned value of memfs and parsec implementation
--- memfs-getattr-root.ron	2025-01-10 16:48:19.951911240 +0100
+++ parsec-getattr-root.ron	2025-01-10 16:48:09.467805743 +0100
@@ -3,27 +3,27 @@
 	size: 0,
 	blocks: 0,
 	atime: SystemTime {
-		tv_sec: 1736506310,
-		tv_nsec: 465954000
+		tv_sec: 947548800,
+		tv_nsec: 0
 	},
 	mtime: SystemTime {
-		tv_sec: 1736506310,
-		tv_nsec: 465954000
+		tv_sec: 947548800,
+		tv_nsec: 0
 	},
 	ctime: SystemTime {
-		tv_sec: 1736506310,
-		tv_nsec: 465954000
+		tv_sec: 947548800,
+		tv_nsec: 0
 	},
 	crtime: SystemTime {
-		tv_sec: 1736506310,
-		tv_nsec: 465954000
+		tv_sec: 947548800,
+		tv_nsec: 0
 	},
 	kind: Directory,
-	perm: 493,
+	perm: 448,
 	nlink: 1,
 	uid: 505,
 	gid: 20,
 	rdev: 0,
-	blksize: 1048576,
+	blksize: 524288,
 	flags: 0
 }

[!NOTE]

  • The most noticeable change seems to be the permission.
  • Nanosec should be handled for new file

Compare the return value of getattr on added folder

First attr returned by memfs
FileAttr {
    ino: 5,
    size: 99961,
    blocks: 1,
    atime: SystemTime {
        tv_sec: 1736765693,
        tv_nsec: 346640000
    },
    mtime: SystemTime {
        tv_sec: 1720189221,
        tv_nsec: 758501053
    },
    ctime: SystemTime {
        tv_sec: 1736765693,
        tv_nsec: 346640000
    },
    crtime: SystemTime {
        tv_sec: 1720189221,
        tv_nsec: 758234143
    },
    kind: RegularFile,
    perm: 33188,
    nlink: 1,
    uid: 505,
    gid: 20,
    rdev: 0,
    blksize: 1048576,
    flags: 0
}

lookup

lookup on added file

For memfs, all the lookup op returned file not found for the added file

lookup on added ._<file>

First attr return by memfs
FileAttr {
    ino: 6,
    size: 8192,
    blocks: 1,
    atime: SystemTime {
        tv_sec: 1736765693,
        tv_nsec: 351021000
    },
    mtime: SystemTime {
        tv_sec: 1736765693,
        tv_nsec: 351021000
    },
    ctime: SystemTime {
        tv_sec: 1736765693,
        tv_nsec: 351021000
    },
    crtime: SystemTime {
        tv_sec: 1736765693,
        tv_nsec: 351021000
    },
    kind: RegularFile,
    perm: 33188,
    nlink: 1,
    uid: 505,
    gid: 20,
    rdev: 0,
    blksize: 1048576,
    flags: 0
}

create

create on added file

Attribute returned when creating a new file with memfs impl
FileAttr {
    ino: 3,
    size: 0,
    blocks: 0,
    atime: SystemTime {
        tv_sec: 1736859377,
        tv_nsec: 411163000
    },
    mtime: SystemTime {
        tv_sec: 1736859377,
        tv_nsec: 411163000
    },
    ctime: SystemTime {
        tv_sec: 1736859377,
        tv_nsec: 411163000
    }, crtime: SystemTime {
        tv_sec: 1736859377,
        tv_nsec: 411163000
    },
    kind: RegularFile,
    perm: 33188,
    nlink: 1,
    uid: 505,
    gid: 20,
    rdev: 0,
    blksize: 1048576,
    flags: 0
}

open

Nothing notable for memfs (it just open the file), flags are set to 0 (a single time the value is set to 1, indicating that O_WRONLY is set)

write

With memfs impl, write write all the provided data and return the expected written bytes value.
For the test file, it's done in 3 call with data.len and offset equals to (65536, 0), (32768, 65536), (1657, 98304)

read

With memfs impl, read return a blob of size bytes from the file.
For the test file, it's done in 2 call (to read the full file) with a size of 65536 and 34425

flush

Flush return nothing

close (or release)

Release return nothing

statfs

First stat

Stat returned by memfs impl
StatFS {
    blocks: 1,
    bfree: 2_097_151,
    bavail: 2_097_152,
    files: 2,
    ffree: 18_446_744_073_709_551_613,
    bsize: 1_048_576,
    namelen: 4_294_967_295,
    frsize: 0,
}
Stat returned by parsec impl
Statfs {
    blocks: 0,
    bfree: 2_097_152,
    bavail: 2_097_152,
    files: 0,
    ffree: 18_446_744_073_709_551_615,
    bsize: 524_288,
    namelen: 255,
    frsize: 524_288,
}
Diff between the returned value or memfs and parsec impl
--- memfs-statfs-first.ron	2025-01-14 14:50:44.979218957 +0100
+++ parsec-statfs-first.ron	2025-01-15 11:54:07.767783575 +0100
@@ -1,10 +1,10 @@
 StatFS {
-    blocks: 1,
+    blocks: 0,
-    bfree: 2_097_151,
+    bfree: 2_097_152,
     bavail: 2_097_152,
-    files: 2,
+    files: 0,
-    ffree: 18_446_744_073_709_551_613,
+    ffree: 18_446_744_073_709_551_615,
-    bsize: 1_048_576,
+    bsize: 524_288,
-    namelen: 4_294_967_295,
+    namelen: 255,
-    frsize: 0,
+    frsize: 524_288,
 }

Explanation of the diff

For blocks: parsec always return 0, this could cause problem if the OS expect that value to be updated to check if a write operation is correct done,

For bfree: Since blocks is always 0, bfree is always set to bavail value

For files: parsec return the value 0 which is incorrect here since the mountpoint is initialized with some files at the start (root dir, a folder and a file)

For ffree: Since files is incorrect that value also is.

For bsize: Both impl use different block size

For namelen: Both impl use different limitation

For frsize: Parsec set the value to BLOCK_SIZE since it does not handle block smaller than BLOCK_SIZE,
here memfs value is incorrect since like parsec it does not handle smaller block

A test was made with memfs where the value blocks, bfree, files and ffree where constant value, and the test still succeeded.

After added file

Stat returned by memfs impl
StatFS {
    blocks: 2,
    bfree: 2_097_150,
    bavail: 2_097_152,
    files: 4,
    ffree: 18_446_744_073_709_551_611,
    bsize: 1_048_576,
    namelen: 4_294_967_295,
    frsize: 0,
}

The impl of parsec is not correct:

  • The number files used stay at 0 all the time (causing ffree to remain unchanged)
  • The number blocks stay at 0 all the time (causing bfree to remain unchanged)

Note

As indicated in statfs.first stat, those values do not seem to be causing problem
Since having them set to fixed values did not prevent memfs to conclude the test.

fuse-memfs impl allow for ._<file> to be created but not parsec-cloud impl, why is that ?

TODO: Test manually creating the ._<file>

@FirelightFlagboy
Copy link
Contributor Author

FirelightFlagboy commented Nov 26, 2024

More recent build on top

Commit Workflow Run
2e89ef0 12065830863
d52e651 12065193885
7e340a1 12064566460
18e5bfc 12034152119
8e142a2 12034083497

@FirelightFlagboy FirelightFlagboy force-pushed the 8907-macfuse-overflow-time branch 2 times, most recently from d52e651 to ff598df Compare November 28, 2024 09:06
@FirelightFlagboy FirelightFlagboy changed the base branch from master to releases/3.2 November 28, 2024 09:07
@FirelightFlagboy FirelightFlagboy force-pushed the 8907-macfuse-overflow-time branch from 4c0bd83 to 5e67f47 Compare November 28, 2024 12:51
@FirelightFlagboy FirelightFlagboy mentioned this pull request Dec 4, 2024
7 tasks
@FirelightFlagboy
Copy link
Contributor Author

While be merged once the other blocking issues listed in #9115 are merged or ready to review

@FirelightFlagboy FirelightFlagboy force-pushed the 8907-macfuse-overflow-time branch from ba0eee2 to c0904c5 Compare January 21, 2025 10:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants