Tuesday, August 27, 2019
Hello list,
I've noticed ActiveStorage::Service::DiskService does two levels of directory sharding for Blob pathnames:
$ tree storage/
storage/
+-- aa/
+-- bb/
| +-- aabb00...
+-- cc/
+-- aacc11...
However, when it comes to variants, the sharding is circumvented:
$ tree storage/
storage/
+-- aa/
| +-- bb/
| | +-- aabb00...
| +-- cc/
| +-- aacc11...
+-- va/
+-- ri/
+-- variants/
+-- aabb00.../
| +-- <encoded variant file name>
+-- aacc11.../
+-- <encoded variant file name>
Should sharding exclude variant/ key prefixes? Maybe the directory layout can look like this:
$ tree storage/
storage/
+-- aa/
| +-- bb/
| | +-- aabb00...
| +-- cc/
| +-- aacc11...
+-- variants/
+-- aa/
+-- bb/
| +-- aabb00.../
| +-- <encoded variant file name>
+-- cc/
+-- aacc11.../
+-- <encoded variant file name>
This might not actually a problem at all, as the limiting factor on ext4 file systems is the inode index (which allow directories to contain ~10 million entries with 32 character long names (reference); the Base58 blob id is only 24 characters long). I guess with ZFS it's even less of a problem.
Kind Regards,
Dominik
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/6624225b-b14e-4143-82d2-c75a94e56c6c%40googlegroups.com.
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment