Is there a reason something like Sqlite isn’t used to store/read the PLY files or other metadata? Surely thousands of files on disk isn’t an optional solution … I assume you’re taking steps to parition out the files to say 1000 per directory to maximize ulimit and file read operations … Using g Sqlite could alleviate some of these issues.
Please pardon my ignorance, I’m not a graphics guy although I find it fascinating. I’m a webdev/polygot programmer …
It’s almost certainly fine. SQLite is useful when you want to query data or manipulate it in ways SQL is designed for. If you’re accessing objects by name, a filesystem is fine, already supported in your programming environment, easy to replicate across a large cluster, and on modern systems, is totally fine with tens of thousands of files per directory.
Part 2 is at http://pharr.org/matt/blog/2018/07/09/moana-island-pbrt-2.html
Is there a reason something like Sqlite isn’t used to store/read the PLY files or other metadata? Surely thousands of files on disk isn’t an optional solution … I assume you’re taking steps to parition out the files to say 1000 per directory to maximize ulimit and file read operations … Using g Sqlite could alleviate some of these issues.
Please pardon my ignorance, I’m not a graphics guy although I find it fascinating. I’m a webdev/polygot programmer …
It’s almost certainly fine. SQLite is useful when you want to query data or manipulate it in ways SQL is designed for. If you’re accessing objects by name, a filesystem is fine, already supported in your programming environment, easy to replicate across a large cluster, and on modern systems, is totally fine with tens of thousands of files per directory.