Thanks guys!
@pshanew Since posting this, and naively expecting this problem to vacate my headspace so I can move on as a result, I jumped right back into testing other ideas.
I like those observations. Even as I was typing things out, it felt strange thinking that zero-length files could possibly get through the most basic testing. I can clearly picture a process where tonnes of zero-length files are used for development purposes. Thinking that, I started tinkering with a few other things, such as what might be happening from inside the container vs the host.
When running commands like head -c 1014 foobarguys.mkv on a file that is on the local device, it will properly show the first 1024 bytes of the file. An example of that output is just below. Regardless of this being run from the host or the container, the output is the same for a file on a local device. This is good.
EߣBBBBmatroskaBBSg
.MtMSIfSMSTkSvMSSkS MSTgS
*MSCpSO
When running the same command on a file that is not on the local device, in other words it has been “optimized,” the output differs from host to container.
On the host, being MacOS, the deep integration of iCloud Drive kicks in and the process is held until the file downloads, then the usual output is provided. Also good. Doing the same inside the Docker container results in different output, which is probably a helpful clue.
The output in the Docker container for the very same command shows the following:
$ head -c foobarguys.mkv
head: error reading 'foobarguys.mkv': Resource deadlock avoided
In fact, numerous commands that open up files, like tail, head, and cat show the same output: Resource deadlock avoided.
Based on this alone, I think my hypothesis of the zero-length file goes out the window. The files are clearly something else. Tied in with your own testing, it’s not this, then.
Zero blocks
I tried thinking of other useful commands to get me more information about a file, so I ran a trusty stat command on the file. The output is interesting.
On a file that is local to the device, it provides output like this, both on the host and the container:
File: foobarguys.mkv
Size: 20511046260 Blocks: 40060640 IO Block: 4096 regular file
Device: 1,14 Inode: 125991320 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 501/danemacmillan) Gid: ( 20/ staff)
Access: 2025-01-14 09:29:24.564701351 -0500
Modify: 2025-01-10 18:06:48.137549678 -0500
Change: 2025-01-10 19:07:40.439014854 -050
Birth: 2025-01-10 18:06:48.137549678 -0500
When running the same command on a file that has been optimized, they fortunately are also the same, so instead of the container just providing some obscure output, it actually can get some information about the file, even though it’s not on the local device.
File: foobarguys.mkv
Size: 20511046260 Blocks: 0 IO Block: 4096 regular file
Device: 1,14 Inode: 125991320 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 501/danemacmillan) Gid: ( 20/ staff)
Access: 2025-01-14 09:29:24.564701351 -0500
Modify: 2025-01-10 18:06:48.137549678 -0500
Change: 2025-01-10 19:07:40.439014854 -050
Birth: 2025-01-10 18:06:48.137549678 -0500
In the former and the latter, the Size is reported correctly at all times (even using a basic ls -lah in the directory it knows the size , but the Blocks change. This is where my limits in knowledge are, but I believe this means the filesystem is not actually allocating any device space for it.
Local PMS
I haven’t installed it locally on MacOS, but I suspect this will be my next test. I’ll update when I have more. My hypothesis is that either PMS on MacOS will natively trigger the iCloud Drive download, in which case the crash and corruption won’t occur, or it will do the same thing as the container.
Does hardware accelerated transcoding work for you in that environment?
I haven’t tried this yet, but it was something I wanted to test when I got this sorted out. One thing I’ll say is that with Apple Silicon I pretty much never see the computer struggle (or make noise) ever, so it might not be as relevant on this hardware. The M2 Max would just eat up whatever’s thrown at it.
