Scanner Limitations

[coderforlife]

I am new to Plex development and I am trying to figure out a few things.

 

First off, the custom scanners must be in the Scanners bundle, correct? There is no way to have them in another bundle (I am creating a plugin with an Agent as well, and the Agent will work with the scanner). I guess upon installing the bundle I could copy the scanner to the right place, but I haven't looked into that yet.

 

Next, scanners do not have access to any of the standard plug-in infrastructure like the Prefs and Data objects? Is there a way to get access to them? Additionally, there seems to be no way to write to the log files except by throwing an exception.

 

Third, what is the difference between the Turbo and Deep scan calls to the Scan function?

 

Thanks!

[coderforlife]

One last thing: Can I store extra data on the Media objects for us by my custom agent later? (such as an ID).

[mmccurdy]

Custom scanners should go in the "Scanners" directory in the app support directory (the one that also contains Plug-ins, etc. on your platform of choice).  Agents should live in their own bundles.  Scanners do not have access to Framework functions (including logging) but you can use normal "print" statements and run the scanner directly at the command line for debugging purposes.  In the latest versions, there is no difference between turbo and deep scans -- that distinction will also go away in the UI eventually.  Finally, no, there's no way to store arbitrary metadata, but there has been some discussion of better support for additional/shared ID's.

[coderforlife]

So basically, they are as limited as I was realizing.

I can get around the Scanners bundle thing by copying it when the plugin is first installed/used. (I believe, maybe you can confirm that if a bundle is representing an Agent that some of its code is run BEFORE the agent is run the first time).

Are there any known workarounds for the lack of Data and Prefs? Is the working directory for the scanner scripts predictable/consistent across installs and platforms?

Thanks for the workaround for logging.

For the ID part, if the Media object is simply copied (or serialized/unserialized) from the scanner's results to the agent, wouldn't any extra attributes be copied as well? If the agent doesn't use the extra attribute, then it would be dropped, but if you had an agent which knew to look for it, then it would be able to use it?

For Turbo vs Deep, why was this removed? I have a wonderful use for it. The metadata is all in a database and I can check that the database has not changed since the last scan and I would know not to do any more scanning. Additionally, each entry in the database is time-stamped, allowing to skip individual items if only some have changed. This would greatly speed things up. As it stands right now, if I return only a partial list of items all the other items are deleted.

[mmccurdy]

Can you explain a little more about what you're trying to accomplish?  I think there may be some disconnects in understanding how the plug-in framework operates.  For example,  well-behaved plugins should not be copying things in the filesystem or maintaining their own external databases, objects are not blindly serialized and deserialized by agents, and turbo and deep scans were (and "scan" still is) fairly well proscribed behaviors dictated by the Media Server itself, also not under the control of any given plug-in.

[coderforlife]

I guess I could say that all of my plugin parts that are "mis-behaved" are because the framework is "mis-behaved"... I do try to do things right, and when I can't I start to be a little more creative. I came to this forum to make sure there wasn't a "proper" way to do these things.

The copying thing is simply so the end-user doesn't have to copy the scanner file into scanner bundle. There is no other way, to my understanding, for a bundle/plugin to add a new scanner. Everyone has to manually copy this file into the appropriate directory.

Being able for the scanner to read Prefs and persistent Data seems well-behaved, but there is no standard way to do that as those modules are unavailable to scanners. Although I was only planning to use persistent data for a "Turbo Scan" which isn't going to happen anyways.

The external database of music files is generated by another program but I want to make it accessible via Plex. It is not maintaining a database, simply reading it. This is how Plex normally operates - it reads the filesystem and periodically looks for changes. I am just looking in a database instead of filesystem.

For the Turbo vs Deep scan, I simply wanted my Virtual scanner to take advantage of this fact. If the Scan method was called in "Turbo" mode I would be able to skip adding most media files or something similar and thus speed up the scanner and  the resulting matching of media/Agent processing. I wasn't looking to change how it worked for other scanners, but had a very nice setup for my own scanner. If there is no difference and only "Deep Scans" are ever done, then I will always return the complete set of media objects.

My understanding of how the scanner and agent work together are as follows:

• The specific scanner script generates a list of Media objects, each with basic information about the media
• The overall scanner program takes the list and saves it (presumably serializes the object)
• The main framework loads the list (presumably deserializes)
• Each item is matched against items already in the database, and if it has never been seen before is sent to the Agent
• The Agent receives a Media object with basic information and creates a new ObjectContainer with corrected/complete information about the media

So, I was thinking that if I saved a Media object with an extra attribute that by the time it got to the Agent it would still have that extra attribute. Most Agents would ignore that extra attribute and not use it was trying to find the corrected/complete attribute. However I could make an Agent that would check for that extra attribute and use it as necessary (with a fallback if it isn't there). This extra attribute would never show up in the final ObjectContainer, no other Agents would have to be aware of it, and the Agent would not be doing the serialization/deserialization - the Framework presumably already does it.

I am not a haphazard coder. I do write "well-behaved" code and try not to do things that shouldn't be done. Everything I want to do seems like it should be possible (installing a scanner non-manually, access to minimal Framework functions from scanner, reading an external file/database that the user directly pointed to, take advantage of modes other scanners used to be able to, and sending one additional int from scanner to agent). If you would also like to know, I try to trap all my exceptions and check return values...

[iphoneuser]

I found some strang behaviour with that unified turbo/deep scans, too. If you change the scanner / agent later in a section most of the time no rescan is done. Although using deep scan does not work anymore.

The only way to get this running is to remove and add the section. There seem to be several bugs in the scanner. Sometimes you even have to completely restart pms.