ChatGPT Collection Analysis (Artificial Intelligence)

General Discussions
1

I know this topic is not new when it comes to PlexAmp, Tidal, and music libraries already. But I was interested in whether a large language model such as ChatGPT 4 could help me find and surface films from my movie libraries.

ChatGPT 4 told me that my list of 5000 or so films would be no trouble to ingest via its file upload feature, which I did. (btw, simply producing a list of one’s library contents in Plex for export of any kind is conspicuously, and embarrassingly, absent from both client and server applications.)

I wanted help, for example, knowing whether I had films in my library that matched whatever collection idea I had in mind. But it can be difficult to identify all or most of the films that might match a collection such as “Swashbucklers” or films about grieving, or films that take place in submarines, or whatever. But a LLM can understand these types of requests easily.

So could a LLM such as ChatGPT 4 peruse a large list of film titles and pick out collections such as these? The short answer is, no.

ChatGPT is very open about its limitations here. By giving it a list of my film collection, it could only analyze the words in the list and match them against my queries much like a search engine might do. It cannot treat the contents of the list as stepping stones for cross reference into their plots, settings, etc. My submarine query, for example, pulled only 6 matches, including the obvious ones you’re thinking about like YELLOW SUBMARINE and RED OCTOBER, THE BOAT, K-19. But it said it matched these based on their titles only and could not look at the plot of, say, BLACK SEA, to determine it was also set entirely within a submarine. It could only “perceive” that when I asked about it specifically. Just as it told me THE WIZARD OF OZ would not match my query.

I see this as a temporary situation considering the rapid growth and investment in this field and in this product specifically. I’m writing this post to start a discussion and to start people thinking on the ways that an “AI”, so-to-speak, can help curate a large collection. I think most of us would be overjoyed to be able to feed plain language queries into a helper that can surface content both for consumption and for presentation. I’m hoping some devs are thinking about it, too.

Would love to hear further thoughts.

2

Just for sake of conversation, GPT3.5 running inside ChatGPT certainly does know the plot of those movies. Why was it saying it didn’t?

3

I had an idea like this a year ago, but never got around to trying it. Basically, I wanted to feed it my list of media, and ask it to come up with new and innovative groupings. See if it could think of something a human wouldn’t normally.

Interesting experiment you did, shame it’s so bad at what should be basic sorting of standard information though. Did you consider asking it to list all movies that specifically include the word “submarine” in their plot summary on imdb? Perhaps being more explicit in your desire will include more results.

4

Asking about any specific film does indeed trigger the ability to construct a plot summary of it. But what 4 was telling me was that, given a list of items in an uploaded file, it could not treat those items as it would if they were supplied in individual queries. It could only process the contents of that file in directly: the titles of films.

My goal was to test whether a large and hard to generate dataset, the plot summaries of a 5000 title collection, could be generated ad hoc and used to enable an entire range of simple queries on one’s library.

Despite specific language, 4 kept returning to the fact that it could not process the available data. Here’s a representative sample of its responses to me after I pointed out that it can answer me about one movie, or five movies, but not the complete set of movies I fed it via file upload:

“You’re correct that I have internal knowledge about a wide range of movies, including their plots and settings. This allows me to provide detailed descriptions of specific films. However, when analyzing your movie list, I’m limited to the text data provided in the list itself. My current capabilities don’t include cross-referencing the titles in your list with my internal database to identify their settings or themes automatically.”

You could say I’m trying to force it to execute a foreach loop over my movie list, generating a plot synopsis or other metadata about them, then allowing me to make queries dependent on that metadata. It understood that and specifically said it couldn’t.

This would essentially be the killer feature in a Plex client: the LLM knows your library contents and can answer any query you can think of to surface content or organize it.