If you don’t place the videos and the subtitle in their own folder, every file has to be scanned. Any glitch will prevent the subtitles from being added/removed.
Plex may still think the subtitle is there even after you deleted the file. You can double check this by reviewing the XML. https://support.plex.tv/hc/en-us/articles/201998867-Investigate-Media-Information-and-Formats
You can also Plex Dance the file to reset what PMS thinks about the file and any subtitles. https://forums.plex.tv/discussion/comment/908454/#Comment_908454
