Add an option to skip hash as identification method

[anderbytes]

More frequently than it should be, hash doesn't grants that the subtitle being downloaded is the correct one. Probably because of typing error or confusion/copy-paste when the uploaders post it.

In those cases... we should be able to put a parameter in command line to ask Subliminal to NOT use hash as a method of search, but searching for filename / release instead.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[jtwill]

I second this. I frequently see the wrong subtitle downloaded because the provider has the wrong hash for the subtitle.

[Diaoul]

Challenging the hash with other matches is already done in opensubtitles here and there. Please try with develop branch and report back.

[anderbytes]

Hey Diaoul, I've read the codes you quoted. But there are lot of times where even inside the same Title there are errors in hashes.

Eventually I find myself relying much more on full-names (title, year, quality, codec, release group) than hashes.

If you could skip hash check or give it a 0 (zero) score when a specific parameter is given... we would appreciate.

* -- AND MORE: --*
Recently I've began to create a manual (automatic, soon) procedure of adjusting several metadata of my video files. Removing embeeded subtitles, removing unused audio track, tagging it...etc...

When that happens, the adjusted video file comes with a different hash. So I'm forced to remember using subliminal right after download, because later this won't be possible, forever for that file!.

So... using exclusively the filename and/or metadata instead of hash has to be an option.

Thanks.

[Diaoul]

Please provide examples.
Researching by hash doesn't dismiss other searches so searching with a wrong hash will yield no results but the other full text searches will.

[anderbytes]

I'm not in my home computer right now... so testing will only be possible at night.

I'll respond as soon as possible, then.

[Diaoul]

Any updates on this?

[anderbytes]

Sorry. had a lot in my mind and I forgot to come back here.

Debugging some more revealed me an ugly truth... hashes aren`t perfect and they NEED to have an option to be ignored completely.

An episode called The.Big.Bang.Theory.S01E08.The.Grasshopper.Experiment.HDTV-720p.x264.AC3-MRSK.mkv (*_MRSK *_is also know as *_Max *_in some files) in debug mode of subliminal uses the hash c48b1267359eacbf . When I went manually inside the site to search using the hash, it returned me a generic subtitle that "apparently" could serve more than 20 different releases, including ReEnc-Max release.

But thats a lie! the times in release uploaded are totally wrong, Legendas.TV although is perfect for the same release / subtitle name.... go figure.

[Diaoul]

Do you have a link to that subtitle? Or the video hash? Please provide more information. I doubt I will be able to do something about it given the nature of the issue. Full text search might just as well return the same result from opensubtitles if the uploaded did say this subtitle matched all those releases.

The hash search result is already challenged with other properties (series, season and episode for an Episode and title for a Movie).

[h3llrais3r]

Sorry to hijack this issue, but I have the same problem (running on master branch in API mode)
Example:

2016-04-19 21:10:02,262 INFO     [CP Server Thread-2 :: subliminal.video] Scanning video u'Southpaw.2015.1080p.BluRay.x264-SPARKS.mkv' in u'\\\\SERVER\\couchpotato-autosubliminal\\Southpaw.2015.1080p.BluRay.x264-SPARKS[rarbg]'
2016-04-19 21:10:02,371 INFO     [CP Server Thread-2 :: enzyme.mkv] Reading Segment element
2016-04-19 21:10:02,371 INFO     [CP Server Thread-2 :: enzyme.parsers.ebml.core] MasterElement EBML ignored
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: enzyme.parsers.ebml.core] Maximum level 0 reached for children of MasterElement Segment
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: enzyme.mkv] Reading SeekHead element
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: enzyme.mkv] Processing element Info from SeekHead at position 4151
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: enzyme.mkv] Processing element Tracks from SeekHead at position 4284
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: enzyme.mkv] Processing element Tags from SeekHead at position 9385936174
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: enzyme.mkv] Processing element Chapters from SeekHead at position 5555
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: subliminal.api] Listing subtitles for 
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: subliminal.api] Listing subtitles with provider u'opensubtitles' and languages set([])
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: subliminal.api] Initializing provider opensubtitles
2016-04-19 21:10:02,387 INFO     [CP Server Thread-2 :: subliminal.providers.opensubtitles] Logging in
2016-04-19 21:10:02,605 INFO     [CP Server Thread-2 :: subliminal.providers.opensubtitles] Searching subtitles [{'moviebytesize': '9385937011', 'sublanguageid': u'dut', 'moviehash': 'fc2eede6b9d200ec'}]
2016-04-19 21:10:02,667 INFO     [CP Server Thread-2 :: subliminal.api] Skipping provider u'addic7ed': not a valid video
2016-04-19 21:10:02,667 INFO     [CP Server Thread-2 :: subliminal.api] Found 0 subtitle(s)
2016-04-19 21:10:02,667 INFO     [CP Server Thread-2 :: subliminal.api] Terminating provider opensubtitles
2016-04-19 21:10:02,667 INFO     [CP Server Thread-2 :: subliminal.providers.opensubtitles] Logging out

When a hash is present, it only seems to search for the hash.
When I, just to test, use an empty file with the same name, hash calculation is skipped, query is used and a subtitles are found on OpenSubtitles

2016-04-19 22:06:10,769 INFO     [CP Server Thread-9 :: subliminal.video] Scanning video u'Southpaw.2015.1080p.BluRay.x264-SPARKS.mkv' in u'c:\\temp\\autosubliminal\\completed'
2016-04-19 22:06:11,015 WARNING  [CP Server Thread-9 :: subliminal.video] Size is lower than 10MB: hashes not computed
2016-04-19 22:06:11,016 INFO     [CP Server Thread-9 :: enzyme.mkv] Reading Segment element
2016-04-19 22:06:11,115 ERROR    [CP Server Thread-9 :: subliminal.video] Parsing video metadata with enzyme failed
Traceback (most recent call last):
  File "C:\Tools\GitHub\Auto-Subliminal\lib\subliminal\video.py", line 322, in scan_video
    mkv = MKV(f)
  File "C:\Tools\GitHub\Auto-Subliminal\lib\enzyme\mkv.py", line 42, in __init__
    raise MalformedMKVError('No Segment found')
MalformedMKVError: No Segment found
2016-04-19 22:06:12,394 INFO     [CP Server Thread-9 :: subliminal.api] Listing subtitles for 
2016-04-19 22:06:15,760 INFO     [CP Server Thread-9 :: subliminal.api] Listing subtitles with provider u'opensubtitles' and languages set([])
2016-04-19 22:06:15,762 INFO     [CP Server Thread-9 :: subliminal.api] Initializing provider opensubtitles
2016-04-19 22:06:30,119 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Logging in
2016-04-19 22:06:48,733 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Searching subtitles [{'query': u'Southpaw.2015.1080p.BluRay.x264-SPARKS.mkv', 'sublanguageid': u'dut'}]
2016-04-19 22:06:59,391 INFO     [CP Server Thread-9 :: subliminal.api] Skipping provider u'addic7ed': not a valid video
2016-04-19 22:06:59,392 INFO     [CP Server Thread-9 :: subliminal.api] Found 5 subtitle(s)
2016-04-19 22:06:59,394 INFO     [CP Server Thread-9 :: subliminal.api] Terminating provider opensubtitles
2016-04-19 22:06:59,397 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Logging out
2016-04-19 22:06:59,720 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle 
2016-04-19 22:06:59,723 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle 
2016-04-19 22:06:59,726 INFO     [CP Server Thread-9 :: subliminal.api] Initializing provider opensubtitles
2016-04-19 22:07:00,562 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Logging in
2016-04-19 22:07:00,809 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Downloading subtitle 
2016-04-19 22:07:01,759 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle 
2016-04-19 22:07:01,760 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle 
2016-04-19 22:07:01,762 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Downloading subtitle 
2016-04-19 22:07:03,158 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle 
2016-04-19 22:07:03,160 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle 
2016-04-19 22:07:03,161 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Downloading subtitle 
2016-04-19 22:07:03,885 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle 
2016-04-19 22:07:03,887 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle 
2016-04-19 22:07:03,887 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Downloading subtitle 
2016-04-19 22:07:04,677 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle 
2016-04-19 22:07:04,678 INFO     [CP Server Thread-9 :: subliminal.api] Downloading subtitle 
2016-04-19 22:07:04,680 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Downloading subtitle 
2016-04-19 22:07:05,282 INFO     [CP Server Thread-9 :: subliminal.api] Terminating provider opensubtitles
2016-04-19 22:07:05,282 INFO     [CP Server Thread-9 :: subliminal.providers.opensubtitles] Logging out
2016-04-19 22:07:07,332 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computing score for matches set(['year', 'video_codec', 'title', 'resolution']) and 
2016-04-19 22:07:07,334 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computed score 43
2016-04-19 22:07:07,880 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computing score for matches set(['year', 'title']) and 
2016-04-19 22:07:07,881 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computed score 35
2016-04-19 22:07:08,371 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computing score for matches set(['video_codec', 'year', 'title']) and 
2016-04-19 22:07:08,372 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computed score 39
2016-04-19 22:07:08,578 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computing score for matches set(['video_codec', 'year', 'title']) and 
2016-04-19 22:07:08,578 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computed score 39
2016-04-19 22:07:08,910 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computing score for matches set(['year', 'video_codec', 'format', 'title', 'release_group']) and 
2016-04-19 22:07:08,911 INFO     [CP Server Thread-9 :: subliminal.subtitle] Computed score 56

[h3llrais3r]

@Diaoul can you have a look at this? Is this sufficient info for you?