Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ITV] Update ITV extractor from yt-dlp back-port #30266

Merged
merged 10 commits into from
Feb 3, 2023

Conversation

dirkf
Copy link
Contributor

@dirkf dirkf commented Nov 24, 2021

Please follow the guide below


Before submitting a pull request make sure you have:

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

  • I am the original author of this code and I am willing to release it under Unlicense, except where
  • I am not the original author of this code but it is in public domain or released under Unlicense (yt-dlp/yt_dlp/extractor/itv.py)

What is the purpose of your pull request?

  • Bug fix
  • Improvement
  • New extractor
  • New feature

Description of your pull request and other information

This PR back-ports the ITV extractor from yt-dlp, so incorporating all the latest fixes and facilitating future joint updates.

The PR from #28955 is included for the record as an an interim commit.

In addition the following changes are applied:

  • work-around for ITV using : as a decimal point in duration value
  • set default UA for extraction as Mozilla/5.0 to avoid read time-out in extraction
  • ditto for download to avoid 403
  • detect and handle 404 error for Episode not found and We're really sorry.

Resolves #25796.
Resolves #27582.
Resolves #28906.
Resolves #29620.
Resolves #30265.
Resolves #30578.
Resolves #30994.

Closes #28955.

Now updated for ITVX instead/as well.

Resolves #31363.

Also updated ITV BTCC, handles news articles.

Supersedes, closes #25254.

ITV changes require media stream and subtitles to be taken
from different playlist resources.

Original commit 4c57dd2 by @sleaux-meaux 3 May 2021
@dirkf dirkf mentioned this pull request Nov 24, 2021
3 tasks
@dirkf dirkf force-pushed the sleaux-meaux+staubichsauger-itv-patch branch from 700f295 to bbde767 Compare November 25, 2021 11:52
Includes commit 6fbcc16, fix by staubichsauger & pukkandan
@dirkf dirkf force-pushed the sleaux-meaux+staubichsauger-itv-patch branch from bbde767 to a72aca5 Compare November 25, 2021 13:03
@Vangelis66 Vangelis66 mentioned this pull request Dec 1, 2021
11 tasks
@dirkf dirkf force-pushed the sleaux-meaux+staubichsauger-itv-patch branch from 674f0f2 to 8eabd28 Compare January 19, 2022 19:14
@dirkf dirkf force-pushed the sleaux-meaux+staubichsauger-itv-patch branch from 8eabd28 to 0945174 Compare January 19, 2022 19:17
@dirkf dirkf linked an issue Jan 29, 2022 that may be closed by this pull request
5 tasks
@Vangelis66 Vangelis66 mentioned this pull request Feb 1, 2022
5 tasks
@dirkf dirkf mentioned this pull request Mar 30, 2022
@dirkf dirkf changed the title [ITV] Back-port yt-dlp ITV extractor [ITV] Update ITV extractor from yt-dlp back-port Jun 7, 2022
* `Mozilla/5.0` avoids read timeout on extraction and 403 on d/l
* also detect and report `Episode not found` instead of generic 404
@dirkf dirkf force-pushed the sleaux-meaux+staubichsauger-itv-patch branch from c7250bd to c6001d5 Compare June 7, 2022 13:07
@stsquad
Copy link

stsquad commented Oct 7, 2022

I just want to confirm that testing this branch I was able to download what I was looking to where the current upstream master would just timeout and fail.

@dirkf

This comment was marked as outdated.

dirkf added 2 commits January 29, 2023 04:44
* use vanilla UA for ITVIE media
* settle on best effort geo-restriction handling
* handle news articles
@dirkf dirkf mentioned this pull request Jan 29, 2023
10 tasks
@dirkf dirkf merged commit 384f632 into ytdl-org:master Feb 3, 2023
@MysticMerlin
Copy link

Hi,
I'm not a complete novice but I'm having some difficulty following what's going on & whether the problem with the ITVX has been fixed. I can see that dirkf has been very busy & I'h sure there are many thousands of people appreciating all his efforts & anybody else that is capable of contributing. I'm afraid I don't have the experience or knowledge to offer any contribution from my side. So, in layman's terms where are we at as regards ITVX? Thanks.

@dirkf
Copy link
Contributor Author

dirkf commented Feb 3, 2023

The updated extractor is now in the git master repository code that would be used to make a release. A release built from git master should support ITVX. Also, installing from the git master with pip should do so.

github-actions bot added a commit to hellopony/youtube-dl that referenced this pull request Feb 4, 2023
alxlive pushed a commit to alxlive/youtube-dl that referenced this pull request Feb 27, 2023
* support ITVX URLs (thanks Vangelis66)
* support legacy ITV Hub URLs
* include extraction fix 4c57dd2 from sleaux-meaux 3 May 2021
* include extraction fix 6fbcc16, fix by staubichsauger & pukkandan
* work-around duration parsing pending fix to utils.parse_duration
* apply default vanilla UA for pages and media to avoid site blocking
* also detect and report `Episode not found` instead of generic 404
* rework ITVBTCCIE with geo-block detection, best effort geo-restriction handling, news article support
* fix tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment