mirror of
https://github.com/l1ving/youtube-dl
synced 2025-03-11 02:57:20 +08:00
Merge branch 'facebook-improve-meta-data-extraction' into fix.25.12.2018
# Conflicts: # youtube_dl/version.py
This commit is contained in:
Avi Peretz
commit
6f2792c707
14
.github/ISSUE_TEMPLATE.md
vendored
14
.github/ISSUE_TEMPLATE.md
vendored
@ -6,12 +6,12 @@
|
||||
|
||||
---
|
||||
|
||||
### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2019.03.01*. If it's not, read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
|
||||
- [ ] I've **verified** and **I assure** that I'm running youtube-dl **2019.03.01**
|
||||
### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2019.03.09*. If it's not, read [this FAQ entry](https://github.com/ytdl-org/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
|
||||
- [ ] I've **verified** and **I assure** that I'm running youtube-dl **2019.03.09**
|
||||
|
||||
### Before submitting an *issue* make sure you have:
|
||||
- [ ] At least skimmed through the [README](https://github.com/rg3/youtube-dl/blob/master/README.md), **most notably** the [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
|
||||
- [ ] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones
|
||||
- [ ] At least skimmed through the [README](https://github.com/ytdl-org/youtube-dl/blob/master/README.md), **most notably** the [FAQ](https://github.com/ytdl-org/youtube-dl#faq) and [BUGS](https://github.com/ytdl-org/youtube-dl#bugs) sections
|
||||
- [ ] [Searched](https://github.com/ytdl-org/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones
|
||||
- [ ] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser
|
||||
|
||||
### What is the purpose of your *issue*?
|
||||
@ -36,7 +36,7 @@ Add the `-v` flag to **your command line** you run youtube-dl with (`youtube-dl
|
||||
[debug] User config: []
|
||||
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
|
||||
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
|
||||
[debug] youtube-dl version 2019.03.01
|
||||
[debug] youtube-dl version 2019.03.09
|
||||
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
|
||||
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
|
||||
[debug] Proxy map: {}
|
||||
@ -51,11 +51,11 @@ Add the `-v` flag to **your command line** you run youtube-dl with (`youtube-dl
|
||||
- Single video: https://youtu.be/BaW_jenozKc
|
||||
- Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc
|
||||
|
||||
Note that **youtube-dl does not support sites dedicated to [copyright infringement](https://github.com/rg3/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free)**. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
|
||||
Note that **youtube-dl does not support sites dedicated to [copyright infringement](https://github.com/ytdl-org/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free)**. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
|
||||
|
||||
---
|
||||
|
||||
### Description of your *issue*, suggested solution and other information
|
||||
|
||||
Explanation of your *issue* in arbitrary form goes here. Please make sure the [description is worded well enough to be understood](https://github.com/rg3/youtube-dl#is-the-description-of-the-issue-itself-sufficient). Provide as much context and examples as possible.
|
||||
Explanation of your *issue* in arbitrary form goes here. Please make sure the [description is worded well enough to be understood](https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient). Provide as much context and examples as possible.
|
||||
If work on your *issue* requires account credentials please provide them or explain how one can obtain them.
|
||||
|
||||
10
.github/ISSUE_TEMPLATE_tmpl.md
vendored
10
.github/ISSUE_TEMPLATE_tmpl.md
vendored
@ -6,12 +6,12 @@
|
||||
|
||||
---
|
||||
|
||||
### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *%(version)s*. If it's not, read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
|
||||
### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *%(version)s*. If it's not, read [this FAQ entry](https://github.com/ytdl-org/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
|
||||
- [ ] I've **verified** and **I assure** that I'm running youtube-dl **%(version)s**
|
||||
|
||||
### Before submitting an *issue* make sure you have:
|
||||
- [ ] At least skimmed through the [README](https://github.com/rg3/youtube-dl/blob/master/README.md), **most notably** the [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
|
||||
- [ ] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones
|
||||
- [ ] At least skimmed through the [README](https://github.com/ytdl-org/youtube-dl/blob/master/README.md), **most notably** the [FAQ](https://github.com/ytdl-org/youtube-dl#faq) and [BUGS](https://github.com/ytdl-org/youtube-dl#bugs) sections
|
||||
- [ ] [Searched](https://github.com/ytdl-org/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones
|
||||
- [ ] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser
|
||||
|
||||
### What is the purpose of your *issue*?
|
||||
@ -51,11 +51,11 @@ Add the `-v` flag to **your command line** you run youtube-dl with (`youtube-dl
|
||||
- Single video: https://youtu.be/BaW_jenozKc
|
||||
- Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc
|
||||
|
||||
Note that **youtube-dl does not support sites dedicated to [copyright infringement](https://github.com/rg3/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free)**. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
|
||||
Note that **youtube-dl does not support sites dedicated to [copyright infringement](https://github.com/ytdl-org/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free)**. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
|
||||
|
||||
---
|
||||
|
||||
### Description of your *issue*, suggested solution and other information
|
||||
|
||||
Explanation of your *issue* in arbitrary form goes here. Please make sure the [description is worded well enough to be understood](https://github.com/rg3/youtube-dl#is-the-description-of-the-issue-itself-sufficient). Provide as much context and examples as possible.
|
||||
Explanation of your *issue* in arbitrary form goes here. Please make sure the [description is worded well enough to be understood](https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient). Provide as much context and examples as possible.
|
||||
If work on your *issue* requires account credentials please provide them or explain how one can obtain them.
|
||||
|
||||
4
.github/PULL_REQUEST_TEMPLATE.md
vendored
4
.github/PULL_REQUEST_TEMPLATE.md
vendored
@ -7,8 +7,8 @@
|
||||
---
|
||||
|
||||
### Before submitting a *pull request* make sure you have:
|
||||
- [ ] At least skimmed through [adding new extractor tutorial](https://github.com/rg3/youtube-dl#adding-support-for-a-new-site) and [youtube-dl coding conventions](https://github.com/rg3/youtube-dl#youtube-dl-coding-conventions) sections
|
||||
- [ ] [Searched](https://github.com/rg3/youtube-dl/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
|
||||
- [ ] At least skimmed through [adding new extractor tutorial](https://github.com/ytdl-org/youtube-dl#adding-support-for-a-new-site) and [youtube-dl coding conventions](https://github.com/ytdl-org/youtube-dl#youtube-dl-coding-conventions) sections
|
||||
- [ ] [Searched](https://github.com/ytdl-org/youtube-dl/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
|
||||
- [ ] Checked the code with [flake8](https://pypi.python.org/pypi/flake8)
|
||||
|
||||
### In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options:
|
||||
|
||||
@ -42,11 +42,11 @@ Before reporting any issue, type `youtube-dl -U`. This should report that you're
|
||||
|
||||
### Is the issue already documented?
|
||||
|
||||
Make sure that someone has not already opened the issue you're trying to open. Search at the top of the window or browse the [GitHub Issues](https://github.com/rg3/youtube-dl/search?type=Issues) of this repository. If there is an issue, feel free to write something along the lines of "This affects me as well, with version 2015.01.01. Here is some more information on the issue: ...". While some issues may be old, a new post into them often spurs rapid activity.
|
||||
Make sure that someone has not already opened the issue you're trying to open. Search at the top of the window or browse the [GitHub Issues](https://github.com/ytdl-org/youtube-dl/search?type=Issues) of this repository. If there is an issue, feel free to write something along the lines of "This affects me as well, with version 2015.01.01. Here is some more information on the issue: ...". While some issues may be old, a new post into them often spurs rapid activity.
|
||||
|
||||
### Why are existing options not enough?
|
||||
|
||||
Before requesting a new feature, please have a quick peek at [the list of supported options](https://github.com/rg3/youtube-dl/blob/master/README.md#options). Many feature requests are for features that actually exist already! Please, absolutely do show off your work in the issue report and detail how the existing similar options do *not* solve your problem.
|
||||
Before requesting a new feature, please have a quick peek at [the list of supported options](https://github.com/ytdl-org/youtube-dl/blob/master/README.md#options). Many feature requests are for features that actually exist already! Please, absolutely do show off your work in the issue report and detail how the existing similar options do *not* solve your problem.
|
||||
|
||||
### Is there enough context in your bug report?
|
||||
|
||||
@ -70,7 +70,7 @@ It may sound strange, but some bug reports we receive are completely unrelated t
|
||||
|
||||
# DEVELOPER INSTRUCTIONS
|
||||
|
||||
Most users do not need to build youtube-dl and can [download the builds](https://rg3.github.io/youtube-dl/download.html) or get them from their distribution.
|
||||
Most users do not need to build youtube-dl and can [download the builds](https://ytdl-org.github.io/youtube-dl/download.html) or get them from their distribution.
|
||||
|
||||
To run youtube-dl as a developer, you don't need to build anything either. Simply execute
|
||||
|
||||
@ -98,7 +98,7 @@ If you want to add support for a new site, first of all **make sure** this site
|
||||
|
||||
After you have ensured this site is distributing its content legally, you can follow this quick list (assuming your service is called `yourextractor`):
|
||||
|
||||
1. [Fork this repository](https://github.com/rg3/youtube-dl/fork)
|
||||
1. [Fork this repository](https://github.com/ytdl-org/youtube-dl/fork)
|
||||
2. Check out the source code with:
|
||||
|
||||
git clone git@github.com:YOUR_GITHUB_USERNAME/youtube-dl.git
|
||||
@ -150,9 +150,9 @@ After you have ensured this site is distributing its content legally, you can fo
|
||||
# TODO more properties (see youtube_dl/extractor/common.py)
|
||||
}
|
||||
```
|
||||
5. Add an import in [`youtube_dl/extractor/extractors.py`](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/extractors.py).
|
||||
5. Add an import in [`youtube_dl/extractor/extractors.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/extractor/extractors.py).
|
||||
6. Run `python test/test_download.py TestDownload.test_YourExtractor`. This *should fail* at first, but you can continually re-run it until you're done. If you decide to add more than one test, then rename ``_TEST`` to ``_TESTS`` and make it into a list of dictionaries. The tests will then be named `TestDownload.test_YourExtractor`, `TestDownload.test_YourExtractor_1`, `TestDownload.test_YourExtractor_2`, etc. Note that tests with `only_matching` key in test's dict are not counted in.
|
||||
7. Have a look at [`youtube_dl/extractor/common.py`](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py) for possible helper methods and a [detailed description of what your extractor should and may return](https://github.com/rg3/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303). Add tests and code for as many as you want.
|
||||
7. Have a look at [`youtube_dl/extractor/common.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/extractor/common.py) for possible helper methods and a [detailed description of what your extractor should and may return](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303). Add tests and code for as many as you want.
|
||||
8. Make sure your code follows [youtube-dl coding conventions](#youtube-dl-coding-conventions) and check the code with [flake8](http://flake8.pycqa.org/en/latest/index.html#quickstart):
|
||||
|
||||
$ flake8 youtube_dl/extractor/yourextractor.py
|
||||
@ -177,7 +177,7 @@ Extractors are very fragile by nature since they depend on the layout of the sou
|
||||
|
||||
### Mandatory and optional metafields
|
||||
|
||||
For extraction to work youtube-dl relies on metadata your extractor extracts and provides to youtube-dl expressed by an [information dictionary](https://github.com/rg3/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303) or simply *info dict*. Only the following meta fields in the *info dict* are considered mandatory for a successful extraction process by youtube-dl:
|
||||
For extraction to work youtube-dl relies on metadata your extractor extracts and provides to youtube-dl expressed by an [information dictionary](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303) or simply *info dict*. Only the following meta fields in the *info dict* are considered mandatory for a successful extraction process by youtube-dl:
|
||||
|
||||
- `id` (media identifier)
|
||||
- `title` (media title)
|
||||
@ -185,7 +185,7 @@ For extraction to work youtube-dl relies on metadata your extractor extracts and
|
||||
|
||||
In fact only the last option is technically mandatory (i.e. if you can't figure out the download location of the media the extraction does not make any sense). But by convention youtube-dl also treats `id` and `title` as mandatory. Thus the aforementioned metafields are the critical data that the extraction does not make any sense without and if any of them fail to be extracted then the extractor is considered completely broken.
|
||||
|
||||
[Any field](https://github.com/rg3/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L188-L303) apart from the aforementioned ones are considered **optional**. That means that extraction should be **tolerant** to situations when sources for these fields can potentially be unavailable (even if they are always available at the moment) and **future-proof** in order not to break the extraction of general purpose mandatory fields.
|
||||
[Any field](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L188-L303) apart from the aforementioned ones are considered **optional**. That means that extraction should be **tolerant** to situations when sources for these fields can potentially be unavailable (even if they are always available at the moment) and **future-proof** in order not to break the extraction of general purpose mandatory fields.
|
||||
|
||||
#### Example
|
||||
|
||||
@ -341,7 +341,7 @@ Incorrect:
|
||||
|
||||
### Use convenience conversion and parsing functions
|
||||
|
||||
Wrap all extracted numeric data into safe functions from [`youtube_dl/utils.py`](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/utils.py): `int_or_none`, `float_or_none`. Use them for string to number conversions as well.
|
||||
Wrap all extracted numeric data into safe functions from [`youtube_dl/utils.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/utils.py): `int_or_none`, `float_or_none`. Use them for string to number conversions as well.
|
||||
|
||||
Use `url_or_none` for safe URL processing.
|
||||
|
||||
@ -349,7 +349,7 @@ Use `try_get` for safe metadata extraction from parsed JSON.
|
||||
|
||||
Use `unified_strdate` for uniform `upload_date` or any `YYYYMMDD` meta field extraction, `unified_timestamp` for uniform `timestamp` extraction, `parse_filesize` for `filesize` extraction, `parse_count` for count meta fields extraction, `parse_resolution`, `parse_duration` for `duration` extraction, `parse_age_limit` for `age_limit` extraction.
|
||||
|
||||
Explore [`youtube_dl/utils.py`](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/utils.py) for more useful convenience functions.
|
||||
Explore [`youtube_dl/utils.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/utils.py) for more useful convenience functions.
|
||||
|
||||
#### More examples
|
||||
|
||||
|
||||
31
ChangeLog
31
ChangeLog
@ -1,3 +1,34 @@
|
||||
version 2019.03.09
|
||||
|
||||
Core
|
||||
* [extractor/common] Use compat_etree_Element
|
||||
+ [compat] Introduce compat_etree_Element
|
||||
* [extractor/common] Fallback url to base URL for DASH formats
|
||||
* [extractor/common] Do not fail on invalid data while parsing F4M manifest
|
||||
in non fatal mode
|
||||
* [extractor/common] Return MPD manifest as format's url meta field (#20242)
|
||||
* [utils] Strip #HttpOnly_ prefix from cookies files (#20219)
|
||||
|
||||
Extractors
|
||||
* [francetv:site] Relax video id regular expression (#20268)
|
||||
* [toutv] Detect invalid login error
|
||||
* [toutv] Fix authentication (#20261)
|
||||
+ [urplay] Extract timestamp (#20235)
|
||||
+ [openload] Add support for oload.space (#20246)
|
||||
* [facebook] Improve uploader extraction (#20250)
|
||||
* [bbc] Use compat_etree_Element
|
||||
* [crunchyroll] Use compat_etree_Element
|
||||
* [npo] Improve ISM extraction
|
||||
* [rai] Improve extraction (#20253)
|
||||
* [paramountnetwork] Fix mgid extraction (#20241)
|
||||
* [libsyn] Improve extraction (#20229)
|
||||
+ [youtube] Add more invidious instances to URL regular expression (#20228)
|
||||
* [spankbang] Fix extraction (#20023)
|
||||
* [espn] Extend URL regular expression (#20013)
|
||||
* [sixplay] Handle videos with empty assets (#20016)
|
||||
+ [vimeo] Add support for Vimeo Pro portfolio protected videos (#20070)
|
||||
|
||||
|
||||
version 2019.03.01
|
||||
|
||||
Core
|
||||
|
||||
42
README.md
42
README.md
@ -1,4 +1,4 @@
|
||||
[](https://travis-ci.org/rg3/youtube-dl)
|
||||
[](https://travis-ci.org/ytdl-org/youtube-dl)
|
||||
|
||||
youtube-dl - download videos from youtube.com or other video platforms
|
||||
|
||||
@ -43,7 +43,7 @@ Or with [MacPorts](https://www.macports.org/):
|
||||
|
||||
sudo port install youtube-dl
|
||||
|
||||
Alternatively, refer to the [developer instructions](#developer-instructions) for how to check out and work with the git repository. For further options, including PGP signatures, see the [youtube-dl Download Page](https://rg3.github.io/youtube-dl/download.html).
|
||||
Alternatively, refer to the [developer instructions](#developer-instructions) for how to check out and work with the git repository. For further options, including PGP signatures, see the [youtube-dl Download Page](https://ytdl-org.github.io/youtube-dl/download.html).
|
||||
|
||||
# DESCRIPTION
|
||||
**youtube-dl** is a command-line program to download videos from YouTube.com and a few more sites. It requires the Python interpreter, version 2.6, 2.7, or 3.2+, and it is not platform specific. It should work on your Unix box, on Windows or on macOS. It is released to the public domain, which means you can modify it, redistribute it or use it however you like.
|
||||
@ -685,7 +685,7 @@ You can merge the video and audio of two formats into a single file using `-f <v
|
||||
|
||||
Format selectors can also be grouped using parentheses, for example if you want to download the best mp4 and webm formats with a height lower than 480 you can use `-f '(mp4,webm)[height<480]'`.
|
||||
|
||||
Since the end of April 2015 and version 2015.04.26, youtube-dl uses `-f bestvideo+bestaudio/best` as the default format selection (see [#5447](https://github.com/rg3/youtube-dl/issues/5447), [#5456](https://github.com/rg3/youtube-dl/issues/5456)). If ffmpeg or avconv are installed this results in downloading `bestvideo` and `bestaudio` separately and muxing them together into a single file giving the best overall quality available. Otherwise it falls back to `best` and results in downloading the best available quality served as a single file. `best` is also needed for videos that don't come from YouTube because they don't provide the audio and video in two different files. If you want to only download some DASH formats (for example if you are not interested in getting videos with a resolution higher than 1080p), you can add `-f bestvideo[height<=?1080]+bestaudio/best` to your configuration file. Note that if you use youtube-dl to stream to `stdout` (and most likely to pipe it to your media player then), i.e. you explicitly specify output template as `-o -`, youtube-dl still uses `-f best` format selection in order to start content delivery immediately to your player and not to wait until `bestvideo` and `bestaudio` are downloaded and muxed.
|
||||
Since the end of April 2015 and version 2015.04.26, youtube-dl uses `-f bestvideo+bestaudio/best` as the default format selection (see [#5447](https://github.com/ytdl-org/youtube-dl/issues/5447), [#5456](https://github.com/ytdl-org/youtube-dl/issues/5456)). If ffmpeg or avconv are installed this results in downloading `bestvideo` and `bestaudio` separately and muxing them together into a single file giving the best overall quality available. Otherwise it falls back to `best` and results in downloading the best available quality served as a single file. `best` is also needed for videos that don't come from YouTube because they don't provide the audio and video in two different files. If you want to only download some DASH formats (for example if you are not interested in getting videos with a resolution higher than 1080p), you can add `-f bestvideo[height<=?1080]+bestaudio/best` to your configuration file. Note that if you use youtube-dl to stream to `stdout` (and most likely to pipe it to your media player then), i.e. you explicitly specify output template as `-o -`, youtube-dl still uses `-f best` format selection in order to start content delivery immediately to your player and not to wait until `bestvideo` and `bestaudio` are downloaded and muxed.
|
||||
|
||||
If you want to preserve the old format selection behavior (prior to youtube-dl 2015.04.26), i.e. you want to download the best available quality media served as a single file, you should explicitly specify your choice with `-f best`. You may want to add it to the [configuration file](#configuration) in order not to type it every time you run youtube-dl.
|
||||
|
||||
@ -736,7 +736,7 @@ $ youtube-dl --dateafter 20000101 --datebefore 20091231
|
||||
|
||||
### How do I update youtube-dl?
|
||||
|
||||
If you've followed [our manual installation instructions](https://rg3.github.io/youtube-dl/download.html), you can simply run `youtube-dl -U` (or, on Linux, `sudo youtube-dl -U`).
|
||||
If you've followed [our manual installation instructions](https://ytdl-org.github.io/youtube-dl/download.html), you can simply run `youtube-dl -U` (or, on Linux, `sudo youtube-dl -U`).
|
||||
|
||||
If you have used pip, a simple `sudo pip install -U youtube-dl` is sufficient to update.
|
||||
|
||||
@ -746,7 +746,7 @@ As a last resort, you can also uninstall the version installed by your package m
|
||||
|
||||
sudo apt-get remove -y youtube-dl
|
||||
|
||||
Afterwards, simply follow [our manual installation instructions](https://rg3.github.io/youtube-dl/download.html):
|
||||
Afterwards, simply follow [our manual installation instructions](https://ytdl-org.github.io/youtube-dl/download.html):
|
||||
|
||||
```
|
||||
sudo wget https://yt-dl.org/latest/youtube-dl -O /usr/local/bin/youtube-dl
|
||||
@ -780,7 +780,7 @@ Most people asking this question are not aware that youtube-dl now defaults to d
|
||||
|
||||
### I get HTTP error 402 when trying to download a video. What's this?
|
||||
|
||||
Apparently YouTube requires you to pass a CAPTCHA test if you download too much. We're [considering to provide a way to let you solve the CAPTCHA](https://github.com/rg3/youtube-dl/issues/154), but at the moment, your best course of action is pointing a web browser to the youtube URL, solving the CAPTCHA, and restart youtube-dl.
|
||||
Apparently YouTube requires you to pass a CAPTCHA test if you download too much. We're [considering to provide a way to let you solve the CAPTCHA](https://github.com/ytdl-org/youtube-dl/issues/154), but at the moment, your best course of action is pointing a web browser to the youtube URL, solving the CAPTCHA, and restart youtube-dl.
|
||||
|
||||
### Do I need any other programs?
|
||||
|
||||
@ -845,7 +845,7 @@ means you're using an outdated version of Python. Please update to Python 2.6 or
|
||||
|
||||
### What is this binary file? Where has the code gone?
|
||||
|
||||
Since June 2012 ([#342](https://github.com/rg3/youtube-dl/issues/342)) youtube-dl is packed as an executable zipfile, simply unzip it (might need renaming to `youtube-dl.zip` first on some systems) or clone the git repository, as laid out above. If you modify the code, you can run it by executing the `__main__.py` file. To recompile the executable, run `make youtube-dl`.
|
||||
Since June 2012 ([#342](https://github.com/ytdl-org/youtube-dl/issues/342)) youtube-dl is packed as an executable zipfile, simply unzip it (might need renaming to `youtube-dl.zip` first on some systems) or clone the git repository, as laid out above. If you modify the code, you can run it by executing the `__main__.py` file. To recompile the executable, run `make youtube-dl`.
|
||||
|
||||
### The exe throws an error due to missing `MSVCR100.dll`
|
||||
|
||||
@ -904,7 +904,7 @@ When youtube-dl detects an HLS video, it can download it either with the built-i
|
||||
|
||||
When youtube-dl knows that one particular downloader works better for a given website, that downloader will be picked. Otherwise, youtube-dl will pick the best downloader for general compatibility, which at the moment happens to be ffmpeg. This choice may change in future versions of youtube-dl, with improvements of the built-in downloader and/or ffmpeg.
|
||||
|
||||
In particular, the generic extractor (used when your website is not in the [list of supported sites by youtube-dl](https://rg3.github.io/youtube-dl/supportedsites.html) cannot mandate one specific downloader.
|
||||
In particular, the generic extractor (used when your website is not in the [list of supported sites by youtube-dl](https://ytdl-org.github.io/youtube-dl/supportedsites.html) cannot mandate one specific downloader.
|
||||
|
||||
If you put either `--hls-prefer-native` or `--hls-prefer-ffmpeg` into your configuration, a different subset of videos will fail to download correctly. Instead, it is much better to [file an issue](https://yt-dl.org/bug) or a pull request which details why the native or the ffmpeg HLS downloader is a better choice for your use case.
|
||||
|
||||
@ -944,7 +944,7 @@ youtube-dl is an open-source project manned by too few volunteers, so we'd rathe
|
||||
|
||||
# DEVELOPER INSTRUCTIONS
|
||||
|
||||
Most users do not need to build youtube-dl and can [download the builds](https://rg3.github.io/youtube-dl/download.html) or get them from their distribution.
|
||||
Most users do not need to build youtube-dl and can [download the builds](https://ytdl-org.github.io/youtube-dl/download.html) or get them from their distribution.
|
||||
|
||||
To run youtube-dl as a developer, you don't need to build anything either. Simply execute
|
||||
|
||||
@ -972,7 +972,7 @@ If you want to add support for a new site, first of all **make sure** this site
|
||||
|
||||
After you have ensured this site is distributing its content legally, you can follow this quick list (assuming your service is called `yourextractor`):
|
||||
|
||||
1. [Fork this repository](https://github.com/rg3/youtube-dl/fork)
|
||||
1. [Fork this repository](https://github.com/ytdl-org/youtube-dl/fork)
|
||||
2. Check out the source code with:
|
||||
|
||||
git clone git@github.com:YOUR_GITHUB_USERNAME/youtube-dl.git
|
||||
@ -1024,9 +1024,9 @@ After you have ensured this site is distributing its content legally, you can fo
|
||||
# TODO more properties (see youtube_dl/extractor/common.py)
|
||||
}
|
||||
```
|
||||
5. Add an import in [`youtube_dl/extractor/extractors.py`](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/extractors.py).
|
||||
5. Add an import in [`youtube_dl/extractor/extractors.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/extractor/extractors.py).
|
||||
6. Run `python test/test_download.py TestDownload.test_YourExtractor`. This *should fail* at first, but you can continually re-run it until you're done. If you decide to add more than one test, then rename ``_TEST`` to ``_TESTS`` and make it into a list of dictionaries. The tests will then be named `TestDownload.test_YourExtractor`, `TestDownload.test_YourExtractor_1`, `TestDownload.test_YourExtractor_2`, etc. Note that tests with `only_matching` key in test's dict are not counted in.
|
||||
7. Have a look at [`youtube_dl/extractor/common.py`](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py) for possible helper methods and a [detailed description of what your extractor should and may return](https://github.com/rg3/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303). Add tests and code for as many as you want.
|
||||
7. Have a look at [`youtube_dl/extractor/common.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/extractor/common.py) for possible helper methods and a [detailed description of what your extractor should and may return](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303). Add tests and code for as many as you want.
|
||||
8. Make sure your code follows [youtube-dl coding conventions](#youtube-dl-coding-conventions) and check the code with [flake8](http://flake8.pycqa.org/en/latest/index.html#quickstart):
|
||||
|
||||
$ flake8 youtube_dl/extractor/yourextractor.py
|
||||
@ -1051,7 +1051,7 @@ Extractors are very fragile by nature since they depend on the layout of the sou
|
||||
|
||||
### Mandatory and optional metafields
|
||||
|
||||
For extraction to work youtube-dl relies on metadata your extractor extracts and provides to youtube-dl expressed by an [information dictionary](https://github.com/rg3/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303) or simply *info dict*. Only the following meta fields in the *info dict* are considered mandatory for a successful extraction process by youtube-dl:
|
||||
For extraction to work youtube-dl relies on metadata your extractor extracts and provides to youtube-dl expressed by an [information dictionary](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303) or simply *info dict*. Only the following meta fields in the *info dict* are considered mandatory for a successful extraction process by youtube-dl:
|
||||
|
||||
- `id` (media identifier)
|
||||
- `title` (media title)
|
||||
@ -1059,7 +1059,7 @@ For extraction to work youtube-dl relies on metadata your extractor extracts and
|
||||
|
||||
In fact only the last option is technically mandatory (i.e. if you can't figure out the download location of the media the extraction does not make any sense). But by convention youtube-dl also treats `id` and `title` as mandatory. Thus the aforementioned metafields are the critical data that the extraction does not make any sense without and if any of them fail to be extracted then the extractor is considered completely broken.
|
||||
|
||||
[Any field](https://github.com/rg3/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L188-L303) apart from the aforementioned ones are considered **optional**. That means that extraction should be **tolerant** to situations when sources for these fields can potentially be unavailable (even if they are always available at the moment) and **future-proof** in order not to break the extraction of general purpose mandatory fields.
|
||||
[Any field](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L188-L303) apart from the aforementioned ones are considered **optional**. That means that extraction should be **tolerant** to situations when sources for these fields can potentially be unavailable (even if they are always available at the moment) and **future-proof** in order not to break the extraction of general purpose mandatory fields.
|
||||
|
||||
#### Example
|
||||
|
||||
@ -1215,7 +1215,7 @@ Incorrect:
|
||||
|
||||
### Use convenience conversion and parsing functions
|
||||
|
||||
Wrap all extracted numeric data into safe functions from [`youtube_dl/utils.py`](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/utils.py): `int_or_none`, `float_or_none`. Use them for string to number conversions as well.
|
||||
Wrap all extracted numeric data into safe functions from [`youtube_dl/utils.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/utils.py): `int_or_none`, `float_or_none`. Use them for string to number conversions as well.
|
||||
|
||||
Use `url_or_none` for safe URL processing.
|
||||
|
||||
@ -1223,7 +1223,7 @@ Use `try_get` for safe metadata extraction from parsed JSON.
|
||||
|
||||
Use `unified_strdate` for uniform `upload_date` or any `YYYYMMDD` meta field extraction, `unified_timestamp` for uniform `timestamp` extraction, `parse_filesize` for `filesize` extraction, `parse_count` for count meta fields extraction, `parse_resolution`, `parse_duration` for `duration` extraction, `parse_age_limit` for `age_limit` extraction.
|
||||
|
||||
Explore [`youtube_dl/utils.py`](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/utils.py) for more useful convenience functions.
|
||||
Explore [`youtube_dl/utils.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/utils.py) for more useful convenience functions.
|
||||
|
||||
#### More examples
|
||||
|
||||
@ -1242,7 +1242,7 @@ view_count = int_or_none(video.get('views'))
|
||||
|
||||
# EMBEDDING YOUTUBE-DL
|
||||
|
||||
youtube-dl makes the best effort to be a good command-line program, and thus should be callable from any programming language. If you encounter any problems parsing its output, feel free to [create a report](https://github.com/rg3/youtube-dl/issues/new).
|
||||
youtube-dl makes the best effort to be a good command-line program, and thus should be callable from any programming language. If you encounter any problems parsing its output, feel free to [create a report](https://github.com/ytdl-org/youtube-dl/issues/new).
|
||||
|
||||
From a Python program, you can embed youtube-dl in a more powerful fashion, like this:
|
||||
|
||||
@ -1255,7 +1255,7 @@ with youtube_dl.YoutubeDL(ydl_opts) as ydl:
|
||||
ydl.download(['https://www.youtube.com/watch?v=BaW_jenozKc'])
|
||||
```
|
||||
|
||||
Most likely, you'll want to use various options. For a list of options available, have a look at [`youtube_dl/YoutubeDL.py`](https://github.com/rg3/youtube-dl/blob/3e4cedf9e8cd3157df2457df7274d0c842421945/youtube_dl/YoutubeDL.py#L137-L312). For a start, if you want to intercept youtube-dl's output, set a `logger` object.
|
||||
Most likely, you'll want to use various options. For a list of options available, have a look at [`youtube_dl/YoutubeDL.py`](https://github.com/ytdl-org/youtube-dl/blob/3e4cedf9e8cd3157df2457df7274d0c842421945/youtube_dl/YoutubeDL.py#L137-L312). For a start, if you want to intercept youtube-dl's output, set a `logger` object.
|
||||
|
||||
Here's a more complete example of a program that outputs only errors (and a short message after the download is finished), and downloads/converts the video to an mp3 file:
|
||||
|
||||
@ -1296,7 +1296,7 @@ with youtube_dl.YoutubeDL(ydl_opts) as ydl:
|
||||
|
||||
# BUGS
|
||||
|
||||
Bugs and suggestions should be reported at: <https://github.com/rg3/youtube-dl/issues>. Unless you were prompted to or there is another pertinent reason (e.g. GitHub fails to accept the bug report), please do not send bug reports via personal email. For discussions, join us in the IRC channel [#youtube-dl](irc://chat.freenode.net/#youtube-dl) on freenode ([webchat](https://webchat.freenode.net/?randomnick=1&channels=youtube-dl)).
|
||||
Bugs and suggestions should be reported at: <https://github.com/ytdl-org/youtube-dl/issues>. Unless you were prompted to or there is another pertinent reason (e.g. GitHub fails to accept the bug report), please do not send bug reports via personal email. For discussions, join us in the IRC channel [#youtube-dl](irc://chat.freenode.net/#youtube-dl) on freenode ([webchat](https://webchat.freenode.net/?randomnick=1&channels=youtube-dl)).
|
||||
|
||||
**Please include the full output of youtube-dl when run with `-v`**, i.e. **add** `-v` flag to **your command line**, copy the **whole** output and post it in the issue body wrapped in \`\`\` for better formatting. It should look similar to this:
|
||||
```
|
||||
@ -1342,11 +1342,11 @@ Before reporting any issue, type `youtube-dl -U`. This should report that you're
|
||||
|
||||
### Is the issue already documented?
|
||||
|
||||
Make sure that someone has not already opened the issue you're trying to open. Search at the top of the window or browse the [GitHub Issues](https://github.com/rg3/youtube-dl/search?type=Issues) of this repository. If there is an issue, feel free to write something along the lines of "This affects me as well, with version 2015.01.01. Here is some more information on the issue: ...". While some issues may be old, a new post into them often spurs rapid activity.
|
||||
Make sure that someone has not already opened the issue you're trying to open. Search at the top of the window or browse the [GitHub Issues](https://github.com/ytdl-org/youtube-dl/search?type=Issues) of this repository. If there is an issue, feel free to write something along the lines of "This affects me as well, with version 2015.01.01. Here is some more information on the issue: ...". While some issues may be old, a new post into them often spurs rapid activity.
|
||||
|
||||
### Why are existing options not enough?
|
||||
|
||||
Before requesting a new feature, please have a quick peek at [the list of supported options](https://github.com/rg3/youtube-dl/blob/master/README.md#options). Many feature requests are for features that actually exist already! Please, absolutely do show off your work in the issue report and detail how the existing similar options do *not* solve your problem.
|
||||
Before requesting a new feature, please have a quick peek at [the list of supported options](https://github.com/ytdl-org/youtube-dl/blob/master/README.md#options). Many feature requests are for features that actually exist already! Please, absolutely do show off your work in the issue report and detail how the existing similar options do *not* solve your problem.
|
||||
|
||||
### Is there enough context in your bug report?
|
||||
|
||||
|
||||
@ -322,7 +322,7 @@ class GITBuilder(GITInfoBuilder):
|
||||
|
||||
|
||||
class YoutubeDLBuilder(object):
|
||||
authorizedUsers = ['fraca7', 'phihag', 'rg3', 'FiloSottile']
|
||||
authorizedUsers = ['fraca7', 'phihag', 'rg3', 'FiloSottile', 'ytdl-org']
|
||||
|
||||
def __init__(self, **kwargs):
|
||||
if self.repoName != 'youtube-dl':
|
||||
|
||||
@ -27,8 +27,8 @@ from youtube_dl.utils import (
|
||||
|
||||
|
||||
class GitHubReleaser(object):
|
||||
_API_URL = 'https://api.github.com/repos/rg3/youtube-dl/releases'
|
||||
_UPLOADS_URL = 'https://uploads.github.com/repos/rg3/youtube-dl/releases/%s/assets?name=%s'
|
||||
_API_URL = 'https://api.github.com/repos/ytdl-org/youtube-dl/releases'
|
||||
_UPLOADS_URL = 'https://uploads.github.com/repos/ytdl-org/youtube-dl/releases/%s/assets?name=%s'
|
||||
_NETRC_MACHINE = 'github.com'
|
||||
|
||||
def __init__(self, debuglevel=0):
|
||||
|
||||
@ -10,7 +10,7 @@ import textwrap
|
||||
atom_template = textwrap.dedent("""\
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<feed xmlns="http://www.w3.org/2005/Atom">
|
||||
<link rel="self" href="http://rg3.github.io/youtube-dl/update/releases.atom" />
|
||||
<link rel="self" href="http://ytdl-org.github.io/youtube-dl/update/releases.atom" />
|
||||
<title>youtube-dl releases</title>
|
||||
<id>https://yt-dl.org/feed/youtube-dl-updates-feed</id>
|
||||
<updated>@TIMESTAMP@</updated>
|
||||
@ -21,7 +21,7 @@ entry_template = textwrap.dedent("""
|
||||
<entry>
|
||||
<id>https://yt-dl.org/feed/youtube-dl-updates-feed/youtube-dl-@VERSION@</id>
|
||||
<title>New version @VERSION@</title>
|
||||
<link href="http://rg3.github.io/youtube-dl" />
|
||||
<link href="http://ytdl-org.github.io/youtube-dl" />
|
||||
<content type="xhtml">
|
||||
<div xmlns="http://www.w3.org/1999/xhtml">
|
||||
Downloads available at <a href="https://yt-dl.org/downloads/@VERSION@/">https://yt-dl.org/downloads/@VERSION@/</a>
|
||||
|
||||
@ -96,7 +96,7 @@ git push origin "$version"
|
||||
REV=$(git rev-parse HEAD)
|
||||
make youtube-dl youtube-dl.tar.gz
|
||||
read -p "VM running? (y/n) " -n 1
|
||||
wget "http://$buildserver/build/rg3/youtube-dl/youtube-dl.exe?rev=$REV" -O youtube-dl.exe
|
||||
wget "http://$buildserver/build/ytdl-org/youtube-dl/youtube-dl.exe?rev=$REV" -O youtube-dl.exe
|
||||
mkdir -p "build/$version"
|
||||
mv youtube-dl youtube-dl.exe "build/$version"
|
||||
mv youtube-dl.tar.gz "build/$version/youtube-dl-$version.tar.gz"
|
||||
|
||||
@ -24,7 +24,7 @@ total_bytes = 0
|
||||
|
||||
for page in itertools.count(1):
|
||||
releases = json.loads(compat_urllib_request.urlopen(
|
||||
'https://api.github.com/repos/rg3/youtube-dl/releases?page=%s' % page
|
||||
'https://api.github.com/repos/ytdl-org/youtube-dl/releases?page=%s' % page
|
||||
).read().decode('utf-8'))
|
||||
|
||||
if not releases:
|
||||
|
||||
2
setup.py
2
setup.py
@ -104,7 +104,7 @@ setup(
|
||||
version=__version__,
|
||||
description=DESCRIPTION,
|
||||
long_description=LONG_DESCRIPTION,
|
||||
url='https://github.com/rg3/youtube-dl',
|
||||
url='https://github.com/ytdl-org/youtube-dl',
|
||||
author='Ricardo Garcia',
|
||||
author_email='ytdl@yt-dl.org',
|
||||
maintainer='Sergey M.',
|
||||
|
||||
@ -107,6 +107,184 @@ class TestInfoExtractor(unittest.TestCase):
|
||||
self.assertRaises(ExtractorError, self.ie._download_json, uri, None)
|
||||
self.assertEqual(self.ie._download_json(uri, None, fatal=False), None)
|
||||
|
||||
def test_parse_html5_media_entries(self):
|
||||
# from https://www.r18.com/
|
||||
# with kpbs in label
|
||||
expect_dict(
|
||||
self,
|
||||
self.ie._parse_html5_media_entries(
|
||||
'https://www.r18.com/',
|
||||
r'''
|
||||
<video id="samplevideo_amateur" class="js-samplevideo video-js vjs-default-skin vjs-big-play-centered" controls preload="auto" width="400" height="225" poster="//pics.r18.com/digital/amateur/mgmr105/mgmr105jp.jpg">
|
||||
<source id="video_source" src="https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_sm_w.mp4" type="video/mp4" res="240" label="300kbps">
|
||||
<source id="video_source" src="https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_dm_w.mp4" type="video/mp4" res="480" label="1000kbps">
|
||||
<source id="video_source" src="https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_dmb_w.mp4" type="video/mp4" res="740" label="1500kbps">
|
||||
<p>Your browser does not support the video tag.</p>
|
||||
</video>
|
||||
''', None)[0],
|
||||
{
|
||||
'formats': [{
|
||||
'url': 'https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_sm_w.mp4',
|
||||
'ext': 'mp4',
|
||||
'format_id': '300kbps',
|
||||
'height': 240,
|
||||
'tbr': 300,
|
||||
}, {
|
||||
'url': 'https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_dm_w.mp4',
|
||||
'ext': 'mp4',
|
||||
'format_id': '1000kbps',
|
||||
'height': 480,
|
||||
'tbr': 1000,
|
||||
}, {
|
||||
'url': 'https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_dmb_w.mp4',
|
||||
'ext': 'mp4',
|
||||
'format_id': '1500kbps',
|
||||
'height': 740,
|
||||
'tbr': 1500,
|
||||
}],
|
||||
'thumbnail': '//pics.r18.com/digital/amateur/mgmr105/mgmr105jp.jpg'
|
||||
})
|
||||
|
||||
# from https://www.csfd.cz/
|
||||
# with width and height
|
||||
expect_dict(
|
||||
self,
|
||||
self.ie._parse_html5_media_entries(
|
||||
'https://www.csfd.cz/',
|
||||
r'''
|
||||
<video width="770" height="328" preload="none" controls poster="https://img.csfd.cz/files/images/film/video/preview/163/344/163344118_748d20.png?h360" >
|
||||
<source src="https://video.csfd.cz/files/videos/157/750/157750813/163327358_eac647.mp4" type="video/mp4" width="640" height="360">
|
||||
<source src="https://video.csfd.cz/files/videos/157/750/157750813/163327360_3d2646.mp4" type="video/mp4" width="1280" height="720">
|
||||
<source src="https://video.csfd.cz/files/videos/157/750/157750813/163327356_91f258.mp4" type="video/mp4" width="1920" height="1080">
|
||||
<source src="https://video.csfd.cz/files/videos/157/750/157750813/163327359_962b4a.webm" type="video/webm" width="640" height="360">
|
||||
<source src="https://video.csfd.cz/files/videos/157/750/157750813/163327361_6feee0.webm" type="video/webm" width="1280" height="720">
|
||||
<source src="https://video.csfd.cz/files/videos/157/750/157750813/163327357_8ab472.webm" type="video/webm" width="1920" height="1080">
|
||||
<track src="https://video.csfd.cz/files/subtitles/163/344/163344115_4c388b.srt" type="text/x-srt" kind="subtitles" srclang="cs" label="cs">
|
||||
</video>
|
||||
''', None)[0],
|
||||
{
|
||||
'formats': [{
|
||||
'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327358_eac647.mp4',
|
||||
'ext': 'mp4',
|
||||
'width': 640,
|
||||
'height': 360,
|
||||
}, {
|
||||
'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327360_3d2646.mp4',
|
||||
'ext': 'mp4',
|
||||
'width': 1280,
|
||||
'height': 720,
|
||||
}, {
|
||||
'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327356_91f258.mp4',
|
||||
'ext': 'mp4',
|
||||
'width': 1920,
|
||||
'height': 1080,
|
||||
}, {
|
||||
'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327359_962b4a.webm',
|
||||
'ext': 'webm',
|
||||
'width': 640,
|
||||
'height': 360,
|
||||
}, {
|
||||
'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327361_6feee0.webm',
|
||||
'ext': 'webm',
|
||||
'width': 1280,
|
||||
'height': 720,
|
||||
}, {
|
||||
'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327357_8ab472.webm',
|
||||
'ext': 'webm',
|
||||
'width': 1920,
|
||||
'height': 1080,
|
||||
}],
|
||||
'subtitles': {
|
||||
'cs': [{'url': 'https://video.csfd.cz/files/subtitles/163/344/163344115_4c388b.srt'}]
|
||||
},
|
||||
'thumbnail': 'https://img.csfd.cz/files/images/film/video/preview/163/344/163344118_748d20.png?h360'
|
||||
})
|
||||
|
||||
# from https://tamasha.com/v/Kkdjw
|
||||
# with height in label
|
||||
expect_dict(
|
||||
self,
|
||||
self.ie._parse_html5_media_entries(
|
||||
'https://tamasha.com/v/Kkdjw',
|
||||
r'''
|
||||
<video crossorigin="anonymous">
|
||||
<source src="https://s-v2.tamasha.com/statics/videos_file/19/8f/Kkdjw_198feff8577d0057536e905cce1fb61438dd64e0_n_240.mp4" type="video/mp4" label="AUTO" res="0"/>
|
||||
<source src="https://s-v2.tamasha.com/statics/videos_file/19/8f/Kkdjw_198feff8577d0057536e905cce1fb61438dd64e0_n_240.mp4" type="video/mp4"
|
||||
label="240p" res="240"/>
|
||||
<source src="https://s-v2.tamasha.com/statics/videos_file/20/00/Kkdjw_200041c66f657fc967db464d156eafbc1ed9fe6f_n_144.mp4" type="video/mp4"
|
||||
label="144p" res="144"/>
|
||||
</video>
|
||||
''', None)[0],
|
||||
{
|
||||
'formats': [{
|
||||
'url': 'https://s-v2.tamasha.com/statics/videos_file/19/8f/Kkdjw_198feff8577d0057536e905cce1fb61438dd64e0_n_240.mp4',
|
||||
}, {
|
||||
'url': 'https://s-v2.tamasha.com/statics/videos_file/19/8f/Kkdjw_198feff8577d0057536e905cce1fb61438dd64e0_n_240.mp4',
|
||||
'ext': 'mp4',
|
||||
'format_id': '240p',
|
||||
'height': 240,
|
||||
}, {
|
||||
'url': 'https://s-v2.tamasha.com/statics/videos_file/20/00/Kkdjw_200041c66f657fc967db464d156eafbc1ed9fe6f_n_144.mp4',
|
||||
'ext': 'mp4',
|
||||
'format_id': '144p',
|
||||
'height': 144,
|
||||
}]
|
||||
})
|
||||
|
||||
# from https://www.directvnow.com
|
||||
# with data-src
|
||||
expect_dict(
|
||||
self,
|
||||
self.ie._parse_html5_media_entries(
|
||||
'https://www.directvnow.com',
|
||||
r'''
|
||||
<video id="vid1" class="header--video-masked active" muted playsinline>
|
||||
<source data-src="https://cdn.directv.com/content/dam/dtv/prod/website_directvnow-international/videos/DTVN_hdr_HBO_v3.mp4" type="video/mp4" />
|
||||
</video>
|
||||
''', None)[0],
|
||||
{
|
||||
'formats': [{
|
||||
'ext': 'mp4',
|
||||
'url': 'https://cdn.directv.com/content/dam/dtv/prod/website_directvnow-international/videos/DTVN_hdr_HBO_v3.mp4',
|
||||
}]
|
||||
})
|
||||
|
||||
# from https://www.directvnow.com
|
||||
# with data-src
|
||||
expect_dict(
|
||||
self,
|
||||
self.ie._parse_html5_media_entries(
|
||||
'https://www.directvnow.com',
|
||||
r'''
|
||||
<video id="vid1" class="header--video-masked active" muted playsinline>
|
||||
<source data-src="https://cdn.directv.com/content/dam/dtv/prod/website_directvnow-international/videos/DTVN_hdr_HBO_v3.mp4" type="video/mp4" />
|
||||
</video>
|
||||
''', None)[0],
|
||||
{
|
||||
'formats': [{
|
||||
'url': 'https://cdn.directv.com/content/dam/dtv/prod/website_directvnow-international/videos/DTVN_hdr_HBO_v3.mp4',
|
||||
'ext': 'mp4',
|
||||
}]
|
||||
})
|
||||
|
||||
# from https://www.klarna.com/uk/
|
||||
# with data-video-src
|
||||
expect_dict(
|
||||
self,
|
||||
self.ie._parse_html5_media_entries(
|
||||
'https://www.directvnow.com',
|
||||
r'''
|
||||
<video loop autoplay muted class="responsive-video block-kl__video video-on-medium">
|
||||
<source src="" data-video-desktop data-video-src="https://www.klarna.com/uk/wp-content/uploads/sites/11/2019/01/KL062_Smooth3_0_DogWalking_5s_920x080_.mp4" type="video/mp4" />
|
||||
</video>
|
||||
''', None)[0],
|
||||
{
|
||||
'formats': [{
|
||||
'url': 'https://www.klarna.com/uk/wp-content/uploads/sites/11/2019/01/KL062_Smooth3_0_DogWalking_5s_920x080_.mp4',
|
||||
'ext': 'mp4',
|
||||
}],
|
||||
})
|
||||
|
||||
def test_extract_jwplayer_data_realworld(self):
|
||||
# from http://www.suffolk.edu/sjc/
|
||||
expect_dict(
|
||||
@ -201,7 +379,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
|
||||
def test_parse_m3u8_formats(self):
|
||||
_TEST_CASES = [
|
||||
(
|
||||
# https://github.com/rg3/youtube-dl/issues/11507
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/11507
|
||||
# http://pluzz.francetv.fr/videos/le_ministere.html
|
||||
'pluzz_francetv_11507',
|
||||
'http://replayftv-vh.akamaihd.net/i/streaming-adaptatif_france-dom-tom/2017/S16/J2/156589847-58f59130c1f52-,standard1,standard2,standard3,standard4,standard5,.mp4.csmil/master.m3u8?caption=2017%2F16%2F156589847-1492488987.m3u8%3Afra%3AFrancais&audiotrack=0%3Afra%3AFrancais',
|
||||
@ -263,7 +441,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
|
||||
}]
|
||||
),
|
||||
(
|
||||
# https://github.com/rg3/youtube-dl/issues/11995
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/11995
|
||||
# http://teamcoco.com/video/clueless-gamer-super-bowl-for-honor
|
||||
'teamcoco_11995',
|
||||
'http://ak.storage-w.teamcococdn.com/cdn/2017-02/98599/ed8f/main.m3u8',
|
||||
@ -337,7 +515,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
|
||||
}]
|
||||
),
|
||||
(
|
||||
# https://github.com/rg3/youtube-dl/issues/12211
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/12211
|
||||
# http://video.toggle.sg/en/series/whoopie-s-world/ep3/478601
|
||||
'toggle_mobile_12211',
|
||||
'http://cdnapi.kaltura.com/p/2082311/sp/208231100/playManifest/protocol/http/entryId/0_89q6e8ku/format/applehttp/tags/mobile_sd/f/a.m3u8',
|
||||
@ -501,7 +679,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
|
||||
}]
|
||||
),
|
||||
(
|
||||
# https://github.com/rg3/youtube-dl/issues/18923
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/18923
|
||||
# https://www.ted.com/talks/boris_hesser_a_grassroots_healthcare_revolution_in_africa
|
||||
'ted_18923',
|
||||
'http://hls.ted.com/talks/31241.m3u8',
|
||||
@ -570,11 +748,12 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
|
||||
def test_parse_mpd_formats(self):
|
||||
_TEST_CASES = [
|
||||
(
|
||||
# https://github.com/rg3/youtube-dl/issues/13919
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/13919
|
||||
# Also tests duplicate representation ids, see
|
||||
# https://github.com/rg3/youtube-dl/issues/15111
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/15111
|
||||
'float_duration',
|
||||
'http://unknown/manifest.mpd',
|
||||
'http://unknown/manifest.mpd', # mpd_url
|
||||
None, # mpd_base_url
|
||||
[{
|
||||
'manifest_url': 'http://unknown/manifest.mpd',
|
||||
'ext': 'm4a',
|
||||
@ -652,9 +831,10 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
|
||||
'height': 1080,
|
||||
}]
|
||||
), (
|
||||
# https://github.com/rg3/youtube-dl/pull/14844
|
||||
# https://github.com/ytdl-org/youtube-dl/pull/14844
|
||||
'urls_only',
|
||||
'http://unknown/manifest.mpd',
|
||||
'http://unknown/manifest.mpd', # mpd_url
|
||||
None, # mpd_base_url
|
||||
[{
|
||||
'manifest_url': 'http://unknown/manifest.mpd',
|
||||
'ext': 'mp4',
|
||||
@ -733,22 +913,68 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
|
||||
'width': 1920,
|
||||
'height': 1080,
|
||||
}]
|
||||
), (
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/20346
|
||||
# Media considered unfragmented even though it contains
|
||||
# Initialization tag
|
||||
'unfragmented',
|
||||
'https://v.redd.it/hw1x7rcg7zl21/DASHPlaylist.mpd', # mpd_url
|
||||
'https://v.redd.it/hw1x7rcg7zl21', # mpd_base_url
|
||||
[{
|
||||
'url': 'https://v.redd.it/hw1x7rcg7zl21/audio',
|
||||
'manifest_url': 'https://v.redd.it/hw1x7rcg7zl21/DASHPlaylist.mpd',
|
||||
'ext': 'm4a',
|
||||
'format_id': 'AUDIO-1',
|
||||
'format_note': 'DASH audio',
|
||||
'container': 'm4a_dash',
|
||||
'acodec': 'mp4a.40.2',
|
||||
'vcodec': 'none',
|
||||
'tbr': 129.87,
|
||||
'asr': 48000,
|
||||
|
||||
}, {
|
||||
'url': 'https://v.redd.it/hw1x7rcg7zl21/DASH_240',
|
||||
'manifest_url': 'https://v.redd.it/hw1x7rcg7zl21/DASHPlaylist.mpd',
|
||||
'ext': 'mp4',
|
||||
'format_id': 'VIDEO-2',
|
||||
'format_note': 'DASH video',
|
||||
'container': 'mp4_dash',
|
||||
'acodec': 'none',
|
||||
'vcodec': 'avc1.4d401e',
|
||||
'tbr': 608.0,
|
||||
'width': 240,
|
||||
'height': 240,
|
||||
'fps': 30,
|
||||
}, {
|
||||
'url': 'https://v.redd.it/hw1x7rcg7zl21/DASH_360',
|
||||
'manifest_url': 'https://v.redd.it/hw1x7rcg7zl21/DASHPlaylist.mpd',
|
||||
'ext': 'mp4',
|
||||
'format_id': 'VIDEO-1',
|
||||
'format_note': 'DASH video',
|
||||
'container': 'mp4_dash',
|
||||
'acodec': 'none',
|
||||
'vcodec': 'avc1.4d401e',
|
||||
'tbr': 804.261,
|
||||
'width': 360,
|
||||
'height': 360,
|
||||
'fps': 30,
|
||||
}]
|
||||
)
|
||||
]
|
||||
|
||||
for mpd_file, mpd_url, expected_formats in _TEST_CASES:
|
||||
for mpd_file, mpd_url, mpd_base_url, expected_formats in _TEST_CASES:
|
||||
with io.open('./test/testdata/mpd/%s.mpd' % mpd_file,
|
||||
mode='r', encoding='utf-8') as f:
|
||||
formats = self.ie._parse_mpd_formats(
|
||||
compat_etree_fromstring(f.read().encode('utf-8')),
|
||||
mpd_url=mpd_url)
|
||||
mpd_base_url=mpd_base_url, mpd_url=mpd_url)
|
||||
self.ie._sort_formats(formats)
|
||||
expect_value(self, formats, expected_formats, None)
|
||||
|
||||
def test_parse_f4m_formats(self):
|
||||
_TEST_CASES = [
|
||||
(
|
||||
# https://github.com/rg3/youtube-dl/issues/14660
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/14660
|
||||
'custom_base_url',
|
||||
'http://api.new.livestream.com/accounts/6115179/events/6764928/videos/144884262.f4m',
|
||||
[{
|
||||
|
||||
@ -411,7 +411,7 @@ class TestFormatSelection(unittest.TestCase):
|
||||
# For extractors with incomplete formats (all formats are audio-only or
|
||||
# video-only) best and worst should fallback to corresponding best/worst
|
||||
# video-only or audio-only formats (as per
|
||||
# https://github.com/rg3/youtube-dl/pull/5556)
|
||||
# https://github.com/ytdl-org/youtube-dl/pull/5556)
|
||||
formats = [
|
||||
{'format_id': 'low', 'ext': 'mp3', 'preference': 1, 'vcodec': 'none', 'url': TEST_URL},
|
||||
{'format_id': 'high', 'ext': 'mp3', 'preference': 2, 'vcodec': 'none', 'url': TEST_URL},
|
||||
@ -442,7 +442,7 @@ class TestFormatSelection(unittest.TestCase):
|
||||
self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())
|
||||
|
||||
def test_format_selection_issue_10083(self):
|
||||
# See https://github.com/rg3/youtube-dl/issues/10083
|
||||
# See https://github.com/ytdl-org/youtube-dl/issues/10083
|
||||
formats = [
|
||||
{'format_id': 'regular', 'height': 360, 'url': TEST_URL},
|
||||
{'format_id': 'video', 'height': 720, 'acodec': 'none', 'url': TEST_URL},
|
||||
@ -853,7 +853,7 @@ class TestYoutubeDL(unittest.TestCase):
|
||||
self.assertEqual(result, [2, 3, 4])
|
||||
|
||||
def test_urlopen_no_file_protocol(self):
|
||||
# see https://github.com/rg3/youtube-dl/issues/8227
|
||||
# see https://github.com/ytdl-org/youtube-dl/issues/8227
|
||||
ydl = YDL()
|
||||
self.assertRaises(compat_urllib_error.URLError, ydl.urlopen, 'file:///etc/passwd')
|
||||
|
||||
|
||||
@ -110,7 +110,7 @@ class TestAllURLsMatching(unittest.TestCase):
|
||||
self.assertMatch('https://vimeo.com/user7108434/videos', ['vimeo:user'])
|
||||
self.assertMatch('https://vimeo.com/user21297594/review/75524534/3c257a1b5d', ['vimeo:review'])
|
||||
|
||||
# https://github.com/rg3/youtube-dl/issues/1930
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/1930
|
||||
def test_soundcloud_not_matching_sets(self):
|
||||
self.assertMatch('http://soundcloud.com/floex/sets/gone-ep', ['soundcloud:set'])
|
||||
|
||||
@ -119,12 +119,12 @@ class TestAllURLsMatching(unittest.TestCase):
|
||||
self.assertMatch('http://tatianamaslanydaily.tumblr.com/post/54196191430', ['Tumblr'])
|
||||
|
||||
def test_pbs(self):
|
||||
# https://github.com/rg3/youtube-dl/issues/2350
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/2350
|
||||
self.assertMatch('http://video.pbs.org/viralplayer/2365173446/', ['pbs'])
|
||||
self.assertMatch('http://video.pbs.org/widget/partnerplayer/980042464/', ['pbs'])
|
||||
|
||||
def test_yahoo_https(self):
|
||||
# https://github.com/rg3/youtube-dl/issues/2701
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/2701
|
||||
self.assertMatch(
|
||||
'https://screen.yahoo.com/smartwatches-latest-wearable-gadgets-163745379-cbs.html',
|
||||
['Yahoo'])
|
||||
|
||||
@ -13,6 +13,7 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
from youtube_dl.compat import (
|
||||
compat_getenv,
|
||||
compat_setenv,
|
||||
compat_etree_Element,
|
||||
compat_etree_fromstring,
|
||||
compat_expanduser,
|
||||
compat_shlex_split,
|
||||
@ -90,6 +91,12 @@ class TestCompat(unittest.TestCase):
|
||||
self.assertEqual(compat_shlex_split('-option "one\ntwo" \n -flag'), ['-option', 'one\ntwo', '-flag'])
|
||||
self.assertEqual(compat_shlex_split('-val 中文'), ['-val', '中文'])
|
||||
|
||||
def test_compat_etree_Element(self):
|
||||
try:
|
||||
compat_etree_Element.items
|
||||
except AttributeError:
|
||||
self.fail('compat_etree_Element is not a type')
|
||||
|
||||
def test_compat_etree_fromstring(self):
|
||||
xml = '''
|
||||
<root foo="bar" spam="中文">
|
||||
|
||||
@ -55,6 +55,7 @@ from youtube_dl.utils import (
|
||||
parse_count,
|
||||
parse_iso8601,
|
||||
parse_resolution,
|
||||
parse_bitrate,
|
||||
pkcs1pad,
|
||||
read_batch_urls,
|
||||
sanitize_filename,
|
||||
@ -1030,6 +1031,13 @@ class TestUtil(unittest.TestCase):
|
||||
self.assertEqual(parse_resolution('4k'), {'height': 2160})
|
||||
self.assertEqual(parse_resolution('8K'), {'height': 4320})
|
||||
|
||||
def test_parse_bitrate(self):
|
||||
self.assertEqual(parse_bitrate(None), None)
|
||||
self.assertEqual(parse_bitrate(''), None)
|
||||
self.assertEqual(parse_bitrate('300kbps'), 300)
|
||||
self.assertEqual(parse_bitrate('1500kbps'), 1500)
|
||||
self.assertEqual(parse_bitrate('300 kbps'), 300)
|
||||
|
||||
def test_version_tuple(self):
|
||||
self.assertEqual(version_tuple('1'), (1,))
|
||||
self.assertEqual(version_tuple('10.23.344'), (10, 23, 344))
|
||||
|
||||
28
test/testdata/mpd/unfragmented.mpd
vendored
Normal file
28
test/testdata/mpd/unfragmented.mpd
vendored
Normal file
@ -0,0 +1,28 @@
|
||||
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
|
||||
<MPD mediaPresentationDuration="PT54.915S" minBufferTime="PT1.500S" profiles="urn:mpeg:dash:profile:isoff-on-demand:2011" type="static" xmlns="urn:mpeg:dash:schema:mpd:2011">
|
||||
<Period duration="PT54.915S">
|
||||
<AdaptationSet segmentAlignment="true" subsegmentAlignment="true" subsegmentStartsWithSAP="1">
|
||||
<Representation bandwidth="804261" codecs="avc1.4d401e" frameRate="30" height="360" id="VIDEO-1" mimeType="video/mp4" startWithSAP="1" width="360">
|
||||
<BaseURL>DASH_360</BaseURL>
|
||||
<SegmentBase indexRange="915-1114" indexRangeExact="true">
|
||||
<Initialization range="0-914"/>
|
||||
</SegmentBase>
|
||||
</Representation>
|
||||
<Representation bandwidth="608000" codecs="avc1.4d401e" frameRate="30" height="240" id="VIDEO-2" mimeType="video/mp4" startWithSAP="1" width="240">
|
||||
<BaseURL>DASH_240</BaseURL>
|
||||
<SegmentBase indexRange="913-1112" indexRangeExact="true">
|
||||
<Initialization range="0-912"/>
|
||||
</SegmentBase>
|
||||
</Representation>
|
||||
</AdaptationSet>
|
||||
<AdaptationSet>
|
||||
<Representation audioSamplingRate="48000" bandwidth="129870" codecs="mp4a.40.2" id="AUDIO-1" mimeType="audio/mp4" startWithSAP="1">
|
||||
<AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
|
||||
<BaseURL>audio</BaseURL>
|
||||
<SegmentBase indexRange="832-1007" indexRangeExact="true">
|
||||
<Initialization range="0-831"/>
|
||||
</SegmentBase>
|
||||
</Representation>
|
||||
</AdaptationSet>
|
||||
</Period>
|
||||
</MPD>
|
||||
@ -7,7 +7,7 @@
|
||||
# https://github.com/zsh-users/antigen
|
||||
|
||||
# Install youtube-dl:
|
||||
# antigen bundle rg3/youtube-dl
|
||||
# antigen bundle ytdl-org/youtube-dl
|
||||
# Bundles installed by antigen are available for use immediately.
|
||||
|
||||
# Update youtube-dl (and all other antigen bundles):
|
||||
|
||||
@ -889,7 +889,7 @@ class YoutubeDL(object):
|
||||
# url_transparent. In such cases outer metadata (from ie_result)
|
||||
# should be propagated to inner one (info). For this to happen
|
||||
# _type of info should be overridden with url_transparent. This
|
||||
# fixes issue from https://github.com/rg3/youtube-dl/pull/11163.
|
||||
# fixes issue from https://github.com/ytdl-org/youtube-dl/pull/11163.
|
||||
if new_result.get('_type') == 'url':
|
||||
new_result['_type'] = 'url_transparent'
|
||||
|
||||
@ -1606,7 +1606,7 @@ class YoutubeDL(object):
|
||||
# by extractor are incomplete or not (i.e. whether extractor provides only
|
||||
# video-only or audio-only formats) for proper formats selection for
|
||||
# extractors with such incomplete formats (see
|
||||
# https://github.com/rg3/youtube-dl/pull/5556).
|
||||
# https://github.com/ytdl-org/youtube-dl/pull/5556).
|
||||
# Since formats may be filtered during format selection and may not match
|
||||
# the original formats the results may be incorrect. Thus original formats
|
||||
# or pre-calculated metrics should be passed to format selection routines
|
||||
@ -1614,7 +1614,7 @@ class YoutubeDL(object):
|
||||
# We will pass a context object containing all necessary additional data
|
||||
# instead of just formats.
|
||||
# This fixes incorrect format selection issue (see
|
||||
# https://github.com/rg3/youtube-dl/issues/10083).
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/10083).
|
||||
incomplete_formats = (
|
||||
# All formats are video-only or
|
||||
all(f.get('vcodec') != 'none' and f.get('acodec') == 'none' for f in formats) or
|
||||
@ -1810,7 +1810,7 @@ class YoutubeDL(object):
|
||||
if sub_info.get('data') is not None:
|
||||
try:
|
||||
# Use newline='' to prevent conversion of newline characters
|
||||
# See https://github.com/rg3/youtube-dl/issues/10268
|
||||
# See https://github.com/ytdl-org/youtube-dl/issues/10268
|
||||
with io.open(encodeFilename(sub_filename), 'w', encoding='utf-8', newline='') as subfile:
|
||||
subfile.write(sub_info['data'])
|
||||
except (OSError, IOError):
|
||||
@ -2229,7 +2229,7 @@ class YoutubeDL(object):
|
||||
return
|
||||
|
||||
if type('') is not compat_str:
|
||||
# Python 2.6 on SLES11 SP1 (https://github.com/rg3/youtube-dl/issues/3326)
|
||||
# Python 2.6 on SLES11 SP1 (https://github.com/ytdl-org/youtube-dl/issues/3326)
|
||||
self.report_warning(
|
||||
'Your Python is broken! Update to a newer and supported version')
|
||||
|
||||
@ -2323,7 +2323,7 @@ class YoutubeDL(object):
|
||||
proxies = {'http': opts_proxy, 'https': opts_proxy}
|
||||
else:
|
||||
proxies = compat_urllib_request.getproxies()
|
||||
# Set HTTPS proxy to HTTP one if given (https://github.com/rg3/youtube-dl/issues/805)
|
||||
# Set HTTPS proxy to HTTP one if given (https://github.com/ytdl-org/youtube-dl/issues/805)
|
||||
if 'http' in proxies and 'https' not in proxies:
|
||||
proxies['https'] = proxies['http']
|
||||
proxy_handler = PerRequestProxyHandler(proxies)
|
||||
@ -2336,7 +2336,7 @@ class YoutubeDL(object):
|
||||
# When passing our own FileHandler instance, build_opener won't add the
|
||||
# default FileHandler and allows us to disable the file protocol, which
|
||||
# can be used for malicious purposes (see
|
||||
# https://github.com/rg3/youtube-dl/issues/8227)
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/8227)
|
||||
file_handler = compat_urllib_request.FileHandler()
|
||||
|
||||
def file_open(*args, **kwargs):
|
||||
@ -2348,7 +2348,7 @@ class YoutubeDL(object):
|
||||
|
||||
# Delete the default user-agent header, which would otherwise apply in
|
||||
# cases where our custom HTTP handler doesn't come into play
|
||||
# (See https://github.com/rg3/youtube-dl/issues/1309 for details)
|
||||
# (See https://github.com/ytdl-org/youtube-dl/issues/1309 for details)
|
||||
opener.addheaders = []
|
||||
self._opener = opener
|
||||
|
||||
|
||||
@ -48,7 +48,7 @@ from .YoutubeDL import YoutubeDL
|
||||
def _real_main(argv=None):
|
||||
# Compatibility fixes for Windows
|
||||
if sys.platform == 'win32':
|
||||
# https://github.com/rg3/youtube-dl/issues/820
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/820
|
||||
codecs.register(lambda name: codecs.lookup('utf-8') if name == 'cp65001' else None)
|
||||
|
||||
workaround_optparse_bug9161()
|
||||
|
||||
@ -2364,7 +2364,7 @@ except ImportError: # Python 2
|
||||
|
||||
# HACK: The following are the correct unquote_to_bytes, unquote and unquote_plus
|
||||
# implementations from cpython 3.4.3's stdlib. Python 2's version
|
||||
# is apparently broken (see https://github.com/rg3/youtube-dl/pull/6244)
|
||||
# is apparently broken (see https://github.com/ytdl-org/youtube-dl/pull/6244)
|
||||
|
||||
def compat_urllib_parse_unquote_to_bytes(string):
|
||||
"""unquote_to_bytes('abc%20def') -> b'abc def'."""
|
||||
@ -2508,6 +2508,15 @@ class _TreeBuilder(etree.TreeBuilder):
|
||||
pass
|
||||
|
||||
|
||||
try:
|
||||
# xml.etree.ElementTree.Element is a method in Python <=2.6 and
|
||||
# the following will crash with:
|
||||
# TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and types
|
||||
isinstance(None, xml.etree.ElementTree.Element)
|
||||
from xml.etree.ElementTree import Element as compat_etree_Element
|
||||
except TypeError: # Python <=2.6
|
||||
from xml.etree.ElementTree import _ElementInterface as compat_etree_Element
|
||||
|
||||
if sys.version_info[0] >= 3:
|
||||
def compat_etree_fromstring(text):
|
||||
return etree.XML(text, parser=etree.XMLParser(target=_TreeBuilder()))
|
||||
@ -2819,7 +2828,7 @@ else:
|
||||
compat_socket_create_connection = socket.create_connection
|
||||
|
||||
|
||||
# Fix https://github.com/rg3/youtube-dl/issues/4223
|
||||
# Fix https://github.com/ytdl-org/youtube-dl/issues/4223
|
||||
# See http://bugs.python.org/issue9161 for what is broken
|
||||
def workaround_optparse_bug9161():
|
||||
op = optparse.OptionParser()
|
||||
@ -2944,7 +2953,7 @@ if platform.python_implementation() == 'PyPy' and sys.pypy_version_info < (5, 4,
|
||||
# PyPy2 prior to version 5.4.0 expects byte strings as Windows function
|
||||
# names, see the original PyPy issue [1] and the youtube-dl one [2].
|
||||
# 1. https://bitbucket.org/pypy/pypy/issues/2360/windows-ctypescdll-typeerror-function-name
|
||||
# 2. https://github.com/rg3/youtube-dl/pull/4392
|
||||
# 2. https://github.com/ytdl-org/youtube-dl/pull/4392
|
||||
def compat_ctypes_WINFUNCTYPE(*args, **kwargs):
|
||||
real = ctypes.WINFUNCTYPE(*args, **kwargs)
|
||||
|
||||
@ -2969,6 +2978,7 @@ __all__ = [
|
||||
'compat_cookiejar',
|
||||
'compat_cookies',
|
||||
'compat_ctypes_WINFUNCTYPE',
|
||||
'compat_etree_Element',
|
||||
'compat_etree_fromstring',
|
||||
'compat_etree_register_namespace',
|
||||
'compat_expanduser',
|
||||
|
||||
@ -239,7 +239,7 @@ class FFmpegFD(ExternalFD):
|
||||
# setting -seekable prevents ffmpeg from guessing if the server
|
||||
# supports seeking(by adding the header `Range: bytes=0-`), which
|
||||
# can cause problems in some cases
|
||||
# https://github.com/rg3/youtube-dl/issues/11800#issuecomment-275037127
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/11800#issuecomment-275037127
|
||||
# http://trac.ffmpeg.org/ticket/6125#comment:10
|
||||
args += ['-seekable', '1' if seekable else '0']
|
||||
|
||||
@ -334,7 +334,7 @@ class FFmpegFD(ExternalFD):
|
||||
# mp4 file couldn't be played, but if we ask ffmpeg to quit it
|
||||
# produces a file that is playable (this is mostly useful for live
|
||||
# streams). Note that Windows is not affected and produces playable
|
||||
# files (see https://github.com/rg3/youtube-dl/issues/8300).
|
||||
# files (see https://github.com/ytdl-org/youtube-dl/issues/8300).
|
||||
if sys.platform != 'win32':
|
||||
proc.communicate(b'q')
|
||||
raise
|
||||
|
||||
@ -324,8 +324,8 @@ class F4mFD(FragmentFD):
|
||||
urlh = self.ydl.urlopen(self._prepare_url(info_dict, man_url))
|
||||
man_url = urlh.geturl()
|
||||
# Some manifests may be malformed, e.g. prosiebensat1 generated manifests
|
||||
# (see https://github.com/rg3/youtube-dl/issues/6215#issuecomment-121704244
|
||||
# and https://github.com/rg3/youtube-dl/issues/7823)
|
||||
# (see https://github.com/ytdl-org/youtube-dl/issues/6215#issuecomment-121704244
|
||||
# and https://github.com/ytdl-org/youtube-dl/issues/7823)
|
||||
manifest = fix_xml_ampersands(urlh.read().decode('utf-8', 'ignore')).strip()
|
||||
|
||||
doc = compat_etree_fromstring(manifest)
|
||||
@ -409,7 +409,7 @@ class F4mFD(FragmentFD):
|
||||
# In tests, segments may be truncated, and thus
|
||||
# FlvReader may not be able to parse the whole
|
||||
# chunk. If so, write the segment as is
|
||||
# See https://github.com/rg3/youtube-dl/issues/9214
|
||||
# See https://github.com/ytdl-org/youtube-dl/issues/9214
|
||||
dest_stream.write(down_data)
|
||||
break
|
||||
raise
|
||||
|
||||
@ -152,8 +152,8 @@ class HlsFD(FragmentFD):
|
||||
except compat_urllib_error.HTTPError as err:
|
||||
# Unavailable (possibly temporary) fragments may be served.
|
||||
# First we try to retry then either skip or abort.
|
||||
# See https://github.com/rg3/youtube-dl/issues/10165,
|
||||
# https://github.com/rg3/youtube-dl/issues/10448).
|
||||
# See https://github.com/ytdl-org/youtube-dl/issues/10165,
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/10448).
|
||||
count += 1
|
||||
if count <= fragment_retries:
|
||||
self.report_retry_fragment(err, frag_index, count, fragment_retries)
|
||||
|
||||
@ -111,7 +111,7 @@ class HttpFD(FileDownloader):
|
||||
# to match the value of requested Range HTTP header. This is due to a webservers
|
||||
# that don't support resuming and serve a whole file with no Content-Range
|
||||
# set in response despite of requested Range (see
|
||||
# https://github.com/rg3/youtube-dl/issues/6057#issuecomment-126129799)
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/6057#issuecomment-126129799)
|
||||
if has_range:
|
||||
content_range = ctx.data.headers.get('Content-Range')
|
||||
if content_range:
|
||||
|
||||
@ -1,30 +0,0 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from .nuevo import NuevoBaseIE
|
||||
|
||||
|
||||
class AnitubeIE(NuevoBaseIE):
|
||||
IE_NAME = 'anitube.se'
|
||||
_VALID_URL = r'https?://(?:www\.)?anitube\.se/video/(?P<id>\d+)'
|
||||
|
||||
_TEST = {
|
||||
'url': 'http://www.anitube.se/video/36621',
|
||||
'md5': '59d0eeae28ea0bc8c05e7af429998d43',
|
||||
'info_dict': {
|
||||
'id': '36621',
|
||||
'ext': 'mp4',
|
||||
'title': 'Recorder to Randoseru 01',
|
||||
'duration': 180.19,
|
||||
},
|
||||
'skip': 'Blocked in the US',
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
|
||||
webpage = self._download_webpage(url, video_id)
|
||||
key = self._search_regex(
|
||||
r'src=["\']https?://[^/]+/embed/([A-Za-z0-9_-]+)', webpage, 'key')
|
||||
|
||||
return self._extract_nuevo(
|
||||
'http://www.anitube.se/nuevo/econfig.php?key=%s' % key, video_id)
|
||||
@ -1,61 +0,0 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import re
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
parse_duration,
|
||||
int_or_none,
|
||||
)
|
||||
|
||||
|
||||
class AnySexIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?anysex\.com/(?P<id>\d+)'
|
||||
_TEST = {
|
||||
'url': 'http://anysex.com/156592/',
|
||||
'md5': '023e9fbb7f7987f5529a394c34ad3d3d',
|
||||
'info_dict': {
|
||||
'id': '156592',
|
||||
'ext': 'mp4',
|
||||
'title': 'Busty and sexy blondie in her bikini strips for you',
|
||||
'description': 'md5:de9e418178e2931c10b62966474e1383',
|
||||
'categories': ['Erotic'],
|
||||
'duration': 270,
|
||||
'age_limit': 18,
|
||||
}
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
mobj = re.match(self._VALID_URL, url)
|
||||
video_id = mobj.group('id')
|
||||
|
||||
webpage = self._download_webpage(url, video_id)
|
||||
|
||||
video_url = self._html_search_regex(r"video_url\s*:\s*'([^']+)'", webpage, 'video URL')
|
||||
|
||||
title = self._html_search_regex(r'<title>(.*?)</title>', webpage, 'title')
|
||||
description = self._html_search_regex(
|
||||
r'<div class="description"[^>]*>([^<]+)</div>', webpage, 'description', fatal=False)
|
||||
thumbnail = self._html_search_regex(
|
||||
r'preview_url\s*:\s*\'(.*?)\'', webpage, 'thumbnail', fatal=False)
|
||||
|
||||
categories = re.findall(
|
||||
r'<a href="http://anysex\.com/categories/[^"]+" title="[^"]*">([^<]+)</a>', webpage)
|
||||
|
||||
duration = parse_duration(self._search_regex(
|
||||
r'<b>Duration:</b> (?:<q itemprop="duration">)?(\d+:\d+)', webpage, 'duration', fatal=False))
|
||||
view_count = int_or_none(self._html_search_regex(
|
||||
r'<b>Views:</b> (\d+)', webpage, 'view count', fatal=False))
|
||||
|
||||
return {
|
||||
'id': video_id,
|
||||
'url': video_url,
|
||||
'ext': 'mp4',
|
||||
'title': title,
|
||||
'description': description,
|
||||
'thumbnail': thumbnail,
|
||||
'categories': categories,
|
||||
'duration': duration,
|
||||
'view_count': view_count,
|
||||
'age_limit': 18,
|
||||
}
|
||||
@ -103,7 +103,7 @@ class ArkenaIE(InfoExtractor):
|
||||
f_url, video_id, mpd_id=kind, fatal=False))
|
||||
elif kind == 'silverlight':
|
||||
# TODO: process when ism is supported (see
|
||||
# https://github.com/rg3/youtube-dl/issues/8118)
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/8118)
|
||||
continue
|
||||
else:
|
||||
tbr = float_or_none(f.get('Bitrate'), 1000)
|
||||
|
||||
@ -23,7 +23,7 @@ class BambuserIE(InfoExtractor):
|
||||
|
||||
_TEST = {
|
||||
'url': 'http://bambuser.com/v/4050584',
|
||||
# MD5 seems to be flaky, see https://travis-ci.org/rg3/youtube-dl/jobs/14051016#L388
|
||||
# MD5 seems to be flaky, see https://travis-ci.org/ytdl-org/youtube-dl/jobs/14051016#L388
|
||||
# 'md5': 'fba8f7693e48fd4e8641b3fd5539a641',
|
||||
'info_dict': {
|
||||
'id': '4050584',
|
||||
@ -38,7 +38,7 @@ class BambuserIE(InfoExtractor):
|
||||
},
|
||||
'params': {
|
||||
# It doesn't respect the 'Range' header, it would download the whole video
|
||||
# caused the travis builds to fail: https://travis-ci.org/rg3/youtube-dl/jobs/14493845#L59
|
||||
# caused the travis builds to fail: https://travis-ci.org/ytdl-org/youtube-dl/jobs/14493845#L59
|
||||
'skip_download': True,
|
||||
},
|
||||
}
|
||||
|
||||
@ -3,7 +3,6 @@ from __future__ import unicode_literals
|
||||
|
||||
import itertools
|
||||
import re
|
||||
import xml
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
@ -23,6 +22,7 @@ from ..utils import (
|
||||
urljoin,
|
||||
)
|
||||
from ..compat import (
|
||||
compat_etree_Element,
|
||||
compat_HTTPError,
|
||||
compat_urlparse,
|
||||
)
|
||||
@ -208,7 +208,7 @@ class BBCCoUkIE(InfoExtractor):
|
||||
},
|
||||
'skip': 'Now it\'s really geo-restricted',
|
||||
}, {
|
||||
# compact player (https://github.com/rg3/youtube-dl/issues/8147)
|
||||
# compact player (https://github.com/ytdl-org/youtube-dl/issues/8147)
|
||||
'url': 'http://www.bbc.co.uk/programmes/p028bfkf/player',
|
||||
'info_dict': {
|
||||
'id': 'p028bfkj',
|
||||
@ -317,7 +317,7 @@ class BBCCoUkIE(InfoExtractor):
|
||||
continue
|
||||
captions = self._download_xml(
|
||||
cc_url, programme_id, 'Downloading captions', fatal=False)
|
||||
if not isinstance(captions, xml.etree.ElementTree.Element):
|
||||
if not isinstance(captions, compat_etree_Element):
|
||||
continue
|
||||
lang = captions.get('{http://www.w3.org/XML/1998/namespace}lang', 'en')
|
||||
subtitles[lang] = [
|
||||
|
||||
@ -9,8 +9,8 @@ from ..utils import (
|
||||
|
||||
|
||||
class BeegIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?beeg\.com/(?P<id>\d+)'
|
||||
_TEST = {
|
||||
_VALID_URL = r'https?://(?:www\.)?beeg\.(?:com|porn(?:/video)?)/(?P<id>\d+)'
|
||||
_TESTS = [{
|
||||
'url': 'http://beeg.com/5416503',
|
||||
'md5': 'a1a1b1a8bc70a89e49ccfd113aed0820',
|
||||
'info_dict': {
|
||||
@ -24,7 +24,13 @@ class BeegIE(InfoExtractor):
|
||||
'tags': list,
|
||||
'age_limit': 18,
|
||||
}
|
||||
}
|
||||
}, {
|
||||
'url': 'https://beeg.porn/video/5416503',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://beeg.porn/5416503',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
|
||||
@ -2,39 +2,96 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from .common import InfoExtractor
|
||||
from .vk import VKIE
|
||||
from ..utils import (
|
||||
HEADRequest,
|
||||
int_or_none,
|
||||
)
|
||||
|
||||
|
||||
class BIQLEIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?biqle\.(?:com|org|ru)/watch/(?P<id>-?\d+_\d+)'
|
||||
_TESTS = [{
|
||||
'url': 'http://www.biqle.ru/watch/847655_160197695',
|
||||
'md5': 'ad5f746a874ccded7b8f211aeea96637',
|
||||
# Youtube embed
|
||||
'url': 'https://biqle.ru/watch/-115995369_456239081',
|
||||
'md5': '97af5a06ee4c29bbf9c001bdb1cf5c06',
|
||||
'info_dict': {
|
||||
'id': '160197695',
|
||||
'id': '8v4f-avW-VI',
|
||||
'ext': 'mp4',
|
||||
'title': 'Foo Fighters - The Pretender (Live at Wembley Stadium)',
|
||||
'uploader': 'Andrey Rogozin',
|
||||
'upload_date': '20110605',
|
||||
}
|
||||
'title': "PASSE-PARTOUT - L'ete c'est fait pour jouer",
|
||||
'description': 'Passe-Partout',
|
||||
'uploader_id': 'mrsimpsonstef3',
|
||||
'uploader': 'Phanolito',
|
||||
'upload_date': '20120822',
|
||||
},
|
||||
}, {
|
||||
'url': 'https://biqle.org/watch/-44781847_168547604',
|
||||
'url': 'http://biqle.org/watch/-44781847_168547604',
|
||||
'md5': '7f24e72af1db0edf7c1aaba513174f97',
|
||||
'info_dict': {
|
||||
'id': '168547604',
|
||||
'ext': 'mp4',
|
||||
'title': 'Ребенок в шоке от автоматической мойки',
|
||||
'timestamp': 1396633454,
|
||||
'uploader': 'Dmitry Kotov',
|
||||
'upload_date': '20140404',
|
||||
'uploader_id': '47850140',
|
||||
},
|
||||
'skip': ' This video was marked as adult. Embedding adult videos on external sites is prohibited.',
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
webpage = self._download_webpage(url, video_id)
|
||||
embed_url = self._proto_relative_url(self._search_regex(
|
||||
r'<iframe.+?src="((?:http:)?//daxab\.com/[^"]+)".*?></iframe>', webpage, 'embed url'))
|
||||
r'<iframe.+?src="((?:https?:)?//daxab\.com/[^"]+)".*?></iframe>',
|
||||
webpage, 'embed url'))
|
||||
if VKIE.suitable(embed_url):
|
||||
return self.url_result(embed_url, VKIE.ie_key(), video_id)
|
||||
|
||||
self._request_webpage(
|
||||
HEADRequest(embed_url), video_id, headers={'Referer': url})
|
||||
video_id, sig, _, access_token = self._get_cookies(embed_url)['video_ext'].value.split('%3A')
|
||||
item = self._download_json(
|
||||
'https://api.vk.com/method/video.get', video_id,
|
||||
headers={'User-Agent': 'okhttp/3.4.1'}, query={
|
||||
'access_token': access_token,
|
||||
'sig': sig,
|
||||
'v': 5.44,
|
||||
'videos': video_id,
|
||||
})['response']['items'][0]
|
||||
title = item['title']
|
||||
|
||||
formats = []
|
||||
for f_id, f_url in item.get('files', {}).items():
|
||||
if f_id == 'external':
|
||||
return self.url_result(f_url)
|
||||
ext, height = f_id.split('_')
|
||||
formats.append({
|
||||
'format_id': height + 'p',
|
||||
'url': f_url,
|
||||
'height': int_or_none(height),
|
||||
'ext': ext,
|
||||
})
|
||||
self._sort_formats(formats)
|
||||
|
||||
thumbnails = []
|
||||
for k, v in item.items():
|
||||
if k.startswith('photo_') and v:
|
||||
width = k.replace('photo_', '')
|
||||
thumbnails.append({
|
||||
'id': width,
|
||||
'url': v,
|
||||
'width': int_or_none(width),
|
||||
})
|
||||
|
||||
return {
|
||||
'_type': 'url_transparent',
|
||||
'url': embed_url,
|
||||
'id': video_id,
|
||||
'title': title,
|
||||
'formats': formats,
|
||||
'comment_count': int_or_none(item.get('comments')),
|
||||
'description': item.get('description'),
|
||||
'duration': int_or_none(item.get('duration')),
|
||||
'thumbnails': thumbnails,
|
||||
'timestamp': int_or_none(item.get('date')),
|
||||
'uploader': item.get('owner_id'),
|
||||
'view_count': int_or_none(item.get('views')),
|
||||
}
|
||||
|
||||
@ -126,7 +126,7 @@ class BrightcoveLegacyIE(InfoExtractor):
|
||||
'playlist_mincount': 7,
|
||||
},
|
||||
{
|
||||
# playlist with 'playlistTab' (https://github.com/rg3/youtube-dl/issues/9965)
|
||||
# playlist with 'playlistTab' (https://github.com/ytdl-org/youtube-dl/issues/9965)
|
||||
'url': 'http://c.brightcove.com/services/json/experience/runtime/?command=get_programming_for_experience&playerKey=AQ%7E%7E,AAABXlLMdok%7E,NJ4EoMlZ4rZdx9eU1rkMVd8EaYPBBUlg',
|
||||
'info_dict': {
|
||||
'id': '1522758701001',
|
||||
@ -155,10 +155,10 @@ class BrightcoveLegacyIE(InfoExtractor):
|
||||
<object class="BrightcoveExperience">{params}</object>
|
||||
"""
|
||||
|
||||
# Fix up some stupid HTML, see https://github.com/rg3/youtube-dl/issues/1553
|
||||
# Fix up some stupid HTML, see https://github.com/ytdl-org/youtube-dl/issues/1553
|
||||
object_str = re.sub(r'(<param(?:\s+[a-zA-Z0-9_]+="[^"]*")*)>',
|
||||
lambda m: m.group(1) + '/>', object_str)
|
||||
# Fix up some stupid XML, see https://github.com/rg3/youtube-dl/issues/1608
|
||||
# Fix up some stupid XML, see https://github.com/ytdl-org/youtube-dl/issues/1608
|
||||
object_str = object_str.replace('<--', '<!--')
|
||||
# remove namespace to simplify extraction
|
||||
object_str = re.sub(r'(<object[^>]*)(xmlns=".*?")', r'\1', object_str)
|
||||
|
||||
@ -360,7 +360,7 @@ class CBCWatchVideoIE(CBCWatchBaseIE):
|
||||
|
||||
class CBCWatchIE(CBCWatchBaseIE):
|
||||
IE_NAME = 'cbc.ca:watch'
|
||||
_VALID_URL = r'https?://watch\.cbc\.ca/(?:[^/]+/)+(?P<id>[0-9a-f-]+)'
|
||||
_VALID_URL = r'https?://(?:gem|watch)\.cbc\.ca/(?:[^/]+/)+(?P<id>[0-9a-f-]+)'
|
||||
_TESTS = [{
|
||||
# geo-restricted to Canada, bypassable
|
||||
'url': 'http://watch.cbc.ca/doc-zone/season-6/customer-disservice/38e815a-009e3ab12e4',
|
||||
@ -386,6 +386,9 @@ class CBCWatchIE(CBCWatchBaseIE):
|
||||
'description': 'Arthur, the sweetest 8-year-old aardvark, and his pals solve all kinds of problems with humour, kindness and teamwork.',
|
||||
},
|
||||
'playlist_mincount': 30,
|
||||
}, {
|
||||
'url': 'https://gem.cbc.ca/media/this-hour-has-22-minutes/season-26/episode-20/38e815a-0108c6c6a42',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
|
||||
@ -155,7 +155,7 @@ class CeskaTelevizeIE(InfoExtractor):
|
||||
stream_formats = self._extract_mpd_formats(
|
||||
stream_url, playlist_id,
|
||||
mpd_id='dash-%s' % format_id, fatal=False)
|
||||
# See https://github.com/rg3/youtube-dl/issues/12119#issuecomment-280037031
|
||||
# See https://github.com/ytdl-org/youtube-dl/issues/12119#issuecomment-280037031
|
||||
if format_id == 'audioDescription':
|
||||
for f in stream_formats:
|
||||
f['source_preference'] = -10
|
||||
|
||||
@ -65,8 +65,8 @@ class CiscoLiveBaseIE(InfoExtractor):
|
||||
|
||||
|
||||
class CiscoLiveSessionIE(CiscoLiveBaseIE):
|
||||
_VALID_URL = r'https?://ciscolive\.cisco\.com/on-demand-library/\??[^#]*#/session/(?P<id>[^/?&]+)'
|
||||
_TEST = {
|
||||
_VALID_URL = r'https?://(?:www\.)?ciscolive(?:\.cisco)?\.com/[^#]*#/session/(?P<id>[^/?&]+)'
|
||||
_TESTS = [{
|
||||
'url': 'https://ciscolive.cisco.com/on-demand-library/?#/session/1423353499155001FoSs',
|
||||
'md5': 'c98acf395ed9c9f766941c70f5352e22',
|
||||
'info_dict': {
|
||||
@ -79,7 +79,13 @@ class CiscoLiveSessionIE(CiscoLiveBaseIE):
|
||||
'uploader_id': '5647924234001',
|
||||
'location': '16B Mezz.',
|
||||
},
|
||||
}
|
||||
}, {
|
||||
'url': 'https://www.ciscolive.com/global/on-demand-library.html?search.event=ciscoliveemea2019#/session/15361595531500013WOU',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://www.ciscolive.com/global/on-demand-library.html?#/session/1490051371645001kNaS',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
rf_id = self._match_id(url)
|
||||
@ -88,7 +94,7 @@ class CiscoLiveSessionIE(CiscoLiveBaseIE):
|
||||
|
||||
|
||||
class CiscoLiveSearchIE(CiscoLiveBaseIE):
|
||||
_VALID_URL = r'https?://ciscolive\.cisco\.com/on-demand-library/'
|
||||
_VALID_URL = r'https?://(?:www\.)?ciscolive(?:\.cisco)?\.com/(?:global/)?on-demand-library(?:\.html|/)'
|
||||
_TESTS = [{
|
||||
'url': 'https://ciscolive.cisco.com/on-demand-library/?search.event=ciscoliveus2018&search.technicallevel=scpsSkillLevel_aintroductory&search.focus=scpsSessionFocus_designAndDeployment#/',
|
||||
'info_dict': {
|
||||
@ -98,6 +104,9 @@ class CiscoLiveSearchIE(CiscoLiveBaseIE):
|
||||
}, {
|
||||
'url': 'https://ciscolive.cisco.com/on-demand-library/?search.technology=scpsTechnology_applicationDevelopment&search.technology=scpsTechnology_ipv6&search.focus=scpsSessionFocus_troubleshootingTroubleshooting#/',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://www.ciscolive.com/global/on-demand-library.html?search.technicallevel=scpsSkillLevel_aintroductory&search.event=ciscoliveemea2019&search.technology=scpsTechnology_dataCenter&search.focus=scpsSessionFocus_bestPractices#/',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
@classmethod
|
||||
|
||||
@ -17,6 +17,7 @@ import math
|
||||
from ..compat import (
|
||||
compat_cookiejar,
|
||||
compat_cookies,
|
||||
compat_etree_Element,
|
||||
compat_etree_fromstring,
|
||||
compat_getpass,
|
||||
compat_integer_types,
|
||||
@ -43,6 +44,7 @@ from ..utils import (
|
||||
compiled_regex_type,
|
||||
determine_ext,
|
||||
determine_protocol,
|
||||
dict_get,
|
||||
error_to_compat_str,
|
||||
ExtractorError,
|
||||
extract_attributes,
|
||||
@ -55,13 +57,16 @@ from ..utils import (
|
||||
JSON_LD_RE,
|
||||
mimetype2ext,
|
||||
orderedSet,
|
||||
parse_bitrate,
|
||||
parse_codecs,
|
||||
parse_duration,
|
||||
parse_iso8601,
|
||||
parse_m3u8_attributes,
|
||||
parse_resolution,
|
||||
RegexNotFoundError,
|
||||
sanitized_Request,
|
||||
sanitize_filename,
|
||||
str_or_none,
|
||||
unescapeHTML,
|
||||
unified_strdate,
|
||||
unified_timestamp,
|
||||
@ -102,10 +107,26 @@ class InfoExtractor(object):
|
||||
from worst to best quality.
|
||||
|
||||
Potential fields:
|
||||
* url Mandatory. The URL of the video file
|
||||
* url The mandatory URL representing the media:
|
||||
for plain file media - HTTP URL of this file,
|
||||
for RTMP - RTMP URL,
|
||||
for HLS - URL of the M3U8 media playlist,
|
||||
for HDS - URL of the F4M manifest,
|
||||
for DASH
|
||||
- HTTP URL to plain file media (in case of
|
||||
unfragmented media)
|
||||
- URL of the MPD manifest or base URL
|
||||
representing the media if MPD manifest
|
||||
is parsed froma string (in case of
|
||||
fragmented media)
|
||||
for MSS - URL of the ISM manifest.
|
||||
* manifest_url
|
||||
The URL of the manifest file in case of
|
||||
fragmented media (DASH, hls, hds)
|
||||
fragmented media:
|
||||
for HLS - URL of the M3U8 master playlist,
|
||||
for HDS - URL of the F4M manifest,
|
||||
for DASH - URL of the MPD manifest,
|
||||
for MSS - URL of the ISM manifest.
|
||||
* ext Will be calculated from URL if missing
|
||||
* format A human-readable description of the format
|
||||
("mp4 container with h264/opus").
|
||||
@ -788,7 +809,7 @@ class InfoExtractor(object):
|
||||
fatal=True, encoding=None, data=None, headers={}, query={},
|
||||
expected_status=None):
|
||||
"""
|
||||
Return a tuple (xml as an xml.etree.ElementTree.Element, URL handle).
|
||||
Return a tuple (xml as an compat_etree_Element, URL handle).
|
||||
|
||||
See _download_webpage docstring for arguments specification.
|
||||
"""
|
||||
@ -809,7 +830,7 @@ class InfoExtractor(object):
|
||||
transform_source=None, fatal=True, encoding=None,
|
||||
data=None, headers={}, query={}, expected_status=None):
|
||||
"""
|
||||
Return the xml as an xml.etree.ElementTree.Element.
|
||||
Return the xml as an compat_etree_Element.
|
||||
|
||||
See _download_webpage docstring for arguments specification.
|
||||
"""
|
||||
@ -1440,7 +1461,7 @@ class InfoExtractor(object):
|
||||
manifest_url, video_id, 'Downloading f4m manifest',
|
||||
'Unable to download f4m manifest',
|
||||
# Some manifests may be malformed, e.g. prosiebensat1 generated manifests
|
||||
# (see https://github.com/rg3/youtube-dl/issues/6215#issuecomment-121704244)
|
||||
# (see https://github.com/ytdl-org/youtube-dl/issues/6215#issuecomment-121704244)
|
||||
transform_source=transform_source,
|
||||
fatal=fatal)
|
||||
|
||||
@ -1454,6 +1475,9 @@ class InfoExtractor(object):
|
||||
def _parse_f4m_formats(self, manifest, manifest_url, video_id, preference=None, f4m_id=None,
|
||||
transform_source=lambda s: fix_xml_ampersands(s).strip(),
|
||||
fatal=True, m3u8_id=None):
|
||||
if not isinstance(manifest, compat_etree_Element) and not fatal:
|
||||
return []
|
||||
|
||||
# currently youtube-dl cannot decode the playerVerificationChallenge as Akamai uses Adobe Alchemy
|
||||
akamai_pv = manifest.find('{http://ns.adobe.com/f4m/1.0}pv-2.0')
|
||||
if akamai_pv is not None and ';' in akamai_pv.text:
|
||||
@ -1468,7 +1492,7 @@ class InfoExtractor(object):
|
||||
manifest_version = '2.0'
|
||||
media_nodes = manifest.findall('{http://ns.adobe.com/f4m/2.0}media')
|
||||
# Remove unsupported DRM protected media from final formats
|
||||
# rendition (see https://github.com/rg3/youtube-dl/issues/8573).
|
||||
# rendition (see https://github.com/ytdl-org/youtube-dl/issues/8573).
|
||||
media_nodes = remove_encrypted_media(media_nodes)
|
||||
if not media_nodes:
|
||||
return formats
|
||||
@ -1598,8 +1622,8 @@ class InfoExtractor(object):
|
||||
|
||||
# References:
|
||||
# 1. https://tools.ietf.org/html/draft-pantos-http-live-streaming-21
|
||||
# 2. https://github.com/rg3/youtube-dl/issues/12211
|
||||
# 3. https://github.com/rg3/youtube-dl/issues/18923
|
||||
# 2. https://github.com/ytdl-org/youtube-dl/issues/12211
|
||||
# 3. https://github.com/ytdl-org/youtube-dl/issues/18923
|
||||
|
||||
# We should try extracting formats only from master playlists [1, 4.3.4],
|
||||
# i.e. playlists that describe available qualities. On the other hand
|
||||
@ -2120,7 +2144,6 @@ class InfoExtractor(object):
|
||||
bandwidth = int_or_none(representation_attrib.get('bandwidth'))
|
||||
f = {
|
||||
'format_id': '%s-%s' % (mpd_id, representation_id) if mpd_id else representation_id,
|
||||
'url': base_url,
|
||||
'manifest_url': mpd_url,
|
||||
'ext': mimetype2ext(mime_type),
|
||||
'width': int_or_none(representation_attrib.get('width')),
|
||||
@ -2141,7 +2164,7 @@ class InfoExtractor(object):
|
||||
# First of, % characters outside $...$ templates
|
||||
# must be escaped by doubling for proper processing
|
||||
# by % operator string formatting used further (see
|
||||
# https://github.com/rg3/youtube-dl/issues/16867).
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/16867).
|
||||
t = ''
|
||||
in_template = False
|
||||
for c in tmpl:
|
||||
@ -2160,7 +2183,7 @@ class InfoExtractor(object):
|
||||
|
||||
# @initialization is a regular template like @media one
|
||||
# so it should be handled just the same way (see
|
||||
# https://github.com/rg3/youtube-dl/issues/11605)
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/11605)
|
||||
if 'initialization' in representation_ms_info:
|
||||
initialization_template = prepare_template(
|
||||
'initialization',
|
||||
@ -2246,7 +2269,7 @@ class InfoExtractor(object):
|
||||
elif 'segment_urls' in representation_ms_info:
|
||||
# Segment URLs with no SegmentTimeline
|
||||
# Example: https://www.seznam.cz/zpravy/clanek/cesko-zasahne-vitr-o-sile-vichrice-muze-byt-i-zivotu-nebezpecny-39091
|
||||
# https://github.com/rg3/youtube-dl/pull/14844
|
||||
# https://github.com/ytdl-org/youtube-dl/pull/14844
|
||||
fragments = []
|
||||
segment_duration = float_or_none(
|
||||
representation_ms_info['segment_duration'],
|
||||
@ -2259,10 +2282,14 @@ class InfoExtractor(object):
|
||||
fragment['duration'] = segment_duration
|
||||
fragments.append(fragment)
|
||||
representation_ms_info['fragments'] = fragments
|
||||
# NB: MPD manifest may contain direct URLs to unfragmented media.
|
||||
# No fragments key is present in this case.
|
||||
# If there is a fragments key available then we correctly recognized fragmented media.
|
||||
# Otherwise we will assume unfragmented media with direct access. Technically, such
|
||||
# assumption is not necessarily correct since we may simply have no support for
|
||||
# some forms of fragmented media renditions yet, but for now we'll use this fallback.
|
||||
if 'fragments' in representation_ms_info:
|
||||
f.update({
|
||||
# NB: mpd_url may be empty when MPD manifest is parsed from a string
|
||||
'url': mpd_url or base_url,
|
||||
'fragment_base_url': base_url,
|
||||
'fragments': [],
|
||||
'protocol': 'http_dash_segments',
|
||||
@ -2273,11 +2300,15 @@ class InfoExtractor(object):
|
||||
f['url'] = initialization_url
|
||||
f['fragments'].append({location_key(initialization_url): initialization_url})
|
||||
f['fragments'].extend(representation_ms_info['fragments'])
|
||||
else:
|
||||
# Assuming direct URL to unfragmented media.
|
||||
f['url'] = base_url
|
||||
|
||||
# According to [1, 5.3.5.2, Table 7, page 35] @id of Representation
|
||||
# is not necessarily unique within a Period thus formats with
|
||||
# the same `format_id` are quite possible. There are numerous examples
|
||||
# of such manifests (see https://github.com/rg3/youtube-dl/issues/15111,
|
||||
# https://github.com/rg3/youtube-dl/issues/13919)
|
||||
# of such manifests (see https://github.com/ytdl-org/youtube-dl/issues/15111,
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/13919)
|
||||
full_info = formats_dict.get(representation_id, {}).copy()
|
||||
full_info.update(f)
|
||||
formats.append(full_info)
|
||||
@ -2438,7 +2469,7 @@ class InfoExtractor(object):
|
||||
media_tags.extend(re.findall(
|
||||
# We only allow video|audio followed by a whitespace or '>'.
|
||||
# Allowing more characters may end up in significant slow down (see
|
||||
# https://github.com/rg3/youtube-dl/issues/11979, example URL:
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/11979, example URL:
|
||||
# http://www.porntrex.com/maps/videositemap.xml).
|
||||
r'(?s)(<(?P<tag>(?:amp-)?(?:video|audio))(?:\s+[^>]*)?>)(.*?)</(?P=tag)>', webpage))
|
||||
for media_tag, media_type, media_content in media_tags:
|
||||
@ -2454,18 +2485,43 @@ class InfoExtractor(object):
|
||||
media_info['thumbnail'] = absolute_url(media_attributes.get('poster'))
|
||||
if media_content:
|
||||
for source_tag in re.findall(r'<source[^>]+>', media_content):
|
||||
source_attributes = extract_attributes(source_tag)
|
||||
src = source_attributes.get('src')
|
||||
s_attr = extract_attributes(source_tag)
|
||||
# data-video-src and data-src are non standard but seen
|
||||
# several times in the wild
|
||||
src = dict_get(s_attr, ('src', 'data-video-src', 'data-src'))
|
||||
if not src:
|
||||
continue
|
||||
f = parse_content_type(source_attributes.get('type'))
|
||||
f = parse_content_type(s_attr.get('type'))
|
||||
is_plain_url, formats = _media_formats(src, media_type, f)
|
||||
if is_plain_url:
|
||||
# res attribute is not standard but seen several times
|
||||
# in the wild
|
||||
# width, height, res, label and title attributes are
|
||||
# all not standard but seen several times in the wild
|
||||
labels = [
|
||||
s_attr.get(lbl)
|
||||
for lbl in ('label', 'title')
|
||||
if str_or_none(s_attr.get(lbl))
|
||||
]
|
||||
width = int_or_none(s_attr.get('width'))
|
||||
height = (int_or_none(s_attr.get('height')) or
|
||||
int_or_none(s_attr.get('res')))
|
||||
if not width or not height:
|
||||
for lbl in labels:
|
||||
resolution = parse_resolution(lbl)
|
||||
if not resolution:
|
||||
continue
|
||||
width = width or resolution.get('width')
|
||||
height = height or resolution.get('height')
|
||||
for lbl in labels:
|
||||
tbr = parse_bitrate(lbl)
|
||||
if tbr:
|
||||
break
|
||||
else:
|
||||
tbr = None
|
||||
f.update({
|
||||
'height': int_or_none(source_attributes.get('res')),
|
||||
'format_id': source_attributes.get('label'),
|
||||
'width': width,
|
||||
'height': height,
|
||||
'tbr': tbr,
|
||||
'format_id': s_attr.get('label') or s_attr.get('title'),
|
||||
})
|
||||
f.update(formats[0])
|
||||
media_info['formats'].append(f)
|
||||
|
||||
@ -36,7 +36,7 @@ class UnicodeBOMIE(InfoExtractor):
|
||||
_VALID_URL = r'(?P<bom>\ufeff)(?P<id>.*)$'
|
||||
|
||||
# Disable test for python 3.2 since BOM is broken in re in this version
|
||||
# (see https://github.com/rg3/youtube-dl/issues/9751)
|
||||
# (see https://github.com/ytdl-org/youtube-dl/issues/9751)
|
||||
_TESTS = [] if (3, 0) < sys.version_info <= (3, 3) else [{
|
||||
'url': '\ufeffhttp://www.youtube.com/watch?v=BaW_jenozKc',
|
||||
'only_matching': True,
|
||||
|
||||
@ -13,9 +13,9 @@ class CorusIE(ThePlatformFeedIE):
|
||||
(?:www\.)?
|
||||
(?P<domain>
|
||||
(?:globaltv|etcanada)\.com|
|
||||
(?:hgtv|foodnetwork|slice|history|showcase)\.ca
|
||||
(?:hgtv|foodnetwork|slice|history|showcase|bigbrothercanada)\.ca
|
||||
)
|
||||
/(?:video/|(?:[^/]+/)+(?:videos/[a-z0-9-]+-|video\.html\?.*?\bv=))
|
||||
/(?:video/(?:[^/]+/)?|(?:[^/]+/)+(?:videos/[a-z0-9-]+-|video\.html\?.*?\bv=))
|
||||
(?P<id>\d+)
|
||||
'''
|
||||
_TESTS = [{
|
||||
@ -42,6 +42,12 @@ class CorusIE(ThePlatformFeedIE):
|
||||
}, {
|
||||
'url': 'http://www.showcase.ca/eyewitness/video/eyewitness++106/video.html?v=955070531919&p=1&s=da#video',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'http://www.bigbrothercanada.ca/video/1457812035894/',
|
||||
'only_matching': True
|
||||
}, {
|
||||
'url': 'https://www.bigbrothercanada.ca/video/big-brother-canada-704/1457812035894/',
|
||||
'only_matching': True
|
||||
}]
|
||||
|
||||
_TP_FEEDS = {
|
||||
@ -73,6 +79,10 @@ class CorusIE(ThePlatformFeedIE):
|
||||
'feed_id': '9H6qyshBZU3E',
|
||||
'account_id': 2414426607,
|
||||
},
|
||||
'bigbrothercanada': {
|
||||
'feed_id': 'ChQqrem0lNUp',
|
||||
'account_id': 2269680845,
|
||||
},
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
|
||||
@ -3,7 +3,6 @@ from __future__ import unicode_literals
|
||||
|
||||
import re
|
||||
import json
|
||||
import xml.etree.ElementTree as etree
|
||||
import zlib
|
||||
|
||||
from hashlib import sha1
|
||||
@ -12,6 +11,7 @@ from .common import InfoExtractor
|
||||
from .vrv import VRVIE
|
||||
from ..compat import (
|
||||
compat_b64decode,
|
||||
compat_etree_Element,
|
||||
compat_etree_fromstring,
|
||||
compat_urllib_parse_urlencode,
|
||||
compat_urllib_request,
|
||||
@ -107,7 +107,7 @@ class CrunchyrollBaseIE(InfoExtractor):
|
||||
request = (url_or_request if isinstance(url_or_request, compat_urllib_request.Request)
|
||||
else sanitized_Request(url_or_request))
|
||||
# Accept-Language must be set explicitly to accept any language to avoid issues
|
||||
# similar to https://github.com/rg3/youtube-dl/issues/6797.
|
||||
# similar to https://github.com/ytdl-org/youtube-dl/issues/6797.
|
||||
# Along with IP address Crunchyroll uses Accept-Language to guess whether georestriction
|
||||
# should be imposed or not (from what I can see it just takes the first language
|
||||
# ignoring the priority and requires it to correspond the IP). By the way this causes
|
||||
@ -124,7 +124,7 @@ class CrunchyrollBaseIE(InfoExtractor):
|
||||
# > This content may be inappropriate for some people.
|
||||
# > Are you sure you want to continue?
|
||||
# since it's not disabled by default in crunchyroll account's settings.
|
||||
# See https://github.com/rg3/youtube-dl/issues/7202.
|
||||
# See https://github.com/ytdl-org/youtube-dl/issues/7202.
|
||||
qs['skip_wall'] = ['1']
|
||||
return compat_urlparse.urlunparse(
|
||||
parsed_url._replace(query=compat_urllib_parse_urlencode(qs, True)))
|
||||
@ -390,7 +390,7 @@ Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
|
||||
'Downloading subtitles for ' + sub_name, data={
|
||||
'subtitle_script_id': sub_id,
|
||||
})
|
||||
if not isinstance(sub_doc, etree.Element):
|
||||
if not isinstance(sub_doc, compat_etree_Element):
|
||||
continue
|
||||
sid = sub_doc.get('id')
|
||||
iv = xpath_text(sub_doc, 'iv', 'subtitle iv')
|
||||
@ -507,7 +507,7 @@ Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
|
||||
'video_quality': stream_quality,
|
||||
'current_page': url,
|
||||
})
|
||||
if isinstance(streamdata, etree.Element):
|
||||
if isinstance(streamdata, compat_etree_Element):
|
||||
stream_info = streamdata.find('./{default}preload/stream_info')
|
||||
if stream_info is not None:
|
||||
stream_infos.append(stream_info)
|
||||
@ -518,7 +518,7 @@ Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
|
||||
'video_format': stream_format,
|
||||
'video_encode_quality': stream_quality,
|
||||
})
|
||||
if isinstance(stream_info, etree.Element):
|
||||
if isinstance(stream_info, compat_etree_Element):
|
||||
stream_infos.append(stream_info)
|
||||
for stream_info in stream_infos:
|
||||
video_encode_id = xpath_text(stream_info, './video_encode_id')
|
||||
@ -593,7 +593,7 @@ Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
|
||||
|
||||
season = episode = episode_number = duration = thumbnail = None
|
||||
|
||||
if isinstance(metadata, etree.Element):
|
||||
if isinstance(metadata, compat_etree_Element):
|
||||
season = xpath_text(metadata, 'series_title')
|
||||
episode = xpath_text(metadata, 'episode_title')
|
||||
episode_number = int_or_none(xpath_text(metadata, 'episode_number'))
|
||||
|
||||
@ -166,11 +166,11 @@ class DailymotionIE(DailymotionBaseInfoExtractor):
|
||||
webpage, 'comment count', default=None))
|
||||
|
||||
player_v5 = self._search_regex(
|
||||
[r'buildPlayer\(({.+?})\);\n', # See https://github.com/rg3/youtube-dl/issues/7826
|
||||
[r'buildPlayer\(({.+?})\);\n', # See https://github.com/ytdl-org/youtube-dl/issues/7826
|
||||
r'playerV5\s*=\s*dmp\.create\([^,]+?,\s*({.+?})\);',
|
||||
r'buildPlayer\(({.+?})\);',
|
||||
r'var\s+config\s*=\s*({.+?});',
|
||||
# New layout regex (see https://github.com/rg3/youtube-dl/issues/13580)
|
||||
# New layout regex (see https://github.com/ytdl-org/youtube-dl/issues/13580)
|
||||
r'__PLAYER_CONFIG__\s*=\s*({.+?});'],
|
||||
webpage, 'player v5', default=None)
|
||||
if player_v5:
|
||||
|
||||
@ -114,7 +114,7 @@ class DreiSatIE(InfoExtractor):
|
||||
video_url, video_id, fatal=False))
|
||||
elif ext == 'm3u8':
|
||||
# the certificates are misconfigured (see
|
||||
# https://github.com/rg3/youtube-dl/issues/8665)
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/8665)
|
||||
if video_url.startswith('https://'):
|
||||
continue
|
||||
formats.extend(self._extract_m3u8_formats(
|
||||
|
||||
@ -1,14 +1,11 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import json
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
determine_ext,
|
||||
clean_html,
|
||||
int_or_none,
|
||||
float_or_none,
|
||||
sanitized_Request,
|
||||
)
|
||||
|
||||
|
||||
@ -36,7 +33,7 @@ def _decrypt_config(key, string):
|
||||
|
||||
|
||||
class EscapistIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://?(?:www\.)?escapistmagazine\.com/videos/view/[^/?#]+/(?P<id>[0-9]+)-[^/?#]*(?:$|[?#])'
|
||||
_VALID_URL = r'https?://?(?:(?:www|v1)\.)?escapistmagazine\.com/videos/view/[^/]+/(?P<id>[0-9]+)'
|
||||
_TESTS = [{
|
||||
'url': 'http://www.escapistmagazine.com/videos/view/the-escapist-presents/6618-Breaking-Down-Baldurs-Gate',
|
||||
'md5': 'ab3a706c681efca53f0a35f1415cf0d1',
|
||||
@ -61,6 +58,12 @@ class EscapistIE(InfoExtractor):
|
||||
'duration': 304,
|
||||
'uploader': 'The Escapist',
|
||||
}
|
||||
}, {
|
||||
'url': 'http://escapistmagazine.com/videos/view/the-escapist-presents/6618',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://v1.escapistmagazine.com/videos/view/the-escapist-presents/6618-Breaking-Down-Baldurs-Gate',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
@ -74,19 +77,20 @@ class EscapistIE(InfoExtractor):
|
||||
video_id = ims_video['videoID']
|
||||
key = ims_video['hash']
|
||||
|
||||
config_req = sanitized_Request(
|
||||
'http://www.escapistmagazine.com/videos/'
|
||||
'vidconfig.php?videoID=%s&hash=%s' % (video_id, key))
|
||||
config_req.add_header('Referer', url)
|
||||
config = self._download_webpage(config_req, video_id, 'Downloading video config')
|
||||
config = self._download_webpage(
|
||||
'http://www.escapistmagazine.com/videos/vidconfig.php',
|
||||
video_id, 'Downloading video config', headers={
|
||||
'Referer': url,
|
||||
}, query={
|
||||
'videoID': video_id,
|
||||
'hash': key,
|
||||
})
|
||||
|
||||
data = json.loads(_decrypt_config(key, config))
|
||||
data = self._parse_json(_decrypt_config(key, config), video_id)
|
||||
|
||||
video_data = data['videoData']
|
||||
|
||||
title = clean_html(video_data['title'])
|
||||
duration = float_or_none(video_data.get('duration'), 1000)
|
||||
uploader = video_data.get('publisher')
|
||||
|
||||
formats = [{
|
||||
'url': video['src'],
|
||||
@ -99,8 +103,9 @@ class EscapistIE(InfoExtractor):
|
||||
'id': video_id,
|
||||
'formats': formats,
|
||||
'title': title,
|
||||
'thumbnail': self._og_search_thumbnail(webpage),
|
||||
'thumbnail': self._og_search_thumbnail(webpage) or data.get('poster'),
|
||||
'description': self._og_search_description(webpage),
|
||||
'duration': duration,
|
||||
'uploader': uploader,
|
||||
'duration': float_or_none(video_data.get('duration'), 1000),
|
||||
'uploader': video_data.get('publisher'),
|
||||
'series': video_data.get('show'),
|
||||
}
|
||||
|
||||
@ -38,9 +38,7 @@ from .alphaporno import AlphaPornoIE
|
||||
from .amcnetworks import AMCNetworksIE
|
||||
from .americastestkitchen import AmericasTestKitchenIE
|
||||
from .animeondemand import AnimeOnDemandIE
|
||||
from .anitube import AnitubeIE
|
||||
from .anvato import AnvatoIE
|
||||
from .anysex import AnySexIE
|
||||
from .aol import AolIE
|
||||
from .allocine import AllocineIE
|
||||
from .aliexpress import AliExpressLiveIE
|
||||
@ -901,7 +899,6 @@ from .puhutv import (
|
||||
PuhuTVSerieIE,
|
||||
)
|
||||
from .presstv import PressTVIE
|
||||
from .primesharetv import PrimeShareTVIE
|
||||
from .promptfile import PromptFileIE
|
||||
from .prosiebensat1 import ProSiebenSat1IE
|
||||
from .puls4 import Puls4IE
|
||||
@ -978,7 +975,6 @@ from .rtvnh import RTVNHIE
|
||||
from .rtvs import RTVSIE
|
||||
from .rudo import RudoIE
|
||||
from .ruhd import RUHDIE
|
||||
from .ruleporn import RulePornIE
|
||||
from .rutube import (
|
||||
RutubeIE,
|
||||
RutubeChannelIE,
|
||||
@ -1299,7 +1295,6 @@ from .viddler import ViddlerIE
|
||||
from .videa import VideaIE
|
||||
from .videodetective import VideoDetectiveIE
|
||||
from .videofyme import VideofyMeIE
|
||||
from .videomega import VideoMegaIE
|
||||
from .videomore import (
|
||||
VideomoreIE,
|
||||
VideomoreVideoIE,
|
||||
@ -1449,12 +1444,13 @@ from .yahoo import (
|
||||
YahooIE,
|
||||
YahooSearchIE,
|
||||
)
|
||||
from .yandexdisk import YandexDiskIE
|
||||
from .yandexmusic import (
|
||||
YandexMusicTrackIE,
|
||||
YandexMusicAlbumIE,
|
||||
YandexMusicPlaylistIE,
|
||||
)
|
||||
from .yandexdisk import YandexDiskIE
|
||||
from .yandexvideo import YandexVideoIE
|
||||
from .yapfiles import YapFilesIE
|
||||
from .yesjapan import YesJapanIE
|
||||
from .yinyuetai import YinYueTaiIE
|
||||
|
||||
@ -427,6 +427,7 @@ class FacebookIE(InfoExtractor):
|
||||
r'\"ownerName\":"(.+?)"', tahoe_data.secondary,
|
||||
'uploader_id', fatal=False)
|
||||
|
||||
|
||||
timestamp = int_or_none(self._search_regex(
|
||||
r'<abbr[^>]+data-utime=["\'](\d+)', webpage,
|
||||
'timestamp', default=None) or self._search_regex(
|
||||
|
||||
@ -6,10 +6,12 @@ import uuid
|
||||
|
||||
from .adobepass import AdobePassIE
|
||||
from ..compat import (
|
||||
compat_HTTPError,
|
||||
compat_str,
|
||||
compat_urllib_parse_unquote,
|
||||
)
|
||||
from ..utils import (
|
||||
ExtractorError,
|
||||
int_or_none,
|
||||
parse_age_limit,
|
||||
parse_duration,
|
||||
@ -48,6 +50,7 @@ class FOXIE(AdobePassIE):
|
||||
'url': 'https://www.fox.com/watch/30056b295fb57f7452aeeb4920bc3024/',
|
||||
'only_matching': True,
|
||||
}]
|
||||
_GEO_BYPASS = False
|
||||
_HOME_PAGE_URL = 'https://www.fox.com/'
|
||||
_API_KEY = 'abdcbed02c124d393b39e818a4312055'
|
||||
_access_token = None
|
||||
@ -58,9 +61,22 @@ class FOXIE(AdobePassIE):
|
||||
}
|
||||
if self._access_token:
|
||||
headers['Authorization'] = 'Bearer ' + self._access_token
|
||||
try:
|
||||
return self._download_json(
|
||||
'https://api2.fox.com/v2.0/' + path,
|
||||
video_id, data=data, headers=headers)
|
||||
except ExtractorError as e:
|
||||
if isinstance(e.cause, compat_HTTPError) and e.cause.status == 403:
|
||||
entitlement_issues = self._parse_json(
|
||||
e.cause.read().decode(), video_id)['entitlementIssues']
|
||||
for e in entitlement_issues:
|
||||
if e.get('errorCode') == 1005:
|
||||
raise ExtractorError(
|
||||
'This video is only available via cable service provider '
|
||||
'subscription. You may want to use --cookies.', expected=True)
|
||||
messages = ', '.join([e['message'] for e in entitlement_issues])
|
||||
raise ExtractorError(messages, expected=True)
|
||||
raise
|
||||
|
||||
def _real_initialize(self):
|
||||
if not self._access_token:
|
||||
@ -81,7 +97,15 @@ class FOXIE(AdobePassIE):
|
||||
|
||||
title = video['name']
|
||||
release_url = video['url']
|
||||
try:
|
||||
m3u8_url = self._download_json(release_url, video_id)['playURL']
|
||||
except ExtractorError as e:
|
||||
if isinstance(e.cause, compat_HTTPError) and e.cause.status == 403:
|
||||
error = self._parse_json(e.cause.read().decode(), video_id)
|
||||
if error.get('exception') == 'GeoLocationBlocked':
|
||||
self.raise_geo_restricted(countries=['US'])
|
||||
raise ExtractorError(error['description'], expected=True)
|
||||
raise
|
||||
formats = self._extract_m3u8_formats(
|
||||
m3u8_url, video_id, 'mp4',
|
||||
entry_protocol='m3u8_native', m3u8_id='hls')
|
||||
|
||||
@ -143,7 +143,7 @@ class FranceTVIE(InfoExtractor):
|
||||
ext = determine_ext(video_url)
|
||||
if ext == 'f4m':
|
||||
if georestricted:
|
||||
# See https://github.com/rg3/youtube-dl/issues/3963
|
||||
# See https://github.com/ytdl-org/youtube-dl/issues/3963
|
||||
# m3u8 urls work fine
|
||||
continue
|
||||
formats.extend(self._extract_f4m_formats(
|
||||
@ -215,7 +215,7 @@ class FranceTVSiteIE(FranceTVBaseInfoExtractor):
|
||||
_TESTS = [{
|
||||
'url': 'https://www.france.tv/france-2/13h15-le-dimanche/140921-les-mysteres-de-jesus.html',
|
||||
'info_dict': {
|
||||
'id': '162311093',
|
||||
'id': 'ec217ecc-0733-48cf-ac06-af1347b849d1',
|
||||
'ext': 'mp4',
|
||||
'title': '13h15, le dimanche... - Les mystères de Jésus',
|
||||
'description': 'md5:75efe8d4c0a8205e5904498ffe1e1a42',
|
||||
@ -271,7 +271,7 @@ class FranceTVSiteIE(FranceTVBaseInfoExtractor):
|
||||
|
||||
catalogue = None
|
||||
video_id = self._search_regex(
|
||||
r'(?:data-main-video\s*=|videoId\s*:)\s*(["\'])(?P<id>(?:(?!\1).)+)\1',
|
||||
r'(?:data-main-video\s*=|videoId["\']?\s*[:=])\s*(["\'])(?P<id>(?:(?!\1).)+)\1',
|
||||
webpage, 'video id', default=None, group='id')
|
||||
|
||||
if not video_id:
|
||||
|
||||
@ -430,7 +430,7 @@ class GenericIE(InfoExtractor):
|
||||
},
|
||||
},
|
||||
{
|
||||
# https://github.com/rg3/youtube-dl/issues/2253
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/2253
|
||||
'url': 'http://bcove.me/i6nfkrc3',
|
||||
'md5': '0ba9446db037002366bab3b3eb30c88c',
|
||||
'info_dict': {
|
||||
@ -455,7 +455,7 @@ class GenericIE(InfoExtractor):
|
||||
},
|
||||
},
|
||||
{
|
||||
# https://github.com/rg3/youtube-dl/issues/3541
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/3541
|
||||
'add_ie': ['BrightcoveLegacy'],
|
||||
'url': 'http://www.kijk.nl/sbs6/leermijvrouwenkennen/videos/jqMiXKAYan2S/aflevering-1',
|
||||
'info_dict': {
|
||||
@ -919,7 +919,7 @@ class GenericIE(InfoExtractor):
|
||||
}
|
||||
},
|
||||
# Multiple brightcove videos
|
||||
# https://github.com/rg3/youtube-dl/issues/2283
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/2283
|
||||
{
|
||||
'url': 'http://www.newyorker.com/online/blogs/newsdesk/2014/01/always-never-nuclear-command-and-control.html',
|
||||
'info_dict': {
|
||||
@ -2371,7 +2371,7 @@ class GenericIE(InfoExtractor):
|
||||
return camtasia_res
|
||||
|
||||
# Sometimes embedded video player is hidden behind percent encoding
|
||||
# (e.g. https://github.com/rg3/youtube-dl/issues/2448)
|
||||
# (e.g. https://github.com/ytdl-org/youtube-dl/issues/2448)
|
||||
# Unescaping the whole page allows to handle those cases in a generic way
|
||||
webpage = compat_urllib_parse_unquote(webpage)
|
||||
|
||||
@ -3177,7 +3177,7 @@ class GenericIE(InfoExtractor):
|
||||
jwplayer_data, video_id, require_title=False, base_url=url)
|
||||
return merge_dicts(info, info_dict)
|
||||
except ExtractorError:
|
||||
# See https://github.com/rg3/youtube-dl/pull/16735
|
||||
# See https://github.com/ytdl-org/youtube-dl/pull/16735
|
||||
pass
|
||||
|
||||
# Video.js embed
|
||||
|
||||
@ -36,7 +36,7 @@ class GoogleDriveIE(InfoExtractor):
|
||||
}
|
||||
}, {
|
||||
# video can't be watched anonymously due to view count limit reached,
|
||||
# but can be downloaded (see https://github.com/rg3/youtube-dl/issues/14046)
|
||||
# but can be downloaded (see https://github.com/ytdl-org/youtube-dl/issues/14046)
|
||||
'url': 'https://drive.google.com/file/d/0B-vUyvmDLdWDcEt4WjBqcmI2XzQ/view',
|
||||
'md5': 'bfbd670d03a470bb1e6d4a257adec12e',
|
||||
'info_dict': {
|
||||
|
||||
@ -1,18 +1,11 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import json
|
||||
import time
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..compat import compat_urllib_parse_urlencode
|
||||
from ..utils import (
|
||||
ExtractorError,
|
||||
sanitized_Request,
|
||||
)
|
||||
from ..utils import int_or_none
|
||||
|
||||
|
||||
class HypemIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?hypem\.com/track/(?P<id>[^/]+)/'
|
||||
_VALID_URL = r'https?://(?:www\.)?hypem\.com/track/(?P<id>[0-9a-z]{5})'
|
||||
_TEST = {
|
||||
'url': 'http://hypem.com/track/1v6ga/BODYWORK+-+TAME',
|
||||
'md5': 'b9cc91b5af8995e9f0c1cee04c575828',
|
||||
@ -21,41 +14,36 @@ class HypemIE(InfoExtractor):
|
||||
'ext': 'mp3',
|
||||
'title': 'Tame',
|
||||
'uploader': 'BODYWORK',
|
||||
'timestamp': 1371810457,
|
||||
'upload_date': '20130621',
|
||||
}
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
track_id = self._match_id(url)
|
||||
|
||||
data = {'ax': 1, 'ts': time.time()}
|
||||
request = sanitized_Request(url + '?' + compat_urllib_parse_urlencode(data))
|
||||
response, urlh = self._download_webpage_handle(
|
||||
request, track_id, 'Downloading webpage with the url')
|
||||
response = self._download_webpage(url, track_id)
|
||||
|
||||
html_tracks = self._html_search_regex(
|
||||
r'(?ms)<script type="application/json" id="displayList-data">(.+?)</script>',
|
||||
response, 'tracks')
|
||||
try:
|
||||
track_list = json.loads(html_tracks)
|
||||
track = track_list['tracks'][0]
|
||||
except ValueError:
|
||||
raise ExtractorError('Hypemachine contained invalid JSON.')
|
||||
track = self._parse_json(self._html_search_regex(
|
||||
r'(?s)<script\s+type="application/json"\s+id="displayList-data">(.+?)</script>',
|
||||
response, 'tracks'), track_id)['tracks'][0]
|
||||
|
||||
key = track['key']
|
||||
track_id = track['id']
|
||||
title = track['song']
|
||||
|
||||
request = sanitized_Request(
|
||||
'http://hypem.com/serve/source/%s/%s' % (track_id, key),
|
||||
'', {'Content-Type': 'application/json'})
|
||||
song_data = self._download_json(request, track_id, 'Downloading metadata')
|
||||
final_url = song_data['url']
|
||||
artist = track.get('artist')
|
||||
final_url = self._download_json(
|
||||
'http://hypem.com/serve/source/%s/%s' % (track_id, track['key']),
|
||||
track_id, 'Downloading metadata', headers={
|
||||
'Content-Type': 'application/json'
|
||||
})['url']
|
||||
|
||||
return {
|
||||
'id': track_id,
|
||||
'url': final_url,
|
||||
'ext': 'mp3',
|
||||
'title': title,
|
||||
'uploader': artist,
|
||||
'uploader': track.get('artist'),
|
||||
'duration': int_or_none(track.get('time')),
|
||||
'timestamp': int_or_none(track.get('ts')),
|
||||
'track': title,
|
||||
}
|
||||
|
||||
@ -215,7 +215,7 @@ class KuwoSingerIE(InfoExtractor):
|
||||
'title': 'Ali',
|
||||
},
|
||||
'playlist_mincount': 95,
|
||||
'skip': 'Regularly stalls travis build', # See https://travis-ci.org/rg3/youtube-dl/jobs/78878540
|
||||
'skip': 'Regularly stalls travis build', # See https://travis-ci.org/ytdl-org/youtube-dl/jobs/78878540
|
||||
}]
|
||||
|
||||
PAGE_SIZE = 15
|
||||
|
||||
@ -44,7 +44,7 @@ class LiveLeakIE(InfoExtractor):
|
||||
},
|
||||
'skip': 'Video is dead',
|
||||
}, {
|
||||
# Covers https://github.com/rg3/youtube-dl/pull/5983
|
||||
# Covers https://github.com/ytdl-org/youtube-dl/pull/5983
|
||||
# Multiple resolutions
|
||||
'url': 'http://www.liveleak.com/view?i=801_1409392012',
|
||||
'md5': 'c3a449dbaca5c0d1825caecd52a57d7b',
|
||||
@ -57,7 +57,7 @@ class LiveLeakIE(InfoExtractor):
|
||||
'thumbnail': r're:^https?://.*\.jpg$'
|
||||
}
|
||||
}, {
|
||||
# Covers https://github.com/rg3/youtube-dl/pull/10664#issuecomment-247439521
|
||||
# Covers https://github.com/ytdl-org/youtube-dl/pull/10664#issuecomment-247439521
|
||||
'url': 'http://m.liveleak.com/view?i=763_1473349649',
|
||||
'add_ie': ['Youtube'],
|
||||
'info_dict': {
|
||||
@ -130,7 +130,7 @@ class LiveLeakIE(InfoExtractor):
|
||||
|
||||
# Removing '.*.mp4' gives the raw video, which is essentially
|
||||
# the same video without the LiveLeak logo at the top (see
|
||||
# https://github.com/rg3/youtube-dl/pull/4768)
|
||||
# https://github.com/ytdl-org/youtube-dl/pull/4768)
|
||||
orig_url = re.sub(r'\.mp4\.[^.]+', '', a_format['url'])
|
||||
if a_format['url'] != orig_url:
|
||||
format_id = a_format.get('format_id')
|
||||
|
||||
@ -1,15 +1,12 @@
|
||||
# coding: utf-8
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import json
|
||||
import re
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
ExtractorError,
|
||||
clean_html,
|
||||
int_or_none,
|
||||
sanitized_Request,
|
||||
urlencode_postdata,
|
||||
)
|
||||
|
||||
|
||||
@ -17,8 +14,8 @@ class MoeVideoIE(InfoExtractor):
|
||||
IE_DESC = 'LetitBit video services: moevideo.net, playreplay.net and videochart.net'
|
||||
_VALID_URL = r'''(?x)
|
||||
https?://(?P<host>(?:www\.)?
|
||||
(?:(?:moevideo|playreplay|videochart)\.net))/
|
||||
(?:video|framevideo)/(?P<id>[0-9]+\.[0-9A-Za-z]+)'''
|
||||
(?:(?:moevideo|playreplay|videochart)\.net|thesame\.tv))/
|
||||
(?:video|framevideo|embed)/(?P<id>[0-9a-z]+\.[0-9A-Za-z]+)'''
|
||||
_API_URL = 'http://api.letitbit.net/'
|
||||
_API_KEY = 'tVL0gjqo5'
|
||||
_TESTS = [
|
||||
@ -57,58 +54,26 @@ class MoeVideoIE(InfoExtractor):
|
||||
]
|
||||
|
||||
def _real_extract(self, url):
|
||||
mobj = re.match(self._VALID_URL, url)
|
||||
video_id = mobj.group('id')
|
||||
host, video_id = re.match(self._VALID_URL, url).groups()
|
||||
|
||||
webpage = self._download_webpage(
|
||||
'http://%s/video/%s' % (mobj.group('host'), video_id),
|
||||
'http://%s/video/%s' % (host, video_id),
|
||||
video_id, 'Downloading webpage')
|
||||
|
||||
title = self._og_search_title(webpage)
|
||||
thumbnail = self._og_search_thumbnail(webpage)
|
||||
description = self._og_search_description(webpage)
|
||||
|
||||
r = [
|
||||
self._API_KEY,
|
||||
[
|
||||
'preview/flv_link',
|
||||
{
|
||||
'uid': video_id,
|
||||
},
|
||||
],
|
||||
]
|
||||
r_json = json.dumps(r)
|
||||
post = urlencode_postdata({'r': r_json})
|
||||
req = sanitized_Request(self._API_URL, post)
|
||||
req.add_header('Content-type', 'application/x-www-form-urlencoded')
|
||||
|
||||
response = self._download_json(req, video_id)
|
||||
if response['status'] != 'OK':
|
||||
raise ExtractorError(
|
||||
'%s returned error: %s' % (self.IE_NAME, response['data']),
|
||||
expected=True
|
||||
)
|
||||
item = response['data'][0]
|
||||
video_url = item['link']
|
||||
duration = int_or_none(item['length'])
|
||||
width = int_or_none(item['width'])
|
||||
height = int_or_none(item['height'])
|
||||
filesize = int_or_none(item['convert_size'])
|
||||
|
||||
formats = [{
|
||||
'format_id': 'sd',
|
||||
'http_headers': {'Range': 'bytes=0-'}, # Required to download
|
||||
'url': video_url,
|
||||
'width': width,
|
||||
'height': height,
|
||||
'filesize': filesize,
|
||||
}]
|
||||
embed_webpage = self._download_webpage(
|
||||
'http://%s/embed/%s' % (host, video_id),
|
||||
video_id, 'Downloading embed webpage')
|
||||
video = self._parse_json(self._search_regex(
|
||||
r'mvplayer\("#player"\s*,\s*({.+})',
|
||||
embed_webpage, 'mvplayer'), video_id)['video']
|
||||
|
||||
return {
|
||||
'id': video_id,
|
||||
'title': title,
|
||||
'thumbnail': thumbnail,
|
||||
'description': description,
|
||||
'duration': duration,
|
||||
'formats': formats,
|
||||
'thumbnail': video.get('poster') or self._og_search_thumbnail(webpage),
|
||||
'description': clean_html(self._og_search_description(webpage)),
|
||||
'duration': int_or_none(self._og_search_property('video:duration', webpage)),
|
||||
'url': video['ourUrl'],
|
||||
}
|
||||
|
||||
@ -70,7 +70,7 @@ class MSNIE(InfoExtractor):
|
||||
continue
|
||||
if 'm3u8' in format_url:
|
||||
# m3u8_native should not be used here until
|
||||
# https://github.com/rg3/youtube-dl/issues/9913 is fixed
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/9913 is fixed
|
||||
m3u8_formats = self._extract_m3u8_formats(
|
||||
format_url, display_id, 'mp4',
|
||||
m3u8_id='hls', fatal=False)
|
||||
|
||||
@ -108,7 +108,7 @@ class NHLIE(NHLBaseIE):
|
||||
'timestamp': 1454544904,
|
||||
},
|
||||
}, {
|
||||
# Some m3u8 URLs are invalid (https://github.com/rg3/youtube-dl/issues/10713)
|
||||
# Some m3u8 URLs are invalid (https://github.com/ytdl-org/youtube-dl/issues/10713)
|
||||
'url': 'https://www.nhl.com/predators/video/poile-laviolette-on-subban-trade/t-277437416/c-44315003',
|
||||
'md5': '50b2bb47f405121484dda3ccbea25459',
|
||||
'info_dict': {
|
||||
|
||||
@ -115,7 +115,7 @@ class NocoIE(InfoExtractor):
|
||||
|
||||
# Timestamp adjustment offset between server time and local time
|
||||
# must be calculated in order to use timestamps closest to server's
|
||||
# in all API requests (see https://github.com/rg3/youtube-dl/issues/7864)
|
||||
# in all API requests (see https://github.com/ytdl-org/youtube-dl/issues/7864)
|
||||
webpage = self._download_webpage(url, video_id)
|
||||
|
||||
player_url = self._search_regex(
|
||||
|
||||
@ -238,7 +238,7 @@ class NPOIE(NPOBaseIE):
|
||||
formats.extend(self._extract_m3u8_formats(
|
||||
stream_url, video_id, ext='mp4',
|
||||
entry_protocol='m3u8_native', m3u8_id='hls', fatal=False))
|
||||
elif '.ism/Manifest' in stream_url:
|
||||
elif re.search(r'\.isml?/Manifest', stream_url):
|
||||
formats.extend(self._extract_ism_formats(
|
||||
stream_url, video_id, ism_id='mss', fatal=False))
|
||||
else:
|
||||
|
||||
@ -1,7 +1,6 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..compat import compat_urllib_parse_urlencode
|
||||
from ..utils import (
|
||||
int_or_none,
|
||||
qualities,
|
||||
@ -9,16 +8,16 @@ from ..utils import (
|
||||
|
||||
|
||||
class NprIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?npr\.org/player/v2/mediaPlayer\.html\?.*\bid=(?P<id>\d+)'
|
||||
_VALID_URL = r'https?://(?:www\.)?npr\.org/(?:sections/[^/]+/)?\d{4}/\d{2}/\d{2}/(?P<id>\d+)'
|
||||
_TESTS = [{
|
||||
'url': 'http://www.npr.org/player/v2/mediaPlayer.html?id=449974205',
|
||||
'url': 'https://www.npr.org/sections/allsongs/2015/10/21/449974205/new-music-from-beach-house-chairlift-cmj-discoveries-and-more',
|
||||
'info_dict': {
|
||||
'id': '449974205',
|
||||
'title': 'New Music From Beach House, Chairlift, CMJ Discoveries And More'
|
||||
},
|
||||
'playlist_count': 7,
|
||||
}, {
|
||||
'url': 'http://www.npr.org/player/v2/mediaPlayer.html?action=1&t=1&islist=false&id=446928052&m=446929930&live=1',
|
||||
'url': 'https://www.npr.org/sections/deceptivecadence/2015/10/09/446928052/music-from-the-shadows-ancient-armenian-hymns-and-piano-jazz',
|
||||
'info_dict': {
|
||||
'id': '446928052',
|
||||
'title': "Songs We Love: Tigran Hamasyan, 'Your Mercy is Boundless'"
|
||||
@ -32,30 +31,46 @@ class NprIE(InfoExtractor):
|
||||
'duration': 402,
|
||||
},
|
||||
}],
|
||||
}, {
|
||||
# mutlimedia, not media title
|
||||
'url': 'https://www.npr.org/2017/06/19/533198237/tigers-jaw-tiny-desk-concert',
|
||||
'info_dict': {
|
||||
'id': '533198237',
|
||||
'title': 'Tigers Jaw: Tiny Desk Concert',
|
||||
},
|
||||
'playlist': [{
|
||||
'md5': '12fa60cb2d3ed932f53609d4aeceabf1',
|
||||
'info_dict': {
|
||||
'id': '533201718',
|
||||
'ext': 'mp4',
|
||||
'title': 'Tigers Jaw: Tiny Desk Concert',
|
||||
'duration': 402,
|
||||
},
|
||||
}],
|
||||
'expected_warnings': ['Failed to download m3u8 information'],
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
playlist_id = self._match_id(url)
|
||||
|
||||
config = self._download_json(
|
||||
'http://api.npr.org/query?%s' % compat_urllib_parse_urlencode({
|
||||
story = self._download_json(
|
||||
'http://api.npr.org/query', playlist_id, query={
|
||||
'id': playlist_id,
|
||||
'fields': 'titles,audio,show',
|
||||
'fields': 'audio,multimedia,title',
|
||||
'format': 'json',
|
||||
'apiKey': 'MDAzMzQ2MjAyMDEyMzk4MTU1MDg3ZmM3MQ010',
|
||||
}), playlist_id)
|
||||
})['list']['story'][0]
|
||||
playlist_title = story.get('title', {}).get('$text')
|
||||
|
||||
story = config['list']['story'][0]
|
||||
|
||||
KNOWN_FORMATS = ('threegp', 'mp4', 'mp3')
|
||||
KNOWN_FORMATS = ('threegp', 'm3u8', 'smil', 'mp4', 'mp3')
|
||||
quality = qualities(KNOWN_FORMATS)
|
||||
|
||||
entries = []
|
||||
for audio in story.get('audio', []):
|
||||
title = audio.get('title', {}).get('$text')
|
||||
duration = int_or_none(audio.get('duration', {}).get('$text'))
|
||||
for media in story.get('audio', []) + story.get('multimedia', []):
|
||||
media_id = media['id']
|
||||
|
||||
formats = []
|
||||
for format_id, formats_entry in audio.get('format', {}).items():
|
||||
for format_id, formats_entry in media.get('format', {}).items():
|
||||
if not formats_entry:
|
||||
continue
|
||||
if isinstance(formats_entry, list):
|
||||
@ -64,19 +79,30 @@ class NprIE(InfoExtractor):
|
||||
if not format_url:
|
||||
continue
|
||||
if format_id in KNOWN_FORMATS:
|
||||
if format_id == 'm3u8':
|
||||
formats.extend(self._extract_m3u8_formats(
|
||||
format_url, media_id, 'mp4', 'm3u8_native',
|
||||
m3u8_id='hls', fatal=False))
|
||||
elif format_id == 'smil':
|
||||
smil_formats = self._extract_smil_formats(
|
||||
format_url, media_id, transform_source=lambda s: s.replace(
|
||||
'rtmp://flash.npr.org/ondemand/', 'https://ondemand.npr.org/'))
|
||||
self._check_formats(smil_formats, media_id)
|
||||
formats.extend(smil_formats)
|
||||
else:
|
||||
formats.append({
|
||||
'url': format_url,
|
||||
'format_id': format_id,
|
||||
'ext': formats_entry.get('type'),
|
||||
'quality': quality(format_id),
|
||||
})
|
||||
self._sort_formats(formats)
|
||||
|
||||
entries.append({
|
||||
'id': audio['id'],
|
||||
'title': title,
|
||||
'duration': duration,
|
||||
'id': media_id,
|
||||
'title': media.get('title', {}).get('$text') or playlist_title,
|
||||
'thumbnail': media.get('altImageUrl', {}).get('$text'),
|
||||
'duration': int_or_none(media.get('duration', {}).get('$text')),
|
||||
'formats': formats,
|
||||
})
|
||||
|
||||
playlist_title = story.get('title', {}).get('$text')
|
||||
return self.playlist_result(entries, playlist_id, playlist_title)
|
||||
|
||||
@ -21,7 +21,7 @@ class OnceIE(InfoExtractor):
|
||||
progressive_formats = []
|
||||
for adaptive_format in formats:
|
||||
# Prevent advertisement from embedding into m3u8 playlist (see
|
||||
# https://github.com/rg3/youtube-dl/issues/8893#issuecomment-199912684)
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/8893#issuecomment-199912684)
|
||||
adaptive_format['url'] = re.sub(
|
||||
r'\badsegmentlength=\d+', r'adsegmentlength=0', adaptive_format['url'])
|
||||
rendition_id = self._search_regex(
|
||||
|
||||
@ -243,18 +243,16 @@ class PhantomJSwrapper(object):
|
||||
|
||||
|
||||
class OpenloadIE(InfoExtractor):
|
||||
_DOMAINS = r'(?:openload\.(?:co|io|link|pw)|oload\.(?:tv|stream|site|xyz|win|download|cloud|cc|icu|fun|club|info|pw|live|space))'
|
||||
_VALID_URL = r'''(?x)
|
||||
https?://
|
||||
(?P<host>
|
||||
(?:www\.)?
|
||||
(?:
|
||||
openload\.(?:co|io|link|pw)|
|
||||
oload\.(?:tv|stream|site|xyz|win|download|cloud|cc|icu|fun|club|info|pw|live)
|
||||
)
|
||||
%s
|
||||
)/
|
||||
(?:f|embed)/
|
||||
(?P<id>[a-zA-Z0-9-_]+)
|
||||
'''
|
||||
''' % _DOMAINS
|
||||
|
||||
_TESTS = [{
|
||||
'url': 'https://openload.co/f/kUEfGclsU9o',
|
||||
@ -349,6 +347,9 @@ class OpenloadIE(InfoExtractor):
|
||||
}, {
|
||||
'url': 'https://oload.live/f/-Z58UZ-GR4M',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://oload.space/f/IY4eZSst3u8/',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
_USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36'
|
||||
@ -356,8 +357,8 @@ class OpenloadIE(InfoExtractor):
|
||||
@staticmethod
|
||||
def _extract_urls(webpage):
|
||||
return re.findall(
|
||||
r'<iframe[^>]+src=["\']((?:https?://)?(?:openload\.(?:co|io)|oload\.tv)/embed/[a-zA-Z0-9-_]+)',
|
||||
webpage)
|
||||
r'<iframe[^>]+src=["\']((?:https?://)?%s/embed/[a-zA-Z0-9-_]+)'
|
||||
% OpenloadIE._DOMAINS, webpage)
|
||||
|
||||
def _real_extract(self, url):
|
||||
mobj = re.match(self._VALID_URL, url)
|
||||
|
||||
@ -176,7 +176,8 @@ class ORFRadioIE(InfoExtractor):
|
||||
'description': subtitle,
|
||||
'duration': (info['end'] - info['start']) / 1000,
|
||||
'timestamp': info['start'] / 1000,
|
||||
'ext': 'mp3'
|
||||
'ext': 'mp3',
|
||||
'series': data.get('programTitle')
|
||||
}
|
||||
|
||||
entries = [extract_entry_dict(t, data['title'], data['subtitle']) for t in data['streams']]
|
||||
|
||||
@ -36,7 +36,7 @@ class PandaTVIE(InfoExtractor):
|
||||
'https://www.panda.tv/api_room_v2?roomid=%s' % video_id, video_id)
|
||||
|
||||
error_code = config.get('errno', 0)
|
||||
if error_code is not 0:
|
||||
if error_code != 0:
|
||||
raise ExtractorError(
|
||||
'%s returned error %s: %s'
|
||||
% (self.IE_NAME, error_code, config['errmsg']),
|
||||
|
||||
@ -305,7 +305,7 @@ class PBSIE(InfoExtractor):
|
||||
{
|
||||
# Video embedded in iframe containing angle brackets as attribute's value (e.g.
|
||||
# "<iframe style='position: absolute;<br />\ntop: 0; left: 0;' ...", see
|
||||
# https://github.com/rg3/youtube-dl/issues/7059)
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/7059)
|
||||
'url': 'http://www.pbs.org/food/features/a-chefs-life-season-3-episode-5-prickly-business/',
|
||||
'md5': '59b0ef5009f9ac8a319cc5efebcd865e',
|
||||
'info_dict': {
|
||||
@ -348,7 +348,7 @@ class PBSIE(InfoExtractor):
|
||||
},
|
||||
},
|
||||
{
|
||||
# https://github.com/rg3/youtube-dl/issues/13801
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/13801
|
||||
'url': 'https://www.pbs.org/video/pbs-newshour-full-episode-july-31-2017-1501539057/',
|
||||
'info_dict': {
|
||||
'id': '3003333873',
|
||||
@ -642,7 +642,7 @@ class PBSIE(InfoExtractor):
|
||||
# we won't try extracting them.
|
||||
# Since summer 2016 higher quality formats (4500k and 6500k) are also available
|
||||
# albeit they are not documented in [2].
|
||||
# 1. https://github.com/rg3/youtube-dl/commit/cbc032c8b70a038a69259378c92b4ba97b42d491#commitcomment-17313656
|
||||
# 1. https://github.com/ytdl-org/youtube-dl/commit/cbc032c8b70a038a69259378c92b4ba97b42d491#commitcomment-17313656
|
||||
# 2. https://projects.pbs.org/confluence/display/coveapi/COVE+Video+Specifications
|
||||
if not bitrate or int(bitrate) < 400:
|
||||
continue
|
||||
|
||||
@ -327,7 +327,7 @@ query viewClip {
|
||||
)
|
||||
|
||||
# Some courses also offer widescreen resolution for high quality (see
|
||||
# https://github.com/rg3/youtube-dl/issues/7766)
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/7766)
|
||||
widescreen = course.get('supportsWideScreenVideoFormats') is True
|
||||
best_quality = 'high-widescreen' if widescreen else 'high'
|
||||
if widescreen:
|
||||
@ -388,8 +388,8 @@ query viewClip {
|
||||
|
||||
# Pluralsight tracks multiple sequential calls to ViewClip API and start
|
||||
# to return 429 HTTP errors after some time (see
|
||||
# https://github.com/rg3/youtube-dl/pull/6989). Moreover it may even lead
|
||||
# to account ban (see https://github.com/rg3/youtube-dl/issues/6842).
|
||||
# https://github.com/ytdl-org/youtube-dl/pull/6989). Moreover it may even lead
|
||||
# to account ban (see https://github.com/ytdl-org/youtube-dl/issues/6842).
|
||||
# To somewhat reduce the probability of these consequences
|
||||
# we will sleep random amount of time before each call to ViewClip.
|
||||
self._sleep(
|
||||
|
||||
@ -332,7 +332,7 @@ class PornHubPlaylistBaseIE(PornHubBaseIE):
|
||||
def _extract_entries(self, webpage, host):
|
||||
# Only process container div with main playlist content skipping
|
||||
# drop-down menu that uses similar pattern for videos (see
|
||||
# https://github.com/rg3/youtube-dl/issues/11594).
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/11594).
|
||||
container = self._search_regex(
|
||||
r'(?s)(<div[^>]+class=["\']container.+)', webpage,
|
||||
'container', default=webpage)
|
||||
|
||||
@ -1,62 +0,0 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
ExtractorError,
|
||||
sanitized_Request,
|
||||
urlencode_postdata,
|
||||
)
|
||||
|
||||
|
||||
class PrimeShareTVIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?primeshare\.tv/download/(?P<id>[\da-zA-Z]+)'
|
||||
|
||||
_TEST = {
|
||||
'url': 'http://primeshare.tv/download/238790B611',
|
||||
'md5': 'b92d9bf5461137c36228009f31533fbc',
|
||||
'info_dict': {
|
||||
'id': '238790B611',
|
||||
'ext': 'mp4',
|
||||
'title': 'Public Domain - 1960s Commercial - Crest Toothpaste-YKsuFona',
|
||||
},
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
|
||||
webpage = self._download_webpage(url, video_id)
|
||||
|
||||
if '>File not exist<' in webpage:
|
||||
raise ExtractorError('Video %s does not exist' % video_id, expected=True)
|
||||
|
||||
fields = self._hidden_inputs(webpage)
|
||||
|
||||
headers = {
|
||||
'Referer': url,
|
||||
'Content-Type': 'application/x-www-form-urlencoded',
|
||||
}
|
||||
|
||||
wait_time = int(self._search_regex(
|
||||
r'var\s+cWaitTime\s*=\s*(\d+)',
|
||||
webpage, 'wait time', default=7)) + 1
|
||||
self._sleep(wait_time, video_id)
|
||||
|
||||
req = sanitized_Request(
|
||||
url, urlencode_postdata(fields), headers)
|
||||
video_page = self._download_webpage(
|
||||
req, video_id, 'Downloading video page')
|
||||
|
||||
video_url = self._search_regex(
|
||||
r"url\s*:\s*'([^']+\.primeshare\.tv(?::443)?/file/[^']+)'",
|
||||
video_page, 'video url')
|
||||
|
||||
title = self._html_search_regex(
|
||||
r'<h1>Watch\s*(?: )?\s*\((.+?)(?:\s*\[\.\.\.\])?\)\s*(?: )?\s*<strong>',
|
||||
video_page, 'title')
|
||||
|
||||
return {
|
||||
'id': video_id,
|
||||
'url': video_url,
|
||||
'title': title,
|
||||
'ext': 'mp4',
|
||||
}
|
||||
@ -140,8 +140,8 @@ class ProSiebenSat1IE(ProSiebenSat1BaseIE):
|
||||
|
||||
_TESTS = [
|
||||
{
|
||||
# Tests changes introduced in https://github.com/rg3/youtube-dl/pull/6242
|
||||
# in response to fixing https://github.com/rg3/youtube-dl/issues/6215:
|
||||
# Tests changes introduced in https://github.com/ytdl-org/youtube-dl/pull/6242
|
||||
# in response to fixing https://github.com/ytdl-org/youtube-dl/issues/6215:
|
||||
# - malformed f4m manifest support
|
||||
# - proper handling of URLs starting with `https?://` in 2.0 manifests
|
||||
# - recursive child f4m manifests extraction
|
||||
|
||||
@ -96,7 +96,7 @@ class RadioCanadaIE(InfoExtractor):
|
||||
return text
|
||||
|
||||
# protectionType does not necessarily mean the video is DRM protected (see
|
||||
# https://github.com/rg3/youtube-dl/pull/18609).
|
||||
# https://github.com/ytdl-org/youtube-dl/pull/18609).
|
||||
if get_meta('protectionType'):
|
||||
self.report_warning('This video is probably DRM protected.')
|
||||
|
||||
@ -147,7 +147,7 @@ class RadioCanadaIE(InfoExtractor):
|
||||
|
||||
|
||||
class RadioCanadaAudioVideoIE(InfoExtractor):
|
||||
'radiocanada:audiovideo'
|
||||
IE_NAME = 'radiocanada:audiovideo'
|
||||
_VALID_URL = r'https?://ici\.radio-canada\.ca/([^/]+/)*media-(?P<id>[0-9]+)'
|
||||
_TESTS = [{
|
||||
'url': 'http://ici.radio-canada.ca/audio-video/media-7527184/barack-obama-au-vietnam',
|
||||
|
||||
@ -74,11 +74,11 @@ class RaiBaseIE(InfoExtractor):
|
||||
if (ext == 'm3u8' and platform != 'mon') or (ext == 'f4m' and platform != 'flash'):
|
||||
continue
|
||||
|
||||
if ext == 'm3u8':
|
||||
if ext == 'm3u8' or 'format=m3u8' in media_url or platform == 'mon':
|
||||
formats.extend(self._extract_m3u8_formats(
|
||||
media_url, video_id, 'mp4', 'm3u8_native',
|
||||
m3u8_id='hls', fatal=False))
|
||||
elif ext == 'f4m':
|
||||
elif ext == 'f4m' or platform == 'flash':
|
||||
manifest_url = update_url_query(
|
||||
media_url.replace('manifest#live_hds.f4m', 'manifest.f4m'),
|
||||
{'hdcore': '3.7.0', 'plugin': 'aasp-3.7.0.39.44'})
|
||||
|
||||
@ -45,7 +45,7 @@ class RtlNlIE(InfoExtractor):
|
||||
'description': 'Er zijn nieuwe beelden vrijgegeven die vlak na de aanslag in Kopenhagen zijn gemaakt. Op de video is goed te zien hoe omstanders zich bekommeren om één van de slachtoffers, terwijl de eerste agenten ter plaatse komen.',
|
||||
}
|
||||
}, {
|
||||
# empty synopsis and missing episodes (see https://github.com/rg3/youtube-dl/issues/6275)
|
||||
# empty synopsis and missing episodes (see https://github.com/ytdl-org/youtube-dl/issues/6275)
|
||||
# best format available nettv
|
||||
'url': 'http://www.rtl.nl/system/videoplayer/derden/rtlnieuws/video_embed.html#uuid=f536aac0-1dc3-4314-920e-3bd1c5b3811a/autoplay=false',
|
||||
'info_dict': {
|
||||
|
||||
@ -1,44 +0,0 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from .nuevo import NuevoBaseIE
|
||||
|
||||
|
||||
class RulePornIE(NuevoBaseIE):
|
||||
_VALID_URL = r'https?://(?:www\.)?ruleporn\.com/(?:[^/?#&]+/)*(?P<id>[^/?#&]+)'
|
||||
_TEST = {
|
||||
'url': 'http://ruleporn.com/brunette-nympho-chick-takes-her-boyfriend-in-every-angle/',
|
||||
'md5': '86861ebc624a1097c7c10eaf06d7d505',
|
||||
'info_dict': {
|
||||
'id': '48212',
|
||||
'display_id': 'brunette-nympho-chick-takes-her-boyfriend-in-every-angle',
|
||||
'ext': 'mp4',
|
||||
'title': 'Brunette Nympho Chick Takes Her Boyfriend In Every Angle',
|
||||
'description': 'md5:6d28be231b981fff1981deaaa03a04d5',
|
||||
'age_limit': 18,
|
||||
'duration': 635.1,
|
||||
}
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
display_id = self._match_id(url)
|
||||
|
||||
webpage = self._download_webpage(url, display_id)
|
||||
|
||||
video_id = self._search_regex(
|
||||
r'lovehomeporn\.com/embed/(\d+)', webpage, 'video id')
|
||||
|
||||
title = self._search_regex(
|
||||
r'<h2[^>]+title=(["\'])(?P<url>.+?)\1',
|
||||
webpage, 'title', group='url')
|
||||
description = self._html_search_meta('description', webpage)
|
||||
|
||||
info = self._extract_nuevo(
|
||||
'http://lovehomeporn.com/media/nuevo/econfig.php?key=%s&rp=true' % video_id,
|
||||
video_id)
|
||||
info.update({
|
||||
'display_id': display_id,
|
||||
'title': title,
|
||||
'description': description,
|
||||
'age_limit': 18
|
||||
})
|
||||
return info
|
||||
@ -46,8 +46,12 @@ class ParamountNetworkIE(MTVServicesInfoExtractor):
|
||||
_GEO_COUNTRIES = ['US']
|
||||
|
||||
def _extract_mgid(self, webpage):
|
||||
cs = self._parse_json(self._search_regex(
|
||||
root_data = self._parse_json(self._search_regex(
|
||||
r'window\.__DATA__\s*=\s*({.+})',
|
||||
webpage, 'data'), None)['children']
|
||||
c = next(c for c in cs if c.get('type') == 'VideoPlayer')
|
||||
webpage, 'data'), None)
|
||||
|
||||
def find_sub_data(data, data_type):
|
||||
return next(c for c in data['children'] if c.get('type') == data_type)
|
||||
|
||||
c = find_sub_data(find_sub_data(root_data, 'MainContainer'), 'VideoPlayer')
|
||||
return c['props']['media']['video']['config']['uri']
|
||||
|
||||
@ -209,7 +209,7 @@ class ThePlatformIE(ThePlatformBaseIE, AdobePassIE):
|
||||
return [m.group('url')]
|
||||
|
||||
# Are whitesapces ignored in URLs?
|
||||
# https://github.com/rg3/youtube-dl/issues/12044
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/12044
|
||||
matches = re.findall(
|
||||
r'(?s)<(?:iframe|script)[^>]+src=(["\'])((?:https?:)?//player\.theplatform\.com/p/.+?)\1', webpage)
|
||||
if matches:
|
||||
@ -271,7 +271,7 @@ class ThePlatformIE(ThePlatformBaseIE, AdobePassIE):
|
||||
|
||||
if smuggled_data.get('force_smil_url', False):
|
||||
smil_url = url
|
||||
# Explicitly specified SMIL (see https://github.com/rg3/youtube-dl/issues/7385)
|
||||
# Explicitly specified SMIL (see https://github.com/ytdl-org/youtube-dl/issues/7385)
|
||||
elif '/guid/' in url:
|
||||
headers = {}
|
||||
source_url = smuggled_data.get('source_url')
|
||||
|
||||
@ -1,14 +1,14 @@
|
||||
# coding: utf-8
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import re
|
||||
import json
|
||||
|
||||
from .radiocanada import RadioCanadaIE
|
||||
from ..compat import compat_HTTPError
|
||||
from ..utils import (
|
||||
extract_attributes,
|
||||
ExtractorError,
|
||||
int_or_none,
|
||||
merge_dicts,
|
||||
urlencode_postdata,
|
||||
)
|
||||
|
||||
|
||||
@ -38,54 +38,37 @@ class TouTvIE(RadioCanadaIE):
|
||||
'url': 'https://ici.tou.tv/l-age-adulte/S01C501',
|
||||
'only_matching': True,
|
||||
}]
|
||||
_CLIENT_KEY = '4dd36440-09d5-4468-8923-b6d91174ad36'
|
||||
|
||||
def _real_initialize(self):
|
||||
email, password = self._get_login_info()
|
||||
if email is None:
|
||||
return
|
||||
login_webpage = self._download_webpage(
|
||||
'https://services.radio-canada.ca/auth/oauth/v2/authorize',
|
||||
None, 'Downloading login page', query={
|
||||
'client_id': '4dd36440-09d5-4468-8923-b6d91174ad36',
|
||||
'redirect_uri': 'https://ici.tou.tv/logincallback',
|
||||
'response_type': 'token',
|
||||
'scope': 'id.write media-validation.read',
|
||||
'state': '/',
|
||||
})
|
||||
|
||||
def extract_form_url_and_data(wp, default_form_url, form_spec_re=''):
|
||||
form, form_elem = re.search(
|
||||
r'(?s)((<form[^>]+?%s[^>]*?>).+?</form>)' % form_spec_re, wp).groups()
|
||||
form_data = self._hidden_inputs(form)
|
||||
form_url = extract_attributes(form_elem).get('action') or default_form_url
|
||||
return form_url, form_data
|
||||
|
||||
post_url, form_data = extract_form_url_and_data(
|
||||
login_webpage,
|
||||
'https://services.radio-canada.ca/auth/oauth/v2/authorize/login',
|
||||
r'(?:id|name)="Form-login"')
|
||||
form_data.update({
|
||||
'login-email': email,
|
||||
'login-password': password,
|
||||
})
|
||||
consent_webpage = self._download_webpage(
|
||||
post_url, None, 'Logging in', data=urlencode_postdata(form_data))
|
||||
post_url, form_data = extract_form_url_and_data(
|
||||
consent_webpage,
|
||||
'https://services.radio-canada.ca/auth/oauth/v2/authorize/consent')
|
||||
_, urlh = self._download_webpage_handle(
|
||||
post_url, None, 'Following Redirection',
|
||||
data=urlencode_postdata(form_data))
|
||||
self._access_token = self._search_regex(
|
||||
r'access_token=([\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12})',
|
||||
urlh.geturl(), 'access token')
|
||||
try:
|
||||
self._access_token = self._download_json(
|
||||
'https://services.radio-canada.ca/toutv/profiling/accounts/login',
|
||||
None, 'Logging in', data=json.dumps({
|
||||
'ClientId': self._CLIENT_KEY,
|
||||
'ClientSecret': '34026772-244b-49b6-8b06-317b30ac9a20',
|
||||
'Email': email,
|
||||
'Password': password,
|
||||
'Scope': 'id.write media-validation.read',
|
||||
}).encode(), headers={
|
||||
'Authorization': 'client-key ' + self._CLIENT_KEY,
|
||||
'Content-Type': 'application/json;charset=utf-8',
|
||||
})['access_token']
|
||||
except ExtractorError as e:
|
||||
if isinstance(e.cause, compat_HTTPError) and e.cause.code == 401:
|
||||
error = self._parse_json(e.cause.read().decode(), None)['Message']
|
||||
raise ExtractorError(error, expected=True)
|
||||
raise
|
||||
self._claims = self._call_api('validation/v2/getClaims')['claims']
|
||||
|
||||
def _real_extract(self, url):
|
||||
path = self._match_id(url)
|
||||
metadata = self._download_json('http://ici.tou.tv/presentation/%s' % path, path)
|
||||
# IsDrm does not necessarily mean the video is DRM protected (see
|
||||
# https://github.com/rg3/youtube-dl/issues/13994).
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/13994).
|
||||
if metadata.get('IsDrm'):
|
||||
self.report_warning('This video is probably DRM protected.', path)
|
||||
video_id = metadata['IdMedia']
|
||||
|
||||
@ -375,7 +375,7 @@ class UdemyIE(InfoExtractor):
|
||||
}, res))
|
||||
|
||||
# react rendition since 2017.04.15 (see
|
||||
# https://github.com/rg3/youtube-dl/issues/12744)
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/12744)
|
||||
data = self._parse_json(
|
||||
self._search_regex(
|
||||
r'videojs-setup-data=(["\'])(?P<data>{.+?})\1', view_html,
|
||||
|
||||
@ -2,18 +2,31 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import unified_timestamp
|
||||
|
||||
|
||||
class URPlayIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?ur(?:play|skola)\.se/(?:program|Produkter)/(?P<id>[0-9]+)'
|
||||
_TESTS = [{
|
||||
'url': 'http://urplay.se/program/190031-tripp-trapp-trad-sovkudde',
|
||||
'md5': 'ad5f0de86f16ca4c8062cd103959a9eb',
|
||||
'url': 'https://urplay.se/program/203704-ur-samtiden-livet-universum-och-rymdens-markliga-musik-om-vetenskap-kritiskt-tankande-och-motstand',
|
||||
'md5': 'ff5b0c89928f8083c74bbd5099c9292d',
|
||||
'info_dict': {
|
||||
'id': '203704',
|
||||
'ext': 'mp4',
|
||||
'title': 'UR Samtiden - Livet, universum och rymdens märkliga musik : Om vetenskap, kritiskt tänkande och motstånd',
|
||||
'description': 'md5:5344508a52aa78c1ced6c1b8b9e44e9a',
|
||||
'timestamp': 1513512768,
|
||||
'upload_date': '20171217',
|
||||
},
|
||||
}, {
|
||||
'url': 'https://urskola.se/Produkter/190031-Tripp-Trapp-Trad-Sovkudde',
|
||||
'info_dict': {
|
||||
'id': '190031',
|
||||
'ext': 'mp4',
|
||||
'title': 'Tripp, Trapp, Träd : Sovkudde',
|
||||
'description': 'md5:b86bffdae04a7e9379d1d7e5947df1d1',
|
||||
'timestamp': 1440093600,
|
||||
'upload_date': '20150820',
|
||||
},
|
||||
}, {
|
||||
'url': 'http://urskola.se/Produkter/155794-Smasagor-meankieli-Grodan-i-vida-varlden',
|
||||
@ -51,6 +64,7 @@ class URPlayIE(InfoExtractor):
|
||||
'title': urplayer_data['title'],
|
||||
'description': self._og_search_description(webpage),
|
||||
'thumbnail': urplayer_data.get('image'),
|
||||
'timestamp': unified_timestamp(self._html_search_meta(('uploadDate', 'schema:uploadDate'), webpage, 'timestamp')),
|
||||
'series': urplayer_data.get('series_title'),
|
||||
'subtitles': subtitles,
|
||||
'formats': formats,
|
||||
|
||||
@ -165,7 +165,7 @@ class UstreamIE(InfoExtractor):
|
||||
m = re.match(self._VALID_URL, url)
|
||||
video_id = m.group('id')
|
||||
|
||||
# some sites use this embed format (see: https://github.com/rg3/youtube-dl/issues/2990)
|
||||
# some sites use this embed format (see: https://github.com/ytdl-org/youtube-dl/issues/2990)
|
||||
if m.group('type') == 'embed/recorded':
|
||||
video_id = m.group('id')
|
||||
desktop_url = 'http://www.ustream.tv/recorded/' + video_id
|
||||
|
||||
@ -54,7 +54,7 @@ class VeeHDIE(InfoExtractor):
|
||||
video_id = self._match_id(url)
|
||||
|
||||
# VeeHD seems to send garbage on the first request.
|
||||
# See https://github.com/rg3/youtube-dl/issues/2102
|
||||
# See https://github.com/ytdl-org/youtube-dl/issues/2102
|
||||
self._download_webpage(url, video_id, 'Requesting webpage')
|
||||
webpage = self._download_webpage(url, video_id)
|
||||
|
||||
|
||||
@ -1,13 +1,10 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import re
|
||||
import json
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
int_or_none,
|
||||
ExtractorError,
|
||||
sanitized_Request,
|
||||
parse_duration,
|
||||
qualities,
|
||||
)
|
||||
|
||||
|
||||
@ -16,9 +13,9 @@ class VeohIE(InfoExtractor):
|
||||
|
||||
_TESTS = [{
|
||||
'url': 'http://www.veoh.com/watch/v56314296nk7Zdmz3',
|
||||
'md5': '620e68e6a3cff80086df3348426c9ca3',
|
||||
'md5': '9e7ecc0fd8bbee7a69fe38953aeebd30',
|
||||
'info_dict': {
|
||||
'id': '56314296',
|
||||
'id': 'v56314296nk7Zdmz3',
|
||||
'ext': 'mp4',
|
||||
'title': 'Straight Backs Are Stronger',
|
||||
'uploader': 'LUMOback',
|
||||
@ -56,29 +53,6 @@ class VeohIE(InfoExtractor):
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
def _extract_formats(self, source):
|
||||
formats = []
|
||||
link = source.get('aowPermalink')
|
||||
if link:
|
||||
formats.append({
|
||||
'url': link,
|
||||
'ext': 'mp4',
|
||||
'format_id': 'aow',
|
||||
})
|
||||
link = source.get('fullPreviewHashLowPath')
|
||||
if link:
|
||||
formats.append({
|
||||
'url': link,
|
||||
'format_id': 'low',
|
||||
})
|
||||
link = source.get('fullPreviewHashHighPath')
|
||||
if link:
|
||||
formats.append({
|
||||
'url': link,
|
||||
'format_id': 'high',
|
||||
})
|
||||
return formats
|
||||
|
||||
def _extract_video(self, source):
|
||||
return {
|
||||
'id': source.get('videoId'),
|
||||
@ -93,38 +67,37 @@ class VeohIE(InfoExtractor):
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
mobj = re.match(self._VALID_URL, url)
|
||||
video_id = mobj.group('id')
|
||||
video_id = self._match_id(url)
|
||||
video = self._download_json(
|
||||
'https://www.veoh.com/watch/getVideo/' + video_id,
|
||||
video_id)['video']
|
||||
title = video['title']
|
||||
|
||||
if video_id.startswith('v'):
|
||||
rsp = self._download_xml(
|
||||
r'http://www.veoh.com/api/findByPermalink?permalink=%s' % video_id, video_id, 'Downloading video XML')
|
||||
stat = rsp.get('stat')
|
||||
if stat == 'ok':
|
||||
return self._extract_video(rsp.find('./videoList/video'))
|
||||
elif stat == 'fail':
|
||||
raise ExtractorError(
|
||||
'%s said: %s' % (self.IE_NAME, rsp.find('./errorList/error').get('errorMessage')), expected=True)
|
||||
thumbnail_url = None
|
||||
q = qualities(['HQ', 'Regular'])
|
||||
formats = []
|
||||
for f_id, f_url in video.get('src', {}).items():
|
||||
if not f_url:
|
||||
continue
|
||||
if f_id == 'poster':
|
||||
thumbnail_url = f_url
|
||||
else:
|
||||
formats.append({
|
||||
'format_id': f_id,
|
||||
'quality': q(f_id),
|
||||
'url': f_url,
|
||||
})
|
||||
self._sort_formats(formats)
|
||||
|
||||
webpage = self._download_webpage(url, video_id)
|
||||
age_limit = 0
|
||||
if 'class="adultwarning-container"' in webpage:
|
||||
self.report_age_confirmation()
|
||||
age_limit = 18
|
||||
request = sanitized_Request(url)
|
||||
request.add_header('Cookie', 'confirmedAdult=true')
|
||||
webpage = self._download_webpage(request, video_id)
|
||||
|
||||
m_youtube = re.search(r'http://www\.youtube\.com/v/(.*?)(\&|"|\?)', webpage)
|
||||
if m_youtube is not None:
|
||||
youtube_id = m_youtube.group(1)
|
||||
self.to_screen('%s: detected Youtube video.' % video_id)
|
||||
return self.url_result(youtube_id, 'Youtube')
|
||||
|
||||
info = json.loads(
|
||||
self._search_regex(r'videoDetailsJSON = \'({.*?})\';', webpage, 'info').replace('\\\'', '\''))
|
||||
|
||||
video = self._extract_video(info)
|
||||
video['age_limit'] = age_limit
|
||||
|
||||
return video
|
||||
return {
|
||||
'id': video_id,
|
||||
'title': title,
|
||||
'description': video.get('description'),
|
||||
'thumbnail': thumbnail_url,
|
||||
'uploader': video.get('author', {}).get('nickname'),
|
||||
'duration': int_or_none(video.get('lengthBySec')) or parse_duration(video.get('length')),
|
||||
'view_count': int_or_none(video.get('views')),
|
||||
'formats': formats,
|
||||
'average_rating': int_or_none(video.get('rating')),
|
||||
'comment_count': int_or_none(video.get('numOfComments')),
|
||||
}
|
||||
|
||||
@ -200,7 +200,7 @@ class VevoIE(VevoBaseIE):
|
||||
fatal=False)
|
||||
|
||||
# Some videos are only available via webpage (e.g.
|
||||
# https://github.com/rg3/youtube-dl/issues/9366)
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/9366)
|
||||
if not video_versions:
|
||||
webpage = self._download_webpage(url, video_id)
|
||||
json_data = self._extract_json(webpage, video_id)
|
||||
|
||||
@ -1,19 +1,16 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import re
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..compat import (
|
||||
compat_urllib_parse_urlencode,
|
||||
compat_urlparse,
|
||||
)
|
||||
from ..utils import (
|
||||
float_or_none,
|
||||
int_or_none,
|
||||
sanitized_Request,
|
||||
)
|
||||
|
||||
|
||||
class ViddlerIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?viddler\.com/(?:v|embed|player)/(?P<id>[a-z0-9]+)'
|
||||
_VALID_URL = r'https?://(?:www\.)?viddler\.com/(?:v|embed|player)/(?P<id>[a-z0-9]+)(?:.+?\bsecret=(\d+))?'
|
||||
_TESTS = [{
|
||||
'url': 'http://www.viddler.com/v/43903784',
|
||||
'md5': '9eee21161d2c7f5b39690c3e325fab2f',
|
||||
@ -78,23 +75,18 @@ class ViddlerIE(InfoExtractor):
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
video_id, secret = re.match(self._VALID_URL, url).groups()
|
||||
|
||||
query = {
|
||||
'video_id': video_id,
|
||||
'key': 'v0vhrt7bg2xq1vyxhkct',
|
||||
}
|
||||
|
||||
qs = compat_urlparse.parse_qs(compat_urlparse.urlparse(url).query)
|
||||
secret = qs.get('secret', [None])[0]
|
||||
if secret:
|
||||
query['secret'] = secret
|
||||
|
||||
headers = {'Referer': 'http://static.cdn-ec.viddler.com/js/arpeggio/v2/embed.html'}
|
||||
request = sanitized_Request(
|
||||
'http://api.viddler.com/api/v2/viddler.videos.getPlaybackDetails.json?%s'
|
||||
% compat_urllib_parse_urlencode(query), None, headers)
|
||||
data = self._download_json(request, video_id)['video']
|
||||
data = self._download_json(
|
||||
'http://api.viddler.com/api/v2/viddler.videos.getPlaybackDetails.json',
|
||||
video_id, headers={'Referer': url}, query=query)['video']
|
||||
|
||||
formats = []
|
||||
for filed in data['files']:
|
||||
|
||||
@ -1,60 +0,0 @@
|
||||
# coding: utf-8
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import re
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
decode_packed_codes,
|
||||
sanitized_Request,
|
||||
)
|
||||
|
||||
|
||||
class VideoMegaIE(InfoExtractor):
|
||||
_VALID_URL = r'(?:videomega:|https?://(?:www\.)?videomega\.tv/(?:(?:view|iframe|cdn)\.php)?\?ref=)(?P<id>[A-Za-z0-9]+)'
|
||||
_TESTS = [{
|
||||
'url': 'http://videomega.tv/cdn.php?ref=AOSQBJYKIDDIKYJBQSOA',
|
||||
'md5': 'cc1920a58add3f05c6a93285b84fb3aa',
|
||||
'info_dict': {
|
||||
'id': 'AOSQBJYKIDDIKYJBQSOA',
|
||||
'ext': 'mp4',
|
||||
'title': '1254207',
|
||||
'thumbnail': r're:^https?://.*\.jpg$',
|
||||
}
|
||||
}, {
|
||||
'url': 'http://videomega.tv/cdn.php?ref=AOSQBJYKIDDIKYJBQSOA&width=1070&height=600',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'http://videomega.tv/view.php?ref=090051111052065112106089103052052103089106112065052111051090',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
|
||||
iframe_url = 'http://videomega.tv/cdn.php?ref=%s' % video_id
|
||||
req = sanitized_Request(iframe_url)
|
||||
req.add_header('Referer', url)
|
||||
req.add_header('Cookie', 'noadvtday=0')
|
||||
webpage = self._download_webpage(req, video_id)
|
||||
|
||||
title = self._html_search_regex(
|
||||
r'<title>(.+?)</title>', webpage, 'title')
|
||||
title = re.sub(
|
||||
r'(?:^[Vv]ideo[Mm]ega\.tv\s-\s*|\s*-\svideomega\.tv$)', '', title)
|
||||
thumbnail = self._search_regex(
|
||||
r'<video[^>]+?poster="([^"]+)"', webpage, 'thumbnail', fatal=False)
|
||||
|
||||
real_codes = decode_packed_codes(webpage)
|
||||
video_url = self._search_regex(
|
||||
r'"src"\s*,\s*"([^"]+)"', real_codes, 'video URL')
|
||||
|
||||
return {
|
||||
'id': video_id,
|
||||
'title': title,
|
||||
'url': video_url,
|
||||
'thumbnail': thumbnail,
|
||||
'http_headers': {
|
||||
'Referer': iframe_url,
|
||||
},
|
||||
}
|
||||
@ -195,6 +195,32 @@ class VimeoBaseInfoExtractor(InfoExtractor):
|
||||
'subtitles': subtitles,
|
||||
}
|
||||
|
||||
def _extract_original_format(self, url, video_id):
|
||||
download_data = self._download_json(
|
||||
url, video_id, fatal=False,
|
||||
query={'action': 'load_download_config'},
|
||||
headers={'X-Requested-With': 'XMLHttpRequest'})
|
||||
if download_data:
|
||||
source_file = download_data.get('source_file')
|
||||
if isinstance(source_file, dict):
|
||||
download_url = source_file.get('download_url')
|
||||
if download_url and not source_file.get('is_cold') and not source_file.get('is_defrosting'):
|
||||
source_name = source_file.get('public_name', 'Original')
|
||||
if self._is_valid_url(download_url, video_id, '%s video' % source_name):
|
||||
ext = (try_get(
|
||||
source_file, lambda x: x['extension'],
|
||||
compat_str) or determine_ext(
|
||||
download_url, None) or 'mp4').lower()
|
||||
return {
|
||||
'url': download_url,
|
||||
'ext': ext,
|
||||
'width': int_or_none(source_file.get('width')),
|
||||
'height': int_or_none(source_file.get('height')),
|
||||
'filesize': parse_filesize(source_file.get('size')),
|
||||
'format_id': source_name,
|
||||
'preference': 1,
|
||||
}
|
||||
|
||||
|
||||
class VimeoIE(VimeoBaseInfoExtractor):
|
||||
"""Information extractor for vimeo.com."""
|
||||
@ -505,7 +531,7 @@ class VimeoIE(VimeoBaseInfoExtractor):
|
||||
orig_url = url
|
||||
if mobj.group('pro'):
|
||||
# some videos require portfolio_id to be present in player url
|
||||
# https://github.com/rg3/youtube-dl/issues/20070
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/20070
|
||||
url = self._extract_url(url, self._download_webpage(url, video_id))
|
||||
elif mobj.group('player'):
|
||||
url = 'https://player.vimeo.com/video/' + video_id
|
||||
@ -560,7 +586,7 @@ class VimeoIE(VimeoBaseInfoExtractor):
|
||||
if not config_url:
|
||||
# Sometimes new react-based page is served instead of old one that require
|
||||
# different config URL extraction approach (see
|
||||
# https://github.com/rg3/youtube-dl/pull/7209)
|
||||
# https://github.com/ytdl-org/youtube-dl/pull/7209)
|
||||
vimeo_clip_page_config = self._search_regex(
|
||||
r'vimeo\.clip_page_config\s*=\s*({.+?});', webpage,
|
||||
'vimeo clip page config')
|
||||
@ -665,29 +691,11 @@ class VimeoIE(VimeoBaseInfoExtractor):
|
||||
comment_count = None
|
||||
|
||||
formats = []
|
||||
download_request = sanitized_Request('https://vimeo.com/%s?action=load_download_config' % video_id, headers={
|
||||
'X-Requested-With': 'XMLHttpRequest'})
|
||||
download_data = self._download_json(download_request, video_id, fatal=False)
|
||||
if download_data:
|
||||
source_file = download_data.get('source_file')
|
||||
if isinstance(source_file, dict):
|
||||
download_url = source_file.get('download_url')
|
||||
if download_url and not source_file.get('is_cold') and not source_file.get('is_defrosting'):
|
||||
source_name = source_file.get('public_name', 'Original')
|
||||
if self._is_valid_url(download_url, video_id, '%s video' % source_name):
|
||||
ext = (try_get(
|
||||
source_file, lambda x: x['extension'],
|
||||
compat_str) or determine_ext(
|
||||
download_url, None) or 'mp4').lower()
|
||||
formats.append({
|
||||
'url': download_url,
|
||||
'ext': ext,
|
||||
'width': int_or_none(source_file.get('width')),
|
||||
'height': int_or_none(source_file.get('height')),
|
||||
'filesize': parse_filesize(source_file.get('size')),
|
||||
'format_id': source_name,
|
||||
'preference': 1,
|
||||
})
|
||||
|
||||
source_format = self._extract_original_format(
|
||||
'https://vimeo.com/' + video_id, video_id)
|
||||
if source_format:
|
||||
formats.append(source_format)
|
||||
|
||||
info_dict_config = self._parse_config(config, video_id)
|
||||
formats.extend(info_dict_config['formats'])
|
||||
@ -946,7 +954,7 @@ class VimeoGroupsIE(VimeoAlbumIE):
|
||||
class VimeoReviewIE(VimeoBaseInfoExtractor):
|
||||
IE_NAME = 'vimeo:review'
|
||||
IE_DESC = 'Review pages on vimeo'
|
||||
_VALID_URL = r'https://vimeo\.com/[^/]+/review/(?P<id>[^/]+)'
|
||||
_VALID_URL = r'(?P<url>https://vimeo\.com/[^/]+/review/(?P<id>[^/]+)/[0-9a-f]{10})'
|
||||
_TESTS = [{
|
||||
'url': 'https://vimeo.com/user21297594/review/75524534/3c257a1b5d',
|
||||
'md5': 'c507a72f780cacc12b2248bb4006d253',
|
||||
@ -998,7 +1006,8 @@ class VimeoReviewIE(VimeoBaseInfoExtractor):
|
||||
data = self._parse_json(self._search_regex(
|
||||
r'window\s*=\s*_extend\(window,\s*({.+?})\);', webpage, 'data',
|
||||
default=NO_DEFAULT if video_password_verified else '{}'), video_id)
|
||||
config_url = data.get('vimeo_esi', {}).get('config', {}).get('configUrl')
|
||||
config = data.get('vimeo_esi', {}).get('config', {})
|
||||
config_url = config.get('configUrl') or try_get(config, lambda x: x['clipData']['configUrl'])
|
||||
if config_url is None:
|
||||
self._verify_video_password(webpage_url, video_id, webpage)
|
||||
config_url = self._get_config_url(
|
||||
@ -1006,10 +1015,13 @@ class VimeoReviewIE(VimeoBaseInfoExtractor):
|
||||
return config_url
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
page_url, video_id = re.match(self._VALID_URL, url).groups()
|
||||
config_url = self._get_config_url(url, video_id)
|
||||
config = self._download_json(config_url, video_id)
|
||||
info_dict = self._parse_config(config, video_id)
|
||||
source_format = self._extract_original_format(page_url, video_id)
|
||||
if source_format:
|
||||
info_dict['formats'].append(source_format)
|
||||
self._vimeo_sort_formats(info_dict['formats'])
|
||||
info_dict['id'] = video_id
|
||||
return info_dict
|
||||
|
||||
@ -50,7 +50,7 @@ class VKBaseIE(InfoExtractor):
|
||||
|
||||
# https://new.vk.com/ serves two same remixlhk cookies in Set-Cookie header
|
||||
# and expects the first one to be set rather than second (see
|
||||
# https://github.com/rg3/youtube-dl/issues/9841#issuecomment-227871201).
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/9841#issuecomment-227871201).
|
||||
# As of RFC6265 the newer one cookie should be set into cookie store
|
||||
# what actually happens.
|
||||
# We will workaround this VK issue by resetting the remixlhk cookie to
|
||||
|
||||
@ -239,7 +239,7 @@ class VLiveChannelIE(InfoExtractor):
|
||||
# Large values of maxNumOfRows (~300 or above) may cause
|
||||
# empty responses (see [1]), e.g. this happens for [2] that
|
||||
# has more than 300 videos.
|
||||
# 1. https://github.com/rg3/youtube-dl/issues/13830
|
||||
# 1. https://github.com/ytdl-org/youtube-dl/issues/13830
|
||||
# 2. http://channels.vlive.tv/EDBF.
|
||||
'maxNumOfRows': 100,
|
||||
'_': int(time.time()),
|
||||
|
||||
@ -76,7 +76,7 @@ class YandexMusicTrackIE(YandexMusicBaseIE):
|
||||
track_id, 'Downloading track location JSON')
|
||||
|
||||
# Each string is now wrapped in a list, this is probably only temporarily thus
|
||||
# supporting both scenarios (see https://github.com/rg3/youtube-dl/issues/10193)
|
||||
# supporting both scenarios (see https://github.com/ytdl-org/youtube-dl/issues/10193)
|
||||
for k, v in data.items():
|
||||
if v and isinstance(v, list):
|
||||
data[k] = v[0]
|
||||
@ -200,7 +200,7 @@ class YandexMusicPlaylistIE(YandexMusicPlaylistBaseIE):
|
||||
'skip': 'Travis CI servers blocked by YandexMusic',
|
||||
}, {
|
||||
# playlist exceeding the limit of 150 tracks shipped with webpage (see
|
||||
# https://github.com/rg3/youtube-dl/issues/6666)
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/6666)
|
||||
'url': 'https://music.yandex.ru/users/ya.playlist/playlists/1036',
|
||||
'info_dict': {
|
||||
'id': '1036',
|
||||
|
||||
90
youtube_dl/extractor/yandexvideo.py
Normal file
90
youtube_dl/extractor/yandexvideo.py
Normal file
@ -0,0 +1,90 @@
|
||||
# coding: utf-8
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
int_or_none,
|
||||
url_or_none,
|
||||
)
|
||||
|
||||
|
||||
class YandexVideoIE(InfoExtractor):
|
||||
_VALID_URL = r'''(?x)
|
||||
https?://
|
||||
(?:
|
||||
yandex\.ru(?:/portal/(?:video|efir))?/?\?.*?stream_id=|
|
||||
frontend\.vh\.yandex\.ru/player/
|
||||
)
|
||||
(?P<id>[\da-f]+)
|
||||
'''
|
||||
_TESTS = [{
|
||||
'url': 'https://yandex.ru/portal/video?stream_id=4dbb262b4fe5cf15a215de4f34eee34d',
|
||||
'md5': '33955d7ae052f15853dc41f35f17581c',
|
||||
'info_dict': {
|
||||
'id': '4dbb262b4fe5cf15a215de4f34eee34d',
|
||||
'ext': 'mp4',
|
||||
'title': 'В Нью-Йорке баржи и теплоход оторвались от причала и расплылись по Гудзону',
|
||||
'description': '',
|
||||
'thumbnail': r're:^https?://.*\.jpg$',
|
||||
'timestamp': 0,
|
||||
'duration': 30,
|
||||
'age_limit': 18,
|
||||
},
|
||||
}, {
|
||||
'url': 'https://yandex.ru/portal/efir?stream_id=4dbb36ec4e0526d58f9f2dc8f0ecf374&from=morda',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://yandex.ru/?stream_id=4dbb262b4fe5cf15a215de4f34eee34d',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://frontend.vh.yandex.ru/player/4dbb262b4fe5cf15a215de4f34eee34d?from=morda',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
# vod-episode, series episode
|
||||
'url': 'https://yandex.ru/portal/video?stream_id=45b11db6e4b68797919c93751a938cee',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
# episode, sports
|
||||
'url': 'https://yandex.ru/?stream_channel=1538487871&stream_id=4132a07f71fb0396be93d74b3477131d',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
|
||||
content = self._download_json(
|
||||
'https://frontend.vh.yandex.ru/v22/player/%s.json' % video_id,
|
||||
video_id, query={
|
||||
'stream_options': 'hires',
|
||||
'disable_trackings': 1,
|
||||
})['content']
|
||||
|
||||
m3u8_url = url_or_none(content.get('content_url')) or url_or_none(
|
||||
content['streams'][0]['url'])
|
||||
title = content.get('title') or content.get('computed_title')
|
||||
|
||||
formats = self._extract_m3u8_formats(
|
||||
m3u8_url, video_id, 'mp4', entry_protocol='m3u8_native',
|
||||
m3u8_id='hls')
|
||||
self._sort_formats(formats)
|
||||
|
||||
description = content.get('description')
|
||||
thumbnail = content.get('thumbnail')
|
||||
timestamp = (int_or_none(content.get('release_date')) or
|
||||
int_or_none(content.get('release_date_ut')) or
|
||||
int_or_none(content.get('start_time')))
|
||||
duration = int_or_none(content.get('duration'))
|
||||
series = content.get('program_title')
|
||||
age_limit = int_or_none(content.get('restriction_age'))
|
||||
|
||||
return {
|
||||
'id': video_id,
|
||||
'title': title,
|
||||
'description': description,
|
||||
'thumbnail': thumbnail,
|
||||
'timestamp': timestamp,
|
||||
'duration': duration,
|
||||
'series': series,
|
||||
'age_limit': age_limit,
|
||||
'formats': formats,
|
||||
}
|
||||
@ -351,7 +351,8 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
(?:www\.)?hooktube\.com/|
|
||||
(?:www\.)?yourepeat\.com/|
|
||||
tube\.majestyc\.net/|
|
||||
(?:www\.)?invidio\.us/|
|
||||
(?:(?:www|dev)\.)?invidio\.us/|
|
||||
(?:www\.)?invidiou\.sh/|
|
||||
(?:www\.)?invidious\.snopyta\.org/|
|
||||
(?:www\.)?invidious\.kabi\.tk/|
|
||||
(?:www\.)?vid\.wxzm\.sx/|
|
||||
@ -430,7 +431,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
'135': {'ext': 'mp4', 'height': 480, 'format_note': 'DASH video', 'vcodec': 'h264'},
|
||||
'136': {'ext': 'mp4', 'height': 720, 'format_note': 'DASH video', 'vcodec': 'h264'},
|
||||
'137': {'ext': 'mp4', 'height': 1080, 'format_note': 'DASH video', 'vcodec': 'h264'},
|
||||
'138': {'ext': 'mp4', 'format_note': 'DASH video', 'vcodec': 'h264'}, # Height can vary (https://github.com/rg3/youtube-dl/issues/4559)
|
||||
'138': {'ext': 'mp4', 'format_note': 'DASH video', 'vcodec': 'h264'}, # Height can vary (https://github.com/ytdl-org/youtube-dl/issues/4559)
|
||||
'160': {'ext': 'mp4', 'height': 144, 'format_note': 'DASH video', 'vcodec': 'h264'},
|
||||
'212': {'ext': 'mp4', 'height': 480, 'format_note': 'DASH video', 'vcodec': 'h264'},
|
||||
'264': {'ext': 'mp4', 'height': 1440, 'format_note': 'DASH video', 'vcodec': 'h264'},
|
||||
@ -695,7 +696,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
'age_limit': 18,
|
||||
},
|
||||
},
|
||||
# video_info is None (https://github.com/rg3/youtube-dl/issues/4421)
|
||||
# video_info is None (https://github.com/ytdl-org/youtube-dl/issues/4421)
|
||||
# YouTube Red ad is not captured for creator
|
||||
{
|
||||
'url': '__2ABJjxzNo',
|
||||
@ -716,7 +717,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
'DASH manifest missing',
|
||||
]
|
||||
},
|
||||
# Olympics (https://github.com/rg3/youtube-dl/issues/4431)
|
||||
# Olympics (https://github.com/ytdl-org/youtube-dl/issues/4431)
|
||||
{
|
||||
'url': 'lqQg6PlCWgI',
|
||||
'info_dict': {
|
||||
@ -767,7 +768,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
},
|
||||
'skip': 'This live event has ended.',
|
||||
},
|
||||
# Extraction from multiple DASH manifests (https://github.com/rg3/youtube-dl/pull/6097)
|
||||
# Extraction from multiple DASH manifests (https://github.com/ytdl-org/youtube-dl/pull/6097)
|
||||
{
|
||||
'url': 'https://www.youtube.com/watch?v=FIl7x6_3R5Y',
|
||||
'info_dict': {
|
||||
@ -870,7 +871,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
'skip': 'This video is not available.',
|
||||
},
|
||||
{
|
||||
# Multifeed video with comma in title (see https://github.com/rg3/youtube-dl/issues/8536)
|
||||
# Multifeed video with comma in title (see https://github.com/ytdl-org/youtube-dl/issues/8536)
|
||||
'url': 'https://www.youtube.com/watch?v=gVfLd0zydlo',
|
||||
'info_dict': {
|
||||
'id': 'gVfLd0zydlo',
|
||||
@ -888,10 +889,10 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
'only_matching': True,
|
||||
},
|
||||
{
|
||||
# Title with JS-like syntax "};" (see https://github.com/rg3/youtube-dl/issues/7468)
|
||||
# Title with JS-like syntax "};" (see https://github.com/ytdl-org/youtube-dl/issues/7468)
|
||||
# Also tests cut-off URL expansion in video description (see
|
||||
# https://github.com/rg3/youtube-dl/issues/1892,
|
||||
# https://github.com/rg3/youtube-dl/issues/8164)
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/1892,
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/8164)
|
||||
'url': 'https://www.youtube.com/watch?v=lsguqyKfVQg',
|
||||
'info_dict': {
|
||||
'id': 'lsguqyKfVQg',
|
||||
@ -913,7 +914,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
},
|
||||
},
|
||||
{
|
||||
# Tags with '};' (see https://github.com/rg3/youtube-dl/issues/7468)
|
||||
# Tags with '};' (see https://github.com/ytdl-org/youtube-dl/issues/7468)
|
||||
'url': 'https://www.youtube.com/watch?v=Ms7iBXnlUO8',
|
||||
'only_matching': True,
|
||||
},
|
||||
@ -977,7 +978,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
'only_matching': True,
|
||||
},
|
||||
{
|
||||
# YouTube Red paid video (https://github.com/rg3/youtube-dl/issues/10059)
|
||||
# YouTube Red paid video (https://github.com/ytdl-org/youtube-dl/issues/10059)
|
||||
'url': 'https://www.youtube.com/watch?v=i1Ko8UG-Tdo',
|
||||
'only_matching': True,
|
||||
},
|
||||
@ -1283,8 +1284,8 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
# regex won't capture the whole JSON. Yet working around by trying more
|
||||
# concrete regex first keeping in mind proper quoted string handling
|
||||
# to be implemented in future that will replace this workaround (see
|
||||
# https://github.com/rg3/youtube-dl/issues/7468,
|
||||
# https://github.com/rg3/youtube-dl/pull/7599)
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/7468,
|
||||
# https://github.com/ytdl-org/youtube-dl/pull/7599)
|
||||
r';ytplayer\.config\s*=\s*({.+?});ytplayer',
|
||||
r';ytplayer\.config\s*=\s*({.+?});',
|
||||
)
|
||||
@ -1599,7 +1600,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
add_dash_mpd(video_info)
|
||||
# Rental video is not rented but preview is available (e.g.
|
||||
# https://www.youtube.com/watch?v=yYr8q0y5Jfg,
|
||||
# https://github.com/rg3/youtube-dl/issues/10532)
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/10532)
|
||||
if not video_info and args.get('ypc_vid'):
|
||||
return self.url_result(
|
||||
args['ypc_vid'], YoutubeIE.ie_key(), video_id=args['ypc_vid'])
|
||||
@ -1619,7 +1620,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
# are missing from DASH manifest pointed by webpage's dashmpd, some - from DASH
|
||||
# manifest pointed by get_video_info's dashmpd).
|
||||
# The general idea is to take a union of itags of both DASH manifests (for example
|
||||
# video with such 'manifest behavior' see https://github.com/rg3/youtube-dl/issues/6093)
|
||||
# video with such 'manifest behavior' see https://github.com/ytdl-org/youtube-dl/issues/6093)
|
||||
self.report_video_info_webpage_download(video_id)
|
||||
for el in ('info', 'embedded', 'detailpage', 'vevo', ''):
|
||||
query = {
|
||||
@ -1654,7 +1655,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
if 'token' in get_video_info:
|
||||
# Different get_video_info requests may report different results, e.g.
|
||||
# some may report video unavailability, but some may serve it without
|
||||
# any complaint (see https://github.com/rg3/youtube-dl/issues/7362,
|
||||
# any complaint (see https://github.com/ytdl-org/youtube-dl/issues/7362,
|
||||
# the original webpage as well as el=info and el=embedded get_video_info
|
||||
# requests report video unavailability due to geo restriction while
|
||||
# el=detailpage succeeds and returns valid data). This is probably
|
||||
@ -1750,7 +1751,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
for feed in multifeed_metadata_list.split(','):
|
||||
# Unquote should take place before split on comma (,) since textual
|
||||
# fields may contain comma as well (see
|
||||
# https://github.com/rg3/youtube-dl/issues/8536)
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/8536)
|
||||
feed_data = compat_parse_qs(compat_urllib_parse_unquote_plus(feed))
|
||||
entries.append({
|
||||
'_type': 'url_transparent',
|
||||
@ -1775,7 +1776,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
|
||||
# Check for "rental" videos
|
||||
if 'ypc_video_rental_bar_text' in video_info and 'author' not in video_info:
|
||||
raise ExtractorError('"rental" videos not supported. See https://github.com/rg3/youtube-dl/issues/359 for more information.', expected=True)
|
||||
raise ExtractorError('"rental" videos not supported. See https://github.com/ytdl-org/youtube-dl/issues/359 for more information.', expected=True)
|
||||
|
||||
def _extract_filesize(media_url):
|
||||
return int_or_none(self._search_regex(
|
||||
@ -1792,7 +1793,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
elif not is_live and (len(video_info.get('url_encoded_fmt_stream_map', [''])[0]) >= 1 or len(video_info.get('adaptive_fmts', [''])[0]) >= 1):
|
||||
encoded_url_map = video_info.get('url_encoded_fmt_stream_map', [''])[0] + ',' + video_info.get('adaptive_fmts', [''])[0]
|
||||
if 'rtmpe%3Dyes' in encoded_url_map:
|
||||
raise ExtractorError('rtmpe downloads are not supported, see https://github.com/rg3/youtube-dl/issues/343 for more information.', expected=True)
|
||||
raise ExtractorError('rtmpe downloads are not supported, see https://github.com/ytdl-org/youtube-dl/issues/343 for more information.', expected=True)
|
||||
formats_spec = {}
|
||||
fmt_list = video_info.get('fmt_list', [''])[0]
|
||||
if fmt_list:
|
||||
@ -1904,7 +1905,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
dct.update(formats_spec[format_id])
|
||||
|
||||
# Some itags are not included in DASH manifest thus corresponding formats will
|
||||
# lack metadata (see https://github.com/rg3/youtube-dl/pull/5993).
|
||||
# lack metadata (see https://github.com/ytdl-org/youtube-dl/pull/5993).
|
||||
# Trying to extract metadata from url_encoded_fmt_stream_map entry.
|
||||
mobj = re.search(r'^(?P<width>\d+)[xX](?P<height>\d+)$', url_data.get('size', [''])[0])
|
||||
width, height = (int(mobj.group('width')), int(mobj.group('height'))) if mobj else (None, None)
|
||||
@ -2155,7 +2156,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
# Remove the formats we found through non-DASH, they
|
||||
# contain less info and it can be wrong, because we use
|
||||
# fixed values (for example the resolution). See
|
||||
# https://github.com/rg3/youtube-dl/issues/5774 for an
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/5774 for an
|
||||
# example.
|
||||
formats = [f for f in formats if f['format_id'] not in dash_formats.keys()]
|
||||
formats.extend(dash_formats.values())
|
||||
@ -2424,7 +2425,7 @@ class YoutubePlaylistIE(YoutubePlaylistBaseInfoExtractor):
|
||||
url = self._TEMPLATE_URL % playlist_id
|
||||
page = self._download_webpage(url, playlist_id)
|
||||
|
||||
# the yt-alert-message now has tabindex attribute (see https://github.com/rg3/youtube-dl/issues/11604)
|
||||
# the yt-alert-message now has tabindex attribute (see https://github.com/ytdl-org/youtube-dl/issues/11604)
|
||||
for match in re.findall(r'<div class="yt-alert-message"[^>]*>([^<]+)</div>', page):
|
||||
match = match.strip()
|
||||
# Check if the playlist exists or is private
|
||||
@ -2517,7 +2518,7 @@ class YoutubePlaylistIE(YoutubePlaylistBaseInfoExtractor):
|
||||
return playlist
|
||||
|
||||
# Some playlist URLs don't actually serve a playlist (see
|
||||
# https://github.com/rg3/youtube-dl/issues/10537).
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/10537).
|
||||
# Fallback to plain video extraction if there is a video id
|
||||
# along with playlist id.
|
||||
return self.url_result(video_id, 'Youtube', video_id=video_id)
|
||||
|
||||
@ -45,7 +45,7 @@ def parseOpts(overrideArguments=None):
|
||||
except IOError:
|
||||
return default # silently skip if file is not present
|
||||
try:
|
||||
# FIXME: https://github.com/rg3/youtube-dl/commit/dfe5fa49aed02cf36ba9f743b11b0903554b5e56
|
||||
# FIXME: https://github.com/ytdl-org/youtube-dl/commit/dfe5fa49aed02cf36ba9f743b11b0903554b5e56
|
||||
contents = optionf.read()
|
||||
if sys.version_info < (3,):
|
||||
contents = contents.decode(preferredencoding())
|
||||
|
||||
@ -31,7 +31,7 @@ def rsa_verify(message, signature, key):
|
||||
def update_self(to_screen, verbose, opener):
|
||||
"""Update the program file with the latest version from the repository"""
|
||||
|
||||
UPDATE_URL = 'https://rg3.github.io/youtube-dl/update/'
|
||||
UPDATE_URL = 'https://yt-dl.org/update/'
|
||||
VERSION_URL = UPDATE_URL + 'LATEST_VERSION'
|
||||
JSON_URL = UPDATE_URL + 'versions.json'
|
||||
UPDATES_RSA_KEY = (0x9d60ee4d8f805312fdb15a62f87b95bd66177b91df176765d13514a0f1754bcd2057295c5b6f1d35daa6742c3ffc9a82d3e118861c207995a8031e151d863c9927e304576bc80692bc8e094896fcf11b66f3e29e04e3a71e9a11558558acea1840aec37fc396fb6b65dc81a1c4144e03bd1c011de62e3f1357b327d08426fe93, 65537)
|
||||
|
||||
@ -546,7 +546,7 @@ def sanitize_url(url):
|
||||
return 'http:%s' % url
|
||||
# Fix some common typos seen so far
|
||||
COMMON_TYPOS = (
|
||||
# https://github.com/rg3/youtube-dl/issues/15649
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/15649
|
||||
(r'^httpss://', r'https://'),
|
||||
# https://bx1.be/lives/direct-tv/
|
||||
(r'^rmtp([es]?)://', r'rtmp\1://'),
|
||||
@ -596,7 +596,7 @@ def _htmlentity_transform(entity_with_semicolon):
|
||||
numstr = '0%s' % numstr
|
||||
else:
|
||||
base = 10
|
||||
# See https://github.com/rg3/youtube-dl/issues/7518
|
||||
# See https://github.com/ytdl-org/youtube-dl/issues/7518
|
||||
try:
|
||||
return compat_chr(int(numstr, base))
|
||||
except ValueError:
|
||||
@ -877,7 +877,7 @@ class XAttrUnavailableError(YoutubeDLError):
|
||||
def _create_http_connection(ydl_handler, http_class, is_https, *args, **kwargs):
|
||||
# Working around python 2 bug (see http://bugs.python.org/issue17849) by limiting
|
||||
# expected HTTP responses to meet HTTP/1.0 or later (see also
|
||||
# https://github.com/rg3/youtube-dl/issues/6727)
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/6727)
|
||||
if sys.version_info < (3, 0):
|
||||
kwargs['strict'] = True
|
||||
hc = http_class(*args, **compat_kwargs(kwargs))
|
||||
@ -1051,7 +1051,7 @@ class YoutubeDLHandler(compat_urllib_request.HTTPHandler):
|
||||
resp.msg = old_resp.msg
|
||||
del resp.headers['Content-encoding']
|
||||
# Percent-encode redirect URL of Location HTTP header to satisfy RFC 3986 (see
|
||||
# https://github.com/rg3/youtube-dl/issues/6457).
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/6457).
|
||||
if 300 <= resp.code < 400:
|
||||
location = resp.headers.get('Location')
|
||||
if location:
|
||||
@ -1190,7 +1190,7 @@ class YoutubeDLCookieProcessor(compat_urllib_request.HTTPCookieProcessor):
|
||||
def http_response(self, request, response):
|
||||
# Python 2 will choke on next HTTP request in row if there are non-ASCII
|
||||
# characters in Set-Cookie HTTP header of last response (see
|
||||
# https://github.com/rg3/youtube-dl/issues/6769).
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/6769).
|
||||
# In order to at least prevent crashing we will percent encode Set-Cookie
|
||||
# header before HTTPCookieProcessor starts processing it.
|
||||
# if sys.version_info < (3, 0) and response.headers:
|
||||
@ -1798,6 +1798,14 @@ def parse_resolution(s):
|
||||
return {}
|
||||
|
||||
|
||||
def parse_bitrate(s):
|
||||
if not isinstance(s, compat_str):
|
||||
return
|
||||
mobj = re.search(r'\b(\d+)\s*kbps', s)
|
||||
if mobj:
|
||||
return int(mobj.group(1))
|
||||
|
||||
|
||||
def month_by_name(name, lang='en'):
|
||||
""" Return the number of a month by (locale-independently) English name """
|
||||
|
||||
@ -2044,7 +2052,7 @@ def get_exe_version(exe, args=['--version'],
|
||||
try:
|
||||
# STDIN should be redirected too. On UNIX-like systems, ffmpeg triggers
|
||||
# SIGTTOU if youtube-dl is run in the background.
|
||||
# See https://github.com/rg3/youtube-dl/issues/955#issuecomment-209789656
|
||||
# See https://github.com/ytdl-org/youtube-dl/issues/955#issuecomment-209789656
|
||||
out, _ = subprocess.Popen(
|
||||
[encodeArgument(exe)] + args,
|
||||
stdin=subprocess.PIPE,
|
||||
@ -2654,7 +2662,7 @@ def _match_one(filter_part, dct):
|
||||
# If the original field is a string and matching comparisonvalue is
|
||||
# a number we should respect the origin of the original field
|
||||
# and process comparison value as a string (see
|
||||
# https://github.com/rg3/youtube-dl/issues/11082).
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/11082).
|
||||
actual_value is not None and m.group('intval') is not None and
|
||||
isinstance(actual_value, compat_str)):
|
||||
if m.group('op') not in ('=', '!='):
|
||||
@ -3804,7 +3812,7 @@ def urshift(val, n):
|
||||
|
||||
|
||||
# Based on png2str() written by @gdkchan and improved by @yokrysty
|
||||
# Originally posted at https://github.com/rg3/youtube-dl/issues/9706
|
||||
# Originally posted at https://github.com/ytdl-org/youtube-dl/issues/9706
|
||||
def decode_png(png_data):
|
||||
# Reference: https://www.w3.org/TR/PNG/
|
||||
header = png_data[8:]
|
||||
@ -3919,7 +3927,7 @@ def write_xattr(path, key, value):
|
||||
if hasattr(xattr, 'set'): # pyxattr
|
||||
# Unicode arguments are not supported in python-pyxattr until
|
||||
# version 0.5.0
|
||||
# See https://github.com/rg3/youtube-dl/issues/5498
|
||||
# See https://github.com/ytdl-org/youtube-dl/issues/5498
|
||||
pyxattr_required_version = '0.5.0'
|
||||
if version_tuple(xattr.__version__) < version_tuple(pyxattr_required_version):
|
||||
# TODO: fallback to CLI tools
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
__version__ = 'vc.2019.03.04'
|
||||
__version__ = 'vc.2019.03.17'
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user