1
0
mirror of https://github.com/l1ving/youtube-dl synced 2024-11-22 06:38:59 +08:00
youtube-dl/youtube_dl/extractor
Philipp Hagemeister b7ab059084 Add infrastructure for paged lists
This commit allows to download pages in playlists as needed instead of all at once.
Before this commit,
    youtube-dl http://www.youtube.com/user/ANNnewsCH/videos --playlist-end 2 --skip-download
took quite some time - now it's almost instantaneous.
As an example, the youtube:user extractor has been converted.
Fixes #2175
2014-01-20 11:36:47 +01:00
..
__init__.py Added support for Dropbox 2014-01-19 10:20:26 +05:30
academicearth.py More unicode literals 2014-01-07 10:06:30 +01:00
addanime.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
anitube.py Use the new '_download_xml' helper in more extractors 2013-11-26 19:17:25 +01:00
aparat.py [aparat] Add support (Fixes #2012) 2013-12-20 17:05:39 +01:00
appletrailers.py More unicode literals 2014-01-07 10:06:30 +01:00
archiveorg.py [archiveorg] Use centralized sorting 2014-01-07 10:16:22 +01:00
ard.py Do not redefine variables in list comprehensions 2013-07-08 02:12:20 +02:00
arte.py More unicode literals 2014-01-07 10:06:30 +01:00
auengine.py More unicode literals 2014-01-07 10:06:30 +01:00
bambuser.py More unicode literals 2014-01-07 10:06:30 +01:00
bandcamp.py [bandcamp] Make thumbnail and uploader optional 2014-01-09 23:04:36 +01:00
blinkx.py Correctly set IE_NAME field 2014-01-07 09:45:58 +01:00
bliptv.py Remove unused imports 2014-01-05 05:48:30 +01:00
bloomberg.py [bloomberg] Fix ooyala url extraction 2014-01-08 18:18:45 +01:00
breakcom.py Fix BreakIE 2013-07-23 11:41:05 +02:00
brightcove.py [brightcove] The ‘id’ attribute is not always present in the object tag (fixes #2132) 2014-01-10 19:39:42 +01:00
c56.py [c56] Add suppot for multiple formats 2014-01-07 10:19:15 +01:00
canalc2.py [canal2c] Accept more urls (fixes #1723) 2013-11-04 22:26:19 +01:00
canalplus.py Use the new '_download_xml' helper in more extractors 2013-11-26 19:17:25 +01:00
cbs.py [cbs] Add extractor (Fixes #1977) 2013-12-16 03:53:43 +01:00
channel9.py [channel 9] Use centralized format sorting 2013-12-26 21:14:43 +01:00
cinemassacre.py Use the 'rtmp_live' field for the live parameter of rtmpdump 2013-11-10 12:45:17 +01:00
clipfish.py [clipfish] Skip test on travis 2013-12-01 01:16:20 +01:00
clipsyndicate.py Use _download_xml in more extractors 2013-12-10 21:03:53 +01:00
cmt.py Add an extractor for cmt.com (closes #2049) 2013-12-31 17:21:44 +01:00
cnn.py [cnn] Improve test 2014-01-17 05:06:13 +01:00
collegehumor.py [collegehumor] Use character strings by default 2014-01-05 03:07:15 +01:00
comedycentral.py [comedycentral] Recognize ‘video-collections’ urls (#2072) 2014-01-01 21:11:35 +01:00
common.py [extractor/common] Limit --write-pages filename to 200 chars 2014-01-17 14:47:47 +01:00
condenast.py [condenast] Allow multiple formats, and sort centralized 2014-01-17 03:36:03 +01:00
criterion.py CriterionIE: simplify some parts and use _html_search_regex 2013-07-13 12:26:05 +02:00
crunchyroll.py [crunchyroll] Fix test (#1721) 2013-12-20 17:20:39 +01:00
cspan.py [cspan] Use HTTP download (Fixes #2098) 2014-01-05 04:30:19 +01:00
d8.py [d8] inherit from CanalplusIE 2013-11-19 20:44:20 +01:00
dailymotion.py [daylimotion] Add support for urls from the mobile site (fixes #1953) 2013-12-14 14:20:12 +01:00
daum.py [daum] Recognize mobile urls (#1952) 2013-12-12 13:05:38 +01:00
defense.py Correctly set IE_NAME field 2014-01-07 09:45:58 +01:00
depositfiles.py Style fixes for extractors: remove spaces around (,),{ and } 2013-11-03 14:06:47 +01:00
dotsub.py DotsubIE: simplify and extract the upload date 2013-07-08 22:40:42 +02:00
dreisat.py [dreisat] Make ‘index.php’ optional in the url (fixes #2080) 2014-01-03 12:02:08 +01:00
dropbox.py [dropbox] Correct test case (#2171) 2014-01-19 06:16:40 +01:00
ebaumsworld.py Use the new '_download_xml' helper in more extractors 2013-11-26 19:17:25 +01:00
ehow.py InfoExtractor: add some helper methods to extract OpenGraph info 2013-07-12 22:12:04 +02:00
eighttracks.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
eitb.py [brightcove] the format function requires to specify the index in python2.6 2013-11-09 18:10:11 +01:00
escapist.py [escapist] Fix title search 2013-11-20 07:23:23 +01:00
everyonesmixtape.py [everyonesmixtape] Add support (Fixes #2161) 2014-01-17 02:56:13 +01:00
exfm.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
extremetube.py Style fixes for extractors: remove spaces around (,),{ and } 2013-11-03 14:06:47 +01:00
facebook.py [facebook] Recognize #! URLs (Fixes #1988) 2013-12-16 21:10:06 +01:00
faz.py Merge remote-tracking branch 'dstftw/correct-valid-urls' 2013-12-04 19:56:05 +01:00
fktv.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
flickr.py [flickr] Use unicode literals 2014-01-17 03:07:01 +01:00
franceinter.py [franceinter] Remove superfluous whitespace 2014-01-17 03:10:54 +01:00
francetv.py [francetv] Add extractor for Culturebox (closes #2117) 2014-01-08 16:16:34 +01:00
freesound.py [freesound] Minor improvements 2013-07-15 21:33:45 +02:00
funnyordie.py [funnyordie] Fix video url extraction 2013-09-20 13:05:34 +02:00
gamekings.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
gamespot.py [gamespot] Use unicode_literals 2014-01-17 03:13:40 +01:00
gametrailers.py Merge remote-tracking branch 'dstftw/correct-valid-urls' 2013-12-04 19:56:05 +01:00
generic.py [generic] Do not fetch XML URLs (Fixes #2162) 2014-01-17 02:13:00 +01:00
googleplus.py [googleplus] Fix upload_date extraction 2013-10-21 15:00:21 +02:00
googlesearch.py Add --list-extractor-descriptions (human-readable list of IEs) 2013-07-01 18:52:19 +02:00
hark.py [hark] get the song info in JSON and extract more information. 2013-08-27 10:25:38 +02:00
hotnewhiphop.py [hotnewhiphop] Update test's title 2013-12-04 20:36:26 +01:00
howcast.py [howcast] update test's checksum 2013-11-22 21:25:12 +01:00
hypem.py Style fixes for extractors: remove spaces around (,),{ and } 2013-11-03 14:06:47 +01:00
ign.py [ign] Update one of test video's title 2013-12-13 17:04:40 +01:00
imdb.py +unicode_literals 2014-01-07 09:41:13 +01:00
ina.py Add a couple of missing http:// in test URLs 2014-01-14 16:01:31 -05:00
infoq.py Tests: Add coding to files 2013-06-27 20:32:02 +02:00
instagram.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
internetvideoarchive.py [internetvideoarchive] Use centralized format sorting 2013-12-26 21:08:52 +01:00
ivi.py [ivi] Use centralized format sorting 2013-12-26 18:40:16 +01:00
jeuxvideo.py Use the new '_download_xml' helper in more extractors 2013-11-26 19:17:25 +01:00
jpopsukitv.py [jpopsuki] Simplify 2014-01-03 12:51:37 +01:00
jukebox.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
justintv.py Use the new '_download_xml' helper in more extractors 2013-11-26 19:17:25 +01:00
kankan.py [kankan] Skip test 2014-01-17 04:21:54 +01:00
keek.py [keek] Fix testcase (Broken by accident in 6625f82940) 2013-07-25 10:10:37 +02:00
keezmovies.py Style fixes for extractors: remove spaces around (,),{ and } 2013-11-03 14:06:47 +01:00
khanacademy.py [khanacademy] Add support (Fixes #2066) 2014-01-07 09:35:34 +01:00
kickstarter.py Merge pull request #1409 from JohnyMoSwag/master (closes #1404) 2013-09-13 19:52:56 +02:00
liveleak.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
livestream.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
lynda.py [lynda] Add support for member accounts and paid videos (Closes #2125) 2014-01-12 05:31:56 +07:00
macgamestore.py [macgamestore] Minor fixes (#2044) 2014-01-03 13:09:39 +01:00
mdr.py [mdr] Use centralized format selection 2013-12-24 23:34:11 +01:00
metacafe.py Remove unused imports 2013-12-09 04:53:23 +01:00
metacritic.py [metacritic] Use centralized sorting and unicode_literals 2014-01-07 10:27:35 +01:00
mit.py Remove unused imports 2013-12-25 15:33:19 +01:00
mixcloud.py [mixcloud] Use unicode_literals 2014-01-17 04:06:18 +01:00
mofosex.py Style fixes for extractors: remove spaces around (,),{ and } 2013-11-03 14:06:47 +01:00
mpora.py [mpora] Fix uploader name extraction 2014-01-17 03:59:42 +01:00
mtv.py Add an extractor for cmt.com (closes #2049) 2013-12-31 17:21:44 +01:00
muzu.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
myspace.py [myspace] More robust mediatype check 2014-01-20 02:44:08 +01:00
myspass.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
myvideo.py [myvideo] Use RTMP instead of RTMPT (Fixes #2032) 2013-12-23 15:57:43 +01:00
naver.py [naver] Recognize mobile urls (fixes #1951) 2013-12-12 13:04:02 +01:00
nba.py InfoExtractor: add some helper methods to extract OpenGraph info 2013-07-12 22:12:04 +02:00
nbc.py Use the new '_download_xml' helper in more extractors 2013-11-26 19:17:25 +01:00
ndtv.py [ndtv] Remove unused imports 2013-12-16 08:16:38 +01:00
newgrounds.py [newgrounds] simplify 2013-09-18 22:14:43 +02:00
nhl.py Use the new '_download_xml' helper in more extractors 2013-11-26 19:17:25 +01:00
niconico.py Use the new '_download_xml' helper in more extractors 2013-11-26 19:17:25 +01:00
ninegag.py [9gag] Like/dislike count (#1895) 2013-12-05 18:29:07 +01:00
novamov.py [novamov] Add embedded player support 2014-01-08 08:09:13 +07:00
nowvideo.py [nowvideo] Add support for .sx version (Fixes #2127) 2014-01-12 01:26:37 +01:00
ooyala.py [generic] Detect ooyala videos (fixes #2013) 2013-12-19 20:32:12 +01:00
orf.py [orf] Remove unused variable name 2014-01-07 05:51:46 +01:00
pbs.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
photobucket.py Move tests to the IE definitions 2013-06-27 20:46:46 +02:00
podomatic.py [podomatic] Add extractor 2013-11-29 03:33:25 +01:00
pornhd.py [pornhd] Add support for ISO-3166 subpages (Fixes #2088) 2014-01-05 03:13:10 +01:00
pornhub.py [pornhub] Use centralized sorting 2014-01-07 10:25:34 +01:00
pornotube.py Add the missing age_limit tags; added a devscript to do a superficial check for porn sites without the age_limit tag in the test 2013-10-28 01:50:17 -04:00
pyvideo.py [pyvideo] add support for videos that don't come from Youtube 2013-12-07 11:19:59 +01:00
radiofrance.py [radiofrance] remove unused imports 2013-12-17 12:35:16 +01:00
rbmaradio.py Move tests to the IE definitions 2013-06-27 20:46:46 +02:00
redtube.py [redtube] Make ‘http:’ not optional (closes #2160) 2014-01-16 11:21:33 +01:00
ringtv.py Add RingTVIE (Thanks @yasoob) 2013-06-28 18:51:00 +02:00
ro220.py [ro220] Add support for 220.ro 2013-08-24 21:10:03 +02:00
rottentomatoes.py Add an extractor for rottentomatoes.com and improve InternetVideoArchiveIE to get the best quality 2013-10-12 22:22:31 +02:00
roxwel.py Add an extractor for roxwell.com (closes #1044) 2013-07-26 13:00:59 +02:00
rtlnow.py Add a couple of missing http:// in test URLs 2014-01-14 16:01:31 -05:00
rutube.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
sina.py Use the new '_download_xml' helper in more extractors 2013-11-26 19:17:25 +01:00
slashdot.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
slideshare.py Add an extractor for Slideshare (closes #1400) 2013-09-10 11:19:58 +02:00
smotri.py [smotri] Fix typo 2013-12-25 09:02:35 +07:00
sohu.py [sohu] add support for my.tv.sohu.com urls (fixes #1398) 2013-09-09 19:56:16 +02:00
soundcloud.py [soundcloud] Use unicode_literals and centralized sorting 2014-01-17 03:29:41 +01:00
southparkstudios.py Move common code for extractors based in MTV services to a new base class 2013-12-03 14:58:24 +01:00
space.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
spankwire.py [spankwire] Use centralized format sorting and unicode_literals 2014-01-17 03:26:05 +01:00
spiegel.py [spiegel] Use centralized sorting 2013-12-24 12:40:23 +01:00
stanfordoc.py Remove unused imports 2013-12-09 04:53:23 +01:00
statigram.py [statigram] change test video 2013-08-21 19:20:27 +02:00
steam.py Set the playlist_index and playlist fields for already resolved video results. 2013-07-10 23:36:30 +02:00
streamcloud.py [streamcloud] skip test on travis 2013-11-23 15:57:42 +01:00
subtitles.py [subtitles] refactor to support websites with subtitle information the 2013-11-02 19:29:25 +01:00
sztvhu.py [sztv] skip test, site is undergoing mid-term maintenance 2013-11-20 09:59:03 +01:00
teamcoco.py [teamcoco] Use centralized sorting 2014-01-17 03:22:02 +01:00
techtalks.py Add an extractor for techtalks.tv (closes #1606) 2013-10-17 08:20:58 +02:00
ted.py [ted] Updated checksums 2014-01-17 03:54:54 +01:00
tf1.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
theplatform.py [theplatform] Use centralized sorting 2013-12-26 21:18:18 +01:00
thisav.py [thisav] add a test for video MD5 2013-07-17 11:18:14 +02:00
toutv.py Use the new '_download_xml' helper in more extractors 2013-11-26 19:17:25 +01:00
traileraddict.py Merge pull request #1114 from alexvh/traileraddict_hd 2013-07-24 10:52:24 +02:00
trilulilu.py Remove the compatibility code used before the new format system was implemented 2013-12-03 14:31:20 +01:00
tube8.py Clean up unused imports and other minor mistakes 2013-11-20 06:27:48 +01:00
tudou.py [tudou] Fix title regex (Fixes #1614) 2013-10-18 11:16:20 +02:00
tumblr.py TumblrIE: update test 2013-06-30 13:52:20 +02:00
tutv.py InfoExtractor: add some helper methods to extract OpenGraph info 2013-07-12 22:12:04 +02:00
tvp.py [tvp] Skip tests 2013-11-16 02:09:30 +01:00
unistra.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
ustream.py [ustream] Simplify channel extraction 2013-09-13 22:05:29 +02:00
vbox7.py [vbox7] New video checksum 2013-12-20 17:27:43 +01:00
veehd.py [veehd] Send requests twice (Fixes #2102) 2014-01-06 12:54:01 +01:00
veoh.py [veoh] Simplify 2014-01-05 05:48:12 +01:00
vevo.py [vevo] Add suppor for videoplayer. URLs (#1957) 2013-12-16 21:48:38 +01:00
vice.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
viddler.py Remove unused imports 2013-12-05 14:31:54 +01:00
videodetective.py Fix the duration field for the VideoDetective and InternetVideoArchive tests 2013-10-21 15:07:33 +02:00
videofyme.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
videopremium.py [videopremium] Skip test 2013-12-16 14:42:07 +01:00
viki.py [viki] Fix subtitle extraction 2013-11-25 06:06:18 +01:00
vimeo.py [vimeo] Support protocol-relative URLs 2014-01-08 22:42:52 +01:00
vine.py [vine] Fix uploader extraction 2013-11-12 20:50:52 +01:00
vk.py Add an extractor for vk.com (closes #1635) 2013-11-01 22:34:18 +01:00
wat.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
websurg.py [websurg] Skipt the test 2013-10-15 08:12:30 +02:00
weibo.py tests: don't run the test if any of the extractors listed in the 'add_ie' field is marked as not working 2013-11-06 16:43:26 +01:00
wimp.py Remove unused imports 2013-12-09 04:53:23 +01:00
wistia.py [wistia] Prefer original video format above all others 2014-01-01 20:23:49 +01:00
worldstarhiphop.py [worldstarhiphop] Small cleanup 2013-07-30 11:10:17 +02:00
xhamster.py [XHamsterIE] Fix md5 in second test 2013-12-07 22:17:13 +01:00
xnxx.py [xnxx] Accept urls that start with 'www' (fixes #1734) 2013-11-06 23:45:01 +01:00
xtube.py [youtube] Do not warn for videos with allow_rating=0 2013-12-17 02:49:56 +01:00
xvideos.py extractor: Set age limit on some adult-related extractors. 2013-10-18 19:32:37 -03:00
yahoo.py [yahoo] Use centralized sorting, and add tbr field 2013-12-25 15:18:40 +01:00
youjizz.py Correct some extractor _VALID_URL regexes 2013-12-04 20:34:47 +07:00
youku.py Style fixes for extractors: remove spaces around (,),{ and } 2013-11-03 14:06:47 +01:00
youporn.py [youporn] Add support for multiple formats 2013-12-26 18:37:12 +01:00
youtube.py Add infrastructure for paged lists 2014-01-20 11:36:47 +01:00
zdf.py [zdf] Make width extraction more robust 2013-12-25 15:33:09 +01:00