544 Commits

• [utils] Process bytestrings in urljoin (closes #12369)
  Sergey M․ · 2017-03-06 03:57:46 +07:00
  4b5de77bdb
• [utils] Carry long doc string
  Sergey M․ · 2017-03-01 23:04:02 +07:00
  158af5242e
• [utils] Add bytes_to_long() and long_to_bytes()

  Used in daisuki.net (#4738)
  
  Both are adapted from public domain PyCrypto:
  https://github.com/dlitz/pycrypto/blob/master/lib/Crypto/Util/number.py

  Yen Chi Hsuan · 2017-02-28 22:10:31 +08:00
  0a5445ddbe
• [utils] Add pkcs1pad

  Used in daisuki.net (#4738)

  Yen Chi Hsuan · 2017-02-28 22:10:31 +08:00
  f48409c7ac
• Improve geo bypass mechanism

  * Rename options to preffixly match with --geo-verification-proxy
  * Introduce _GEO_COUNTRIES for extractors
  * Implement faking IP right away for sites with known geo restriction

  Sergey M․ · 2017-02-19 05:10:08 +08:00
  4248dad92b
• [utils] Make random_ipv4 return unicode string
  Sergey M․ · 2017-02-19 05:10:08 +08:00
  18a0defab0
• Add experimental geo restriction bypass mechanism

  Based on faking X-Forwarded-For HTTP header

  Sergey M․ · 2017-02-19 05:10:08 +08:00
  773f291dcb
• [utils] Introduce YoutubeDLError base class for all youtube-dl exceptions
  Pierre Mdawar · 2017-02-19 05:10:08 +08:00
  bf5b9d859a
• [utils] Add support for quoted string literals in --match-filter (closes #8050, closes #12142, closes #12144)
  Sergey M․ · 2017-02-15 23:12:10 +07:00
  db13c16ef8
• utils.py: Workaround TypeError with Python 2.7.13 in Windows

  Fixes #11540
  
  Tested with Windows Python 2.7.12 and 2.7.13.

  Ricardo Constantino · 2017-02-11 14:51:28 +00:00
  2f49bcd690
• Introduce get_elements_by_class and get_elements_by_attribute utility functions
  Thomas Christlieb · 2017-02-11 17:16:54 +08:00
  2af12ad9d2
• [utils] Improve comments processing in js_to_json (closes #11947)
  Sergey M․ · 2017-02-03 03:04:33 +07:00
  4195096ea8
• [utils] Handle single-line comments in js_to_json
  Michal Čihař · 2017-02-03 03:04:33 +07:00
  b3ee552e4b
• [utils] Improve parse_duration
  Sergey M․ · 2017-01-26 23:23:08 +07:00
  15846398ca
• [utils] Add another date format seen in NextTV
  Yen Chi Hsuan · 2017-01-23 23:37:32 +08:00
  0c1c6f4b9f
• [utils] Add more date formats
  Sergey M․ · 2017-01-12 22:39:45 +07:00
  cb655f34fb
• Fix "invalid escape sequences" error on Python 3.6
  Yen Chi Hsuan · 2017-01-06 00:58:56 +08:00
  ec85ded83c
• [utils] Improve urljoin
  Sergey M․ · 2016-12-17 18:49:55 +07:00
  b0c65c677f
• [utils] Add convenience urljoin
  Sergey M․ · 2016-12-13 02:23:49 +07:00
  e34c33614d
• [utils] Add common user agents map
  Sergey M․ · 2016-12-12 00:49:07 +07:00
  fb37eb25d9
• [utils] Introduce base_url
  Sergey M․ · 2016-11-02 02:30:18 +07:00
  02dc0a36b7
• [utils] Fix --match-filter for int-like strings (closes #11082)
  Sergey M․ · 2016-10-31 23:32:08 +07:00
  e5a088dc4b
• [utils] Clarify for redirecting STDIN in get_exe_version()
  Yen Chi Hsuan · 2016-10-22 13:04:05 +08:00
  b64d04c119
• [get_exe_version] Do version probes with <&-

  When doing version probes for ffmpeg, do the
  equivalent of calling it as:
  
      ffmpeg -version <&-
  
  Where <&- is shell syntax for closing stdin before calling the
  program. This is roughly equivalent to </dev/null without actually
  opening /dev/null.
  
  This prevents ffmpeg -version from hanging when run in the background.
  Fixes #955.
  
  The reason is that ffmpeg tries to manipulate stdin to set up terminal
  characteristic, and that causes the kernel to suspend the parent
  process (youtube-dl).
  
  Note that closing stdin is achieved by calling subprocess.Popen() with
  stdin set to subprocess.PIPE and without passing any input to
  Popen.communicate(). This is somewhat subtle.

  John Hawkinson · 2016-10-22 00:34:08 -04:00
  00ca755231
• [utils] Expose PACKED_CODES_RE
  Sergey M․ · 2016-10-19 23:28:49 +07:00
  06b3fe2926
• Unify coding cookie
  Déstin Reed · 2016-10-03 23:44:29 +07:00
  dcdb292fdd
• [utils] Fix xattr error handling
  Yen Chi Hsuan · 2016-10-02 03:03:41 +08:00
  bd26441205
• [utils] Support xattr as well as pyxattr

  Closes #9054
  
  There are two xattr packages in Python, pyxattr [1] and xattr [2]. They
  have different APIs.
  
  In old days pyxattr supports Linux only and xattr supports Linux, Mac,
  FreeBSD and Solaris, and pyxattr supports Linux only. Recently pyxattr
  adds support for Mac OS X. [3]
  
  An old version of [2] is shipped with Mac OS X. However, some Linux
  distributions have pyxattr only, for example PLD-Linux [4] and old Arch
  Linux. [5] As a result, supporting both is the way to go.
  
  [1] https://github.com/iustin/pyxattr
  [2] https://github.com/xattr/xattr
  [3] https://github.com/iustin/pyxattr/pull/9
  [4] https://github.com/rg3/youtube-dl/issues/5498
  [5] https://git.archlinux.org/svntogit/community.git/commit/?id=427c4c76401e386d865ccddea4fbfdc74df80492
      https://git.archlinux.org/svntogit/community.git/commit/?id=59b40da7b69622a6761d364a8b07909e9cccaa56
      python-xattr is added on 2016/06/29 while pyxattr is there for more
      than 6 years

  Yen Chi Hsuan · 2016-10-01 20:13:04 +08:00
  53a7e3d287
• [utils] Lower priority for rare date formats and add tests
  Sergey M․ · 2016-09-29 23:52:29 +07:00
  c6eed6b8c0
• [vk] Fix date and view count extraction.
  Kacper Michajłow · 2016-09-29 23:52:29 +07:00
  3aa3953d28
• Move write_xattr to utils.py

  There are some other places that use xattr functions. It's better to
  move it to a common place so that others can use it.

  Yen Chi Hsuan · 2016-09-30 00:28:32 +08:00
  efa97bdcf1
• [utils] Use native french month names
  Sergey M․ · 2016-09-14 23:59:38 +07:00
  3e4185c396
• [utils] Improve month_by_name and add tests
  Sergey M․ · 2016-09-14 23:59:38 +07:00
  f6717dec8a
• [utils,franceinter] Add french months' names and fix extraction

  Update of the "FranceInter" radio extractor : webpages HTML structure
  had changed, the extractor didn't work. So I updated this extractor to
  get the mp3 URL and all details.

  renalid · 2016-09-14 23:59:38 +07:00
  a942d6cb48
• [utils] Add quicktime to mimetype2ext
  Sergey M․ · 2016-09-02 23:50:42 +07:00
  c2b2c7e138
• [utils] Improve mimetype2ext
  Sergey M․ · 2016-09-02 22:57:48 +07:00
  6562d34a8c
• [utils] add ac-3 to the list of audio codecs in parse_codecs
  Remita Amine · 2016-08-25 08:49:44 +01:00
  073ac1225f
• [utils] Recognize units with full names in parse_filename

  Reference: https://en.wikipedia.org/wiki/Template:Quantities_of_bytes

  Yen Chi Hsuan · 2016-08-20 00:17:26 +08:00
  70852b47ca
• [utils] Correct octal/hexadecimal number detection in js_to_json
  Yen Chi Hsuan · 2016-08-19 20:37:17 +08:00
  e4659b4547
• [utils] Recognize lowercase units in parse_filesize
  Sergey M․ · 2016-08-18 23:32:00 +07:00
  13585d7682
• [external/curl] respect more downloader options and display progress
  Remita Amine · 2016-08-12 12:30:02 +01:00
  98e698f1ff
• [utils] Recognize more formats in unified_timestamp

  Used in CtsNews

  Yen Chi Hsuan · 2016-08-10 11:37:23 +08:00
  81c13222c6
• [utils] Add support TV Parental Guidelines ratings in parse_age_limit
  Sergey M․ · 2016-08-07 20:45:18 +07:00
  a8795327ca
• [utils] Add decode_png for openload (#9706)
  Yen Chi Hsuan · 2016-08-07 02:42:58 +08:00
  d3f8e038fe
• [utils] Fix unified_timestamp for formats parsed by parsedate_tz()
  Yen Chi Hsuan · 2016-08-05 11:41:55 +08:00
  7dc2a74e0a
• [utils] Add another f4m mimetype to mimetype2ext
  Sergey M․ · 2016-07-23 16:48:59 +07:00
  f164b97123
• [brightcove] skip ism manifests
  Remita Amine · 2016-07-14 14:13:57 +01:00
  e910fe2fe4
• Merge pull request #8876 from remitamine/html5_media

  [extractor/common] add helper method to extract html5 media entries

  Yen Chi Hsuan · 2016-07-10 23:40:45 +08:00
  0b68de3cc1
• [utils] Add get_element_by_class

  For #9950

  Yen Chi Hsuan · 2016-07-06 20:02:52 +08:00
  84c237fb8a
• [utils] add mimetypes to determine manifest ext(m3u8, f4m, mpd)
  Remita Amine · 2016-07-06 09:06:28 +01:00
  b4173f1551