Commit Graph

1884 Commits

Author SHA1 Message Date
AudricV
a3d160edab
[YouTube] Improve and fix YoutubeJavaScriptExtractor
- Enhance documentation;
- Fix the regular expression fallback on HTML embed watch page;
- Use HTML scripts tag search first instead of the regular expression approach,
now used as a last resort;
- Compile regular expressions only once, in order to improve the performance of
subsequent extraction calls when clearing the cache;
- Provide original exceptions when fetching or parsing pages on which the base
JavaScript's player could be found failed, allowing clients to detect network
errors when they are the cause of the failures for instance;
- Remove delegate method which was not taking a video ID and hardcoding one, as
we can provide the video ID in all cases or do not provide a video ID at worse;
- Rename and make extraction methods package-private, as they are not intended
to be used publicly.

These breaking internal changes have been applied where needed, in
YoutubeJavaScriptExtractorTest and YoutubeStreamExtractor (in which an unneeded
initStsFromPlayerJsIfNeeded call have been removed).
2023-08-02 23:05:08 +02:00
AudricV
bb1ab166bf
[YouTube] Test that no banner is returned for carouselHeaderRenders 2023-08-01 22:19:43 +02:00
AudricV
f1fa84b4e3
[YouTube] Don't throw an exception when there is no banner available on a channel
Channels may not have a banner, so no exception should be thrown if no banner
is found.
2023-08-01 12:40:20 +02:00
Tobi
39a911db9f
Merge pull request #1084 from AudricV/yt_android-403s-workaround-and-streams-tests-fixes
[YouTube] Workaround again 403 HTTP issues on the ANDROID InnerTube client and fix stream tests
2023-07-31 23:51:10 +02:00
AudricV
522c78160f
[YouTube] Update stream tests mocks 2023-07-23 19:36:28 +02:00
AudricV
7528eb2bd9
[YouTube] Fix stream tests failures
- Fix testCheckAudioStreams test of
YoutubeStreamExtractorDefaultTest.AudioTrackLanguage test class, by updating
the excepted audio track name test to use the updated English audio track name
(audio track type info has been added on the video tested);
- Fix YoutubeStreamExtractorDefaultTest.PublicBroadcasterTest test class by
using a different video from a French and German public broadcast channel, as
the channel Dinge Erklärt – Kurzgesagt is not affiliated with a public
broadcast channel anymore;
- Fix YoutubeStreamExtractorLivestreamTest test class, by updating the excepted
name of the livestream to the current one.
2023-07-23 19:19:02 +02:00
AudricV
164c8e3abb
[YouTube] Workaround again 403 HTTP issues on the Android client by using new player parameters
These parameters are the only ones currently known to bypass 403 HTTP issues
related to failure of passing Android client integrity checks, as the ones of
stories (and the base of the shorts ones) do not work anymore, which may be
related to end of this format on the service.
2023-07-22 20:22:16 +02:00
FireMasterK
6db0d116fe Add support for AV1 itags. 2023-07-22 13:23:44 +02:00
AudricV
4e22c5ee87
[YouTube] Support multiple declarations for throttling parameter function name array
Also moved the corresponding regex parts in static constants for easier future
modifications
2023-06-26 15:25:53 +02:00
Kavin
d961d349c3
[YouTube] Check whether player responses are valid for all InnerTube clients used (#1070)
Co-authored-by: Audric V <74829229+AudricV@users.noreply.github.com>
2023-06-18 21:54:52 +02:00
ThetaDev
ad97f08048
[YouTube] Fix parsing short relative date formats (English only) (#1068) 2023-06-18 21:41:29 +02:00
Tobi
d294ccb433
Merge pull request #1071 from TeamNewPipe/feat/ServiceList
Init services at the correct place
2023-06-17 20:50:24 +02:00
TobiGr
0e01d90562 Do not init services while creating services array 2023-06-17 18:08:09 +02:00
TobiGr
5809904cf7 Add annotations to MediaFormat 2023-06-17 18:01:54 +02:00
TobiGr
53dfd871e2 Add more audio media formats and MediaFormat.getAllForMimeType(mimeType) 2023-06-17 18:01:29 +02:00
TobiGr
19ce06fe00 Fix BandcampRadioStreamExtractorTest.testGetAudioStreams() 2023-06-06 23:07:56 +02:00
Audric V
533121fb81
Merge pull request #1045 from Theta-Dev/fix/trending-video-tab
[YouTube] Extract trends from A/B tested "Videos" tab and fix extraction of trends name from A/B tested new title design
2023-05-19 11:22:49 +02:00
Audric V
92a0024424
Merge pull request #1052 from TeamNewPipe/peertube/fix/nested-comment-replies
[PeerTube] Fix multi level comment replies
2023-05-18 18:49:06 +02:00
TobiGr
c70bb83801 [Bandcamp] Implement PlaylistExtractor.getDescsription() 2023-05-15 15:23:03 +02:00
TobiGr
ca0ce00753 Add PlaylistInfoItem.getDescription() and PlaylistInfoItemExtractor.getDescription()
[PeerTube] Implement the corresponding extractor method.
TODO: add tests
2023-05-12 01:43:59 +02:00
TobiGr
b218bf69bd Implement PlaylistInfo.getDescription()
Implement PlaylistExtractor.getDescription() for PeerTube and SoundCloud.
Anotate   PlaylistExtractor.getDescription() as Nonnull
2023-05-12 00:44:10 +02:00
chunky programmer
81f29116ba switch from string to Description object 2023-05-11 00:36:57 -04:00
chunky programmer
e147867d41 Add tests 2023-05-11 00:00:34 -04:00
chunky programmer
5ab6cd7420 Extract YouTube playlist description 2023-05-11 00:00:22 -04:00
TobiGr
d358ba1c41 Improve PeertubeCommentsInfoItemExtractor constructor 2023-05-07 22:55:26 +02:00
TobiGr
aff3e795f8 [PeerTube] Fix multi level comment replies 2023-05-07 22:49:14 +02:00
ThetaDev
3673d4ae01
fix: YouTube trending name extraction 2023-05-03 21:16:35 +02:00
ThetaDev
0addb98cd7
tests: update mocks 2023-05-03 21:16:35 +02:00
ThetaDev
24eba62305
fix: extract YouTube trends from new "Videos" tab 2023-05-03 21:16:23 +02:00
Kavin
a9ca5c49e4
Merge pull request #1056 from AudricV/yt-improve-search-suggestions-extraction
[YouTube] Switch to new search suggestion domain and improve error handling
2023-05-02 20:17:48 +01:00
dependabot[bot]
108f8a7a17
Bump org.jsoup:jsoup from 1.15.4 to 1.16.1
Bumps [org.jsoup:jsoup](https://github.com/jhy/jsoup) from 1.15.4 to 1.16.1.
- [Release notes](https://github.com/jhy/jsoup/releases)
- [Changelog](https://github.com/jhy/jsoup/blob/master/CHANGES)
- [Commits](https://github.com/jhy/jsoup/compare/jsoup-1.15.4...jsoup-1.16.1)

---
updated-dependencies:
- dependency-name: org.jsoup:jsoup
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-01 09:58:32 +00:00
AudricV
cf6040ddb3
Update YoutubeSuggestionExtractorTest mocks 2023-04-30 19:53:44 +02:00
AudricV
593122342f
[YouTube] Improve YoutubeSuggestionExtractorTest
- Remove useless concatenation on the downloader path;
- Remove unneeded public test modifier;
- Update license header;
- Specify the service class tested instead of the generic class.
2023-04-30 19:53:43 +02:00
AudricV
e923fca440
[YouTube] Switch to new search suggestion domain and improve error handling
- Switch to the new domain used by YouTube for search suggestions,
suggestqueries-clients6.youtube.com, and add the xhr query parameter with the
t value, to allow getting responses without requiring trim;
- Use the Java 8 Stream API to collect search suggestions and improve invalid
response detection by checking whether the content type of the response
returned is JSON;
- Move the licence header at the top of the file.
2023-04-30 19:53:42 +02:00
AudricV
945165a3c0
[PeerTube] Don't return "No description" when there is no description for a channel or an account
When a description is missing, no description should be returned, even the ones
indicating there is no description. This behavior is represented by a null
return instead.

Also update PeertubeAccountExtractorTest to reflect these changes.
2023-04-30 18:41:38 +02:00
Stypox
2deb023da4
Merge pull request #1050 from Theta-Dev/fix/channel-carousel-header
[YouTube] Add support for CarouselHeaderRenderer
2023-04-25 15:17:31 +02:00
ThetaDev
4aada7f91b refactor: rename carousel header channel test 2023-04-21 22:48:37 +02:00
ThetaDev
47aa9fed40 fix: set musicClientVersion regex capture group 2023-04-16 19:25:05 +02:00
ThetaDev
20370395c5 fix: add support for CarouselHeaderRenderer 2023-04-16 17:40:13 +02:00
Stypox
7dba6e3891
Merge pull request #1033 from petlyh/bandcamp-paywalled-content
[Bandcamp] Handle paywalled tracks
2023-04-12 13:04:26 +02:00
petlyh
e6aad117e7
[Bandcamp] Throw PaidContentException on paywalled albums 2023-04-03 19:27:09 +02:00
fynngodau
69705138e4
[Bandcamp] Fix extraction of related playlist items URL (#1047)
Small change in HTML structure
2023-04-02 22:24:29 +02:00
Björn Sigurbergsson
1b6fe5edd6
[YouTube] Fix ParsingException when comments are unavailable in a video (#1040)
Co-authored-by: bjs <bjs@elect-it.com>
Co-authored-by: Audric V. <74829229+AudricV@users.noreply.github.com>
Co-authored-by: Kavin <20838718+FireMasterK@users.noreply.github.com>
2023-03-30 19:58:06 +02:00
ThetaDev
8d1303e18f
Add track types to audio streams (#1041) 2023-03-28 00:02:20 +02:00
AudricV
80a6fc2c63
[PeerTube] Fix testGetCommentsFromCommentsInfo test of PeertubeCommentsExtractorTest.Default
The tested comment has been removed, so it couldn't be found in the comments
list.

This comment has been replaced by a new one from the current comments of the
video.

Also, in the parent class PeertubeCommentsExtractorTest, final has been used as
much as possible and for-each loops of lists have been replaced by their
forEach method or the Stream API, in order to simplify code.
2023-03-04 16:49:10 +01:00
petlyh
5a9b6ed2e3
[Bandcamp] Support loading additional comments (#1030) 2023-03-04 14:01:06 +01:00
Stypox
6bdd698c25
Merge pull request #1026 from AudricV/audio-streams-descriptive-and-locale-properties
Add descriptive and locale properties to audio streams
2023-03-01 11:15:46 +01:00
Stypox
19e4b216c9
Merge pull request #1032 from AudricV/yt_fix-comments-hashtags-links-extraction
[YouTube] Fix hashtags links extraction and escape HTML links
2023-03-01 10:47:37 +01:00
Stypox
b1298490c0
Merge pull request #1029 from AudricV/yt_fix-no-views-extraction-playlist-items
[YouTube] Fix partial non-extraction of "No views" string in stream items
2023-03-01 10:46:52 +01:00
petlyh
9dc1832733
[Bandcamp] Handle paywalled tracks 2023-02-28 17:51:30 +01:00
fynngodau
3fdb6ee476
Merge pull request #1031 from petlyh/bandcamp-fix-radio-comments
[Bandcamp] Show comments as disabled on radio streams
2023-02-27 20:50:12 +01:00
AudricV
bd79b921e8
[YouTube] Refactor the code to get stream items' view count
This refactoring avoids code duplication as much as possible.
2023-02-27 10:25:46 +01:00
AudricV
51f9b39953
[YouTube] Fix partial non-extraction of no views string in stream items
As the "No views" string is returned in the case there is no view on a video, a
number cannot be parsed in this case, so -1 was returned.

This string is now detected in all methods to get the view count of a stream.
2023-02-27 10:18:45 +01:00
AudricV
95b3f5e391
[MediaCCC] Test audio language property extraction 2023-02-26 19:06:18 +01:00
AudricV
30a0f8c510
[MediaCCC] Extract audio language property for single language audio tracks 2023-02-26 19:06:18 +01:00
AudricV
7f0269c4c7
[YouTube] Edit YoutubeStreamExtractorDefaultTest.AudioTrackLanguage to test audio locale property
The Hindi audio track language presence test has been changed from audio track
label to audio locale.
2023-02-26 19:06:18 +01:00
AudricV
034f82dae7
[YouTube] Test language and descriptive audio in YoutubeDashManifestCreatorsTest 2023-02-26 19:06:17 +01:00
AudricV
05e8cb39f7
[YouTube] Add language and descriptive audio properties to DASH manifests 2023-02-26 19:06:17 +01:00
AudricV
bf30d70152
[YouTube] Add descriptive audio test
This test uses video TjxC-evzxdk.

Also improve a bit YoutubeStreamExtractorDefaultTest.AudioTrackLanguage test.
2023-02-26 19:06:17 +01:00
AudricV
76b7c19c5d
[YouTube] Extract whether a track is a descriptive audio and audio locale when available
Also use audio track setters only for audio itags.
2023-02-26 19:06:17 +01:00
AudricV
3bb5eeef30
[YouTube] Add descriptive and locale audio support in ItagItem 2023-02-26 19:06:16 +01:00
AudricV
14bf3fb05b
Add ability to know the locale of an audio stream
Getting audio tracks locales by parsing their ID or their label, should not be
done by clients, but by the extractor.

This commit adds the ability to store the Locale of an AudioStream, which is
used to compare similar AudioStreams (in the equalStats method).
2023-02-26 19:06:16 +01:00
AudricV
f92426560c
Add descriptive audio properties
Also improve AudioStream's audio language documentation
2023-02-26 19:06:16 +01:00
AudricV
a63f289667
[YouTube] Update mocks of YoutubeCommentsExtractorTest.FormattingTest 2023-02-26 18:50:07 +01:00
AudricV
9483dcd9fa
[YouTube] Update mocks of YoutubeCommentsExtractorTest.RepliesTest 2023-02-26 18:43:36 +01:00
AudricV
1556adbb2d
[YouTube] Fix hashtags links extraction and escape text in attribute descriptions + HTML links
webCommandMetadata object is contained inside a commandMetadata one, so it is
not accessible from the root of the navigationEndpoint object.

The corresponding statement has been moved at the bottom of the specific
endpoints parsing, as the webCommandMetadata object is present almost
everywhere, otherwise URLs of some endpoints would have be changed, such as
uploader URLs (from channel IDs to handles).

As no ParsingException is now thrown by getUrlFromNavigationEndpoint, and so by
getTextFromObject, getUrlFromObject and getTextAtKey, the methods which were
catching ParsingExceptions thrown by these methods had to be updated.

URLs got in the HTML version of getTextFromObject are now escaped properly to
provide valid HTML to clients. This has been also done for attribute
descriptions, with the description text for this type of descriptions.

As YouTube descriptions are in HTML format (except for the fallback on the JSON
player response, which is plain text and only happens when there is no visual
metadata or a breaking change), all URLs returned are escaped, so tests which
are testing presence of URLs with escaped characters had to be updated (it was
only the case for YoutubeStreamExtractorDefaultTest.DescriptionTestUnboxing).
2023-02-26 18:43:36 +01:00
petlyh
f7a7a236fb
[Bandcamp] Show comments as disabled on radio streams 2023-02-23 18:42:43 +01:00
dependabot[bot]
f5599ff08d
Bump org.jsoup:jsoup from 1.15.3 to 1.15.4
Bumps [org.jsoup:jsoup](https://github.com/jhy/jsoup) from 1.15.3 to 1.15.4.
- [Release notes](https://github.com/jhy/jsoup/releases)
- [Changelog](https://github.com/jhy/jsoup/blob/master/CHANGES)
- [Commits](https://github.com/jhy/jsoup/compare/jsoup-1.15.3...jsoup-1.15.4)

---
updated-dependencies:
- dependency-name: org.jsoup:jsoup
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-20 10:05:26 +00:00
TobiGr
3f7df9536e [YouTube] Fix getting the comment text if the comment contains a hashtag 2023-01-29 20:33:51 +01:00
Stypox
999fb7f812
Merge pull request #1024 from AudricV/snd_fix-tracks-like-count
[SoundCloud] Fix extraction of tracks like count
2023-01-29 10:52:54 +01:00
Stypox
3519d4c367
Merge pull request #1015 from AudricV/yt_fix-channel-id-rss-feeds
[YouTube] Fix channel ID extraction of YouTube channels RSS feeds
2023-01-29 10:41:38 +01:00
Stypox
9aca710e86
Merge pull request #1013 from Stypox/fix-music-mixes
[YouTube] Now music mixes can be treated as normal mixes
2023-01-29 09:48:51 +01:00
Stypox
76eeabac45
Merge pull request #1020 from TeamNewPipe/fix/yt-subscriber-count
[YouTube] Fix NPE in search when getting channel items without subscriber count
2023-01-29 09:44:22 +01:00
AudricV
676622f6df
[SoundCloud] Fix expectedLikeCountAtLeast tests of SoundcloudStreamExtractorTest test classes
As like count is now returned by the extractor, we need to assert a positive
minimum like count, which is close to the actual value, in order to avoid test
failures due to lower like counts than the ones excepted.
2023-01-29 01:08:02 +01:00
AudricV
2a24d407d5
[SoundCloud] Fix extraction of tracks like count
SoundCloud is using likes_count to return the like count of a track, like it
was the case before they switched to favoritings_count.
2023-01-29 01:00:49 +01:00
AudricV
ba24976e41
[YouTube] Add live URLs test and do minor improvements to YoutubeStreamLinkHandlerFactoryTest
- Remove unused imports;
- Replace wildcard imports by single class imports;
- Suppress "HTTP links are not secured" warnings from IDEA IDEs;
- Replace removed video jZViOEv90dI by an existing video, 9Dpqou5cI08 (the
corresponding test has been of course renamed).
2023-01-28 19:36:21 +01:00
AudricV
57f850bc2d
[YouTube] Support live URLs and do minor improvements to YoutubeStreamLinkHandlerFactory
- Move license header at the top;
- Use an unmodifiable set for the subpaths instead of a modifiable list;
- Add missing Nonnull and Nullable annotations;
- Improve exception messages.
2023-01-28 19:36:20 +01:00
AudricV
1f4ed9dce9
[YouTube] Fix channel ID extraction of YouTube channel RSS feeds
The yt:channelId element doesn't provide the channel ID anymore and is empty,
like the id element, so we need now to extract it from the channel URL provided
in two elements: author -> uri and feed -> link.

Also avoid a NullPointerException in getUrl and getName methods.
2023-01-28 11:53:33 +01:00
Tobi
c589a2c1a2
Merge pull request #1014 from TeamNewPipe/fix/yt-comments
[YouTube] Fix getting next comments pages
2023-01-27 11:14:55 +01:00
TobiGr
72573932cf [YouTube] Fix NPE in search when getting channel items without subscriber count 2023-01-24 23:03:45 +01:00
TobiGr
f50b7275af [YouTube] Fix getting next comments pages 2023-01-24 22:39:08 +01:00
Kunal
9bdad40b06 Removed topStandaloneBadge 2023-01-20 02:41:21 +05:30
Stypox
5945057227
[YouTube] Add music mix test 2023-01-15 23:30:30 +01:00
Stypox
7293991832
[YouTube] Now music mixes can be treated as normal mixes
Using a playlist extractor on them would result in "Unviewable playlist" errors
2023-01-15 23:28:59 +01:00
Stypox
ff94e9f30b
Merge pull request #1009 from TeamNewPipe/dependabot/gradle/com.google.code.gson-gson-2.10.1
Bump gson from 2.10 to 2.10.1
2023-01-11 15:35:36 +01:00
Stypox
c1040bccac
Merge pull request #794 from FireMasterK/comments-count
[YouTube] Add support to extract total comment count
2023-01-11 15:32:19 +01:00
dependabot[bot]
f43049985e
Bump gson from 2.10 to 2.10.1
Bumps [gson](https://github.com/google/gson) from 2.10 to 2.10.1.
- [Release notes](https://github.com/google/gson/releases)
- [Changelog](https://github.com/google/gson/blob/master/CHANGELOG.md)
- [Commits](https://github.com/google/gson/compare/gson-parent-2.10...gson-parent-2.10.1)

---
updated-dependencies:
- dependency-name: com.google.code.gson:gson
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-09 09:05:55 +00:00
TobiGr
56aab4d971 [YouTube] Fix escaping links in YouTubeParsingHelper.getTextFromObject 2023-01-05 00:28:12 +01:00
Kavin
22a47da8c7
Fix requested change and remove outdated comment. 2023-01-02 20:42:32 +00:00
Kavin
98a90fd9c8
Don't cache comments count and return early on page fetch if no token. 2023-01-02 20:40:48 +00:00
Kavin
2974dfaa48
Only store ajaxJson for initial page and eager fetch the initial continuation. 2023-01-02 20:40:48 +00:00
Kavin
64d24aa09e
Fix request changes. 2023-01-02 20:40:48 +00:00
Kavin
67ef4f4c30
Cleanup and remove optional. 2023-01-02 20:40:48 +00:00
FireMasterK
22f71b010c
Fix for requested changes. 2023-01-02 20:40:48 +00:00
FireMasterK
656b7c1cd9
Improve method documentation. 2023-01-02 20:40:48 +00:00
FireMasterK
981aee4092
Add support to extract total comment count. 2023-01-02 20:40:48 +00:00
Stypox
45636b0d00
Merge pull request #986 from Isira-Seneviratne/Static_maps
Use immutable Map factory methods.
2023-01-02 18:11:14 +01:00
Stypox
219c5c5be5
Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/YoutubeParsingHelper.java 2023-01-02 18:11:03 +01:00
Stypox
259de3cba6
Merge pull request #995 from TeamNewPipe/feat/soundcloud-playlistinfoitemextractor
[SoundCloud] Implement getUploaderUrl() and isUploaderVerified() for PlaylistInfoItemExtractor
2023-01-02 15:10:40 +01:00
Stypox
991394b53a
Merge pull request #1005 from FireMasterK/fix-escaping-xss
Fix for potential XSS attacks and formatting issues
2023-01-02 15:06:17 +01:00
Isira Seneviratne
d8ce08d969 Use immutable Map factory methods. 2023-01-02 07:50:31 +05:30
Kavin
01acf79436
Fix for potential XSS attacks. 2022-12-31 20:05:32 +00:00
TobiGr
292e0d8ce7 [SoundCloud] Implement getUploaderUrl() and isUploaderVerified() for PlaylistInfoItemExtractor 2022-12-31 18:46:39 +01:00
TobiGr
2a8729aeb2 Apply suggestions
Co-authored-by: Stypox <stypox@pm.me>
2022-12-31 18:24:33 +01:00
TobiGr
d75a997611 [PeerTube] Support searching for channels 2022-12-31 18:24:33 +01:00
TobiGr
dea6d8ce4c [PeerTube] Support searching for playlists 2022-12-31 18:24:33 +01:00
Stypox
95cc6aefbb
Merge pull request #994 from TeamNewPipe/fix/peertube-subtitles-exception
[PeerTube] Report Exceptions thrown while getting a stream's subtitles
2022-12-31 15:01:39 +01:00
Stypox
7b54457789
Merge pull request #941 from TeamNewPipe/feat/peertube-comment-replies
[PeerTube]  Support comment replies
2022-12-31 14:57:51 +01:00
AudricV
f45966d449
Merge pull request #910 from Isira-Seneviratne/Locale_forLanguageTag
Add compat Locale.forLanguageTag() implementation.
2022-12-24 23:53:30 +01:00
AudricV
d5437e0bc5
Merge pull request #863 from AudricV/add-content-type-and-content-length-headers-to-post-requests
Add Content-Type header to all POST requests without an empty body
2022-12-16 19:32:56 +01:00
AudricV
0766b1d211
[YouTube] Improve YoutubeStreamInfoItemExtractor
- Return duration of video premieres;
- Add another non-localized method to determine whether a stream is a running
livestream;
- Return view count and upload date of videos in playlists;
- Store isPremiere result;
- Remove shorts workaround code, as it was only useful on channels and shorts
have been moved into a separated channel tab;
- Improve some other code.
2022-12-08 13:59:12 +01:00
Tobi
896d7e09eb
Merge pull request #978 from Theta-Dev/fix/search-channel-handles
[YouTube] Fix search subscriber count extraction with channel handles
2022-12-05 17:52:05 +01:00
TobiGr
cd3262745d [PeerTube] Report Exceptions thrown while getting a stream's subtitles 2022-12-03 16:11:21 +01:00
TobiGr
4e66b2287e [PeerTube] Add support for comment replies 2022-12-01 14:05:18 +01:00
Tobi
41c8dce452
Merge pull request #992 from Isira-Seneviratne/String_isBlank
Use String.isBlank().
2022-11-30 17:48:54 +01:00
Isira Seneviratne
2bca56f0df Use String.isBlank(). 2022-11-30 08:26:21 +05:30
Isira Seneviratne
3b80547976 Add code review suggestions. 2022-11-30 07:57:45 +05:30
ThetaDev
016623131e docs: update comment in YoutubeChannelInfoItemExtractor 2022-11-29 19:06:03 +01:00
Kavin
2e08eaad96
Fix complication error in comment test. 2022-11-29 16:07:48 +00:00
Kavin
abf08e1496
Merge pull request #990 from FireMasterK/bold-italic-strikethrough
[YouTube] Implement bold/italic/strike-through support
2022-11-29 15:59:38 +00:00
Kavin
57e7a6fb7c
Add mocks test. 2022-11-28 20:27:55 +00:00
Kavin
1d3d7fa5c3
Add test for formatting. 2022-11-28 20:26:37 +00:00
Kavin
52fda37915
Implement bold/italic/strike-through support. 2022-11-28 19:06:18 +00:00
Kavin
b566084cac
Use Description object for comments text. 2022-11-28 17:02:19 +00:00
Tobi
f8162b049d
Merge pull request #984 from FireMasterK/unused-dep
Remove unused autolink dependency
2022-11-28 11:28:42 +01:00
Tobi
1da0190056
Merge pull request #980 from TeamNewPipe/fix/yt/unavailable
[YouTube] Fix extracting the detailed error message for unavailable streams
2022-11-28 10:07:34 +01:00
Stypox
60fb30f835
Merge pull request #928 from FireMasterK/comment-urls
Parse YouTube comments as HTML
2022-11-27 19:16:34 +01:00
Kavin
5abea22225
Fix throwing correct reason. 2022-11-26 21:09:08 +00:00
Kavin
faf28f5c11
Remove unused dependency. 2022-11-26 20:17:25 +00:00
Kavin
c043597255
Update supported countries list. 2022-11-26 19:01:33 +00:00
TobiGr
4680df0bdf Fix throwing correct reason 2022-11-23 17:03:22 +01:00
TobiGr
9de8405c9f [YouTube] Fix extracting the detailed error message of streams which are unavailable 2022-11-23 08:33:06 +01:00
Stypox
34d79bd267
[YouTube] Update mocks 2022-11-22 17:10:04 +01:00
AudricV
2ec296e674
Fix YoutubeSearchExtractorTest.MetaInfoTest
Not all the "learn more" button is uppercase anymore, that's only the case for
the first letter.
2022-11-22 16:34:54 +01:00
AudricV
3891542ca1
Use Downloader's postWithContentType and postWithContentTypeJson methods in services and extractors 2022-11-22 11:37:18 +01:00
AudricV
b2862f3cd1
Add postWithContentType and postWithContentTypeJson utility methods in Downloader
Co-authored-by: Stypox <stypox@pm.me>
2022-11-22 11:37:17 +01:00
AudricV
e9a0d3bd95
[YouTube] Send Content-Type header in all POST requests
This header was not sent partially before and was added and guessed by OkHttp. This can create issues when using other HTTP clients than OkHttp, such as Cronet.

Some code in the modified classes has been improved and / or deduplicated, and usages of the UTF_8 constant of the Utils class has been replaced by StandardCharsets.UTF_8 where possible.

Note that this header has been not added in except in YoutubeDashManifestCreatorsUtils, as an empty body is sent in the POST requests made by this class.
2022-11-22 11:37:16 +01:00
AudricV
b9e463de49
[Bandcamp] Send Content-Type header in POST requests
This header was not sent before and was added and guessed by OkHttp. This can create issues when using other HTTP clients than OkHttp, such as Cronet.

Also make use of StandardCharsets.UTF_8 when getting bytes of bodies instead of the platform default's charset, to make sure to prevent some encoding issues on some JVMs.
2022-11-22 11:35:46 +01:00
AudricV
65d6321e3d
Fix typos in Downloader.post JavaDocs
Post methods in Downloader return the result of a POST request and not the one of a GET request.
2022-11-22 11:35:46 +01:00
ThetaDev
5daabd1793 fix: #976 search subscriber count extraction with channel handles 2022-11-22 02:17:10 +01:00
Kavin
c953e23414
Merge pull request #968 from AudricV/yt-support-no-video-info-renderers-for-streams
[YouTube] Support lack of video info renderers for streams
2022-11-16 20:20:01 +00:00
Tobi
2211a24b69
Merge pull request #971 from lrusso96/patch-1
[YouTube] Improve duration parsing
2022-11-16 16:14:54 +01:00
Kavin
86f06b333a
Address review. 2022-11-14 00:05:31 +00:00
Kavin
b16e6082e1
Add test for audio stream languages. 2022-11-13 23:10:44 +00:00
Kavin
30909da1df
Fix audio track similar comparison for IDs. 2022-11-13 23:08:54 +00:00
Kavin
6d59cdbe3a
Add support for extracting audio tracks. 2022-11-13 21:39:29 +00:00
Isira Seneviratne
e4d982c7ea Fix license. 2022-11-12 07:29:15 +05:30
Isira Seneviratne
416089146e Fix failing tests. 2022-11-12 07:29:15 +05:30
Isira Seneviratne
ddbce3b83d Add Utils methods for URL encoding/decoding using UTF-8. 2022-11-12 07:29:15 +05:30
Isira Seneviratne
366f5c1632 Use StandardCharsets.UTF_8. 2022-11-12 07:29:15 +05:30