1988 Commits

Author	SHA1	Message	Date
litetex	65687f3b9b	Removed YT dislike count as it no longer works	2021-12-27 16:54:46 +01:00
litetex	1e2e0029fc	[StreamExtractor] Deduplicated a ton of code by using default methods	2021-12-27 16:54:45 +01:00
litetex	15b98ffdb4	Remove old ways of getting YT dis/likes ... * Added additional check for averageRating (in dislikes)	2021-12-27 16:54:44 +01:00
litetex	7489502cc8	Disabled `testRelatedItems` as it's unreliable	2021-12-11 15:23:11 +01:00
litetex	d9b2a2042a	Updated mock data for YoutubeCommentsExtractorTest	2021-12-08 22:11:45 +01:00
litetex	a3a0c1ee85	Fixed YoutubeCommentsExtractorTest ... Old video was deleted / set to private	2021-12-08 22:11:34 +01:00
litetex	5c288eb8d7	Updated mock data for YoutubeChannelExtractorTest	2021-12-08 21:51:38 +01:00
litetex	75bb83a1b8	Fixed YoutubeChannelExtractorTest ... Channel was restored, picked a new one	2021-12-08 21:51:15 +01:00
litetex	f276bacce5	Fixed SoundcloudStreamExtractorTest	2021-12-08 21:41:27 +01:00
litetex	7f99c1c193	Fixed YoutubeStreamExtractorLivestreamTest	2021-12-08 21:16:39 +01:00
litetex	652fdf2c36	Updated mock-data for YoutubeStreamExtractorAgeRestrictedTest	2021-12-08 21:08:25 +01:00
litetex	3c669d3cb5	Fixed YoutubeStreamExtractorAgeRestrictedTest ... Looks like a lot of stuff (description, tags, name) got changed on the existing video. Use a better suited video.	2021-12-08 21:08:03 +01:00
litetex	f735788a39	Fixed Soundcloud hardcoded client id	2021-12-08 21:00:53 +01:00
bopol	56c8af710e	Code refactoring	2021-11-21 18:16:31 +01:00
bopol	c4eca91be9	Fix YouTube likes + dislikes	2021-11-19 21:36:03 +01:00
TobiGr	5028396405	Improve names and formatting	2021-11-14 13:44:13 +01:00
Tobi	68d7e4e20a	Merge pull request #742 from TeamNewPipe/dependabot/gradle/com.google.code.gson-gson-2.8.9 ... Bump gson from 2.8.8 to 2.8.9	2021-11-08 19:37:09 +01:00
opusforlife2	cb80a646d9	Update hardcoded SoundCloud client_id ... Hah! I got here first, @TobiGr!	2021-11-04 15:28:26 +00:00
bopol	396aecef19	Make sure playlists aren't accepted by PeertubeStreamLinkHandlerFactory	2021-11-03 14:41:39 +01:00
bopol	38ad1eaac5	[PeerTube] Support /w/ short video links part 2 ... We also need to support it for comments	2021-11-03 13:54:37 +01:00
bopol	026751624f	[PeerTube] Support /w/p/ short playlist links	2021-11-03 13:35:30 +01:00
bopol	e0b8e142fc	[PeerTube] Support /a/ and /c/ short links	2021-11-03 13:26:27 +01:00
bopol	599a91c88c	[PeerTube] Support /w/ short video links	2021-11-03 11:51:58 +01:00
dependabot[bot]	198d37090b	Bump gson from 2.8.8 to 2.8.9 ... Bumps [gson](https://github.com/google/gson) from 2.8.8 to 2.8.9. - [Release notes](https://github.com/google/gson/releases) - [Changelog](https://github.com/google/gson/blob/master/CHANGELOG.md) - [Commits](https://github.com/google/gson/compare/gson-parent-2.8.8...gson-parent-2.8.9) --- updated-dependencies: - dependency-name: com.google.code.gson:gson dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>	2021-11-01 09:12:53 +00:00
mhmdanas	3e8e2a1532	Add support for y2u.be links	2021-10-22 22:48:18 +03:00
Tobi	b425394e7a	Merge pull request #731 from FireMasterK/short-description ... Extract Video Short Description in YouTube.	2021-10-15 23:10:00 +02:00
FireMaskterK	62313962a0	Add default implementation for getShortDescription.	2021-10-04 01:42:29 +01:00
Kavin	d1add05bc1	Add requested changes. ... Co-authored-by: TiA4f8R <74829229+TiA4f8R@users.noreply.github.com> Add final. Co-authored-by: TiA4f8R <74829229+TiA4f8R@users.noreply.github.com>	2021-10-04 01:36:51 +01:00
dependabot[bot]	3ca9f51b30	Bump jsoup from 1.14.2 to 1.14.3 ... Bumps [jsoup](https://github.com/jhy/jsoup) from 1.14.2 to 1.14.3. - [Release notes](https://github.com/jhy/jsoup/releases) - [Changelog](https://github.com/jhy/jsoup/blob/master/CHANGES) - [Commits](https://github.com/jhy/jsoup/compare/jsoup-1.14.2...jsoup-1.14.3) --- updated-dependencies: - dependency-name: org.jsoup:jsoup dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>	2021-10-02 08:20:38 +00:00
TobiGr	d8ec3ce370	[SoundCloud] Update client id	2021-10-01 02:51:38 +02:00
FireMasterK	94efe86c71	Extract Video Short Description in YouTube. ... In Trending, and Search results.	2021-09-22 16:26:16 +01:00
Tobi	a9d214478d	Merge pull request #703 from FireMasterK/comment-replies ... Add support for extracting comment replies continuation	2021-09-14 23:58:14 +02:00
FireMasterK	6aabdc6d16	Fix for requested changes.	2021-09-12 01:15:19 +05:30
Kavin	0aad09fa22	Update JavaDocs for comment replies continuation. ... Co-authored-by: Stypox <stypox@pm.me>	2021-09-12 01:05:20 +05:30
FireMasterK	feb09e9997	Extract Channel Avatar in related videos.	2021-09-04 01:09:40 +05:30
Tobi	f0aa46b008	Merge pull request #724 from FireMasterK/streams-uploader-verified ... Extract uploaderVerified to StreamInfo.	2021-09-03 18:18:04 +02:00
FireMasterK	857b2f39e8	Extract uploaderVerified to StreamInfo.	2021-09-03 21:27:58 +05:30
FireMasterK	1db463b55f	Add a default test for the Uploader's Avatar.	2021-09-03 21:05:36 +05:30
Kavin	db6b3b2c29	Extract uploader's avatar in peertube. ... Co-authored-by: Tobi <TobiGr@users.noreply.github.com>	2021-09-02 16:14:45 +05:30
FireMasterK	a0c1dcc8d8	Remove throws parsing exception.	2021-09-02 16:14:45 +05:30
FireMasterK	0ba03c552c	Improve based on review/feedback.	2021-09-02 00:35:56 +05:30
FireMasterK	b9fad4fcc8	Extract Uploader's Avatar on YouTube.	2021-09-02 00:19:00 +05:30
Stypox	68f1fa994a	Merge pull request #721 from Stypox/yt-csv-import ... [YouTube] Small improvements to subscription import	2021-08-31 12:02:59 +02:00
TobiGr	b644f1d67b	[SoundCloud] Update client id	2021-08-31 07:42:21 +02:00
Stypox	a5e9eeb790	[YouTube] Small improvements to subscription import	2021-08-30 15:45:45 +02:00
Tobi	9570882c73	Merge pull request #712 from magicbrothers/add-eduvid-tubus ... Add invidious instance EduVid Tubus	2021-08-24 16:07:07 +02:00
talanc	94a29fd63f	[YouTube] csv and zip subscription import (Google Takeout) ... csv: Improved error messages Exits early if it hasnt found any items in the first few lines zip: Now checks all CSV files instead of hard-coded paths final qualifiers for immutable locals and parameters Co-authored-by: litetex <40789489+litetex@users.noreply.github.com>	2021-08-24 15:42:49 +02:00
dependabot[bot]	816bc7137f	Bump gson from 2.8.7 to 2.8.8 ... Bumps [gson](https://github.com/google/gson) from 2.8.7 to 2.8.8. - [Release notes](https://github.com/google/gson/releases) - [Changelog](https://github.com/google/gson/blob/master/CHANGELOG.md) - [Commits](https://github.com/google/gson/compare/gson-parent-2.8.7...gson-parent-2.8.8) --- updated-dependencies: - dependency-name: com.google.code.gson:gson dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>	2021-08-23 13:04:10 +02:00
dependabot[bot]	72393995bb	Bump jsoup from 1.13.1 to 1.14.2 (#716) ... Bumps [jsoup](https://github.com/jhy/jsoup) from 1.13.1 to 1.14.2. - [Release notes](https://github.com/jhy/jsoup/releases) - [Changelog](https://github.com/jhy/jsoup/blob/master/CHANGES) - [Commits](https://github.com/jhy/jsoup/compare/jsoup-1.13.1...jsoup-1.14.2) --- updated-dependencies: - dependency-name: org.jsoup:jsoup dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-08-22 09:12:14 +02:00
Stypox	d8179dc35a	Merge pull request #713 from TiA4f8R/regexes-yt-signatureciphers ... [YouTube] Consolidate the regular expressions used to find the cipher decryption function	2021-08-17 18:58:27 +02:00
XiangRongLin	61acdae4b8	Fix failing PeertubeStreamExtractorTest	2021-08-17 17:06:36 +02:00
FireMasterK	4f86b8ce43	Add verified info to ChannelInfo. (#714)	2021-08-17 13:08:49 +02:00
Stypox	52910ce970	Keep original exception when there is an error message (#710)	2021-08-13 16:13:58 +02:00
TiA4f8R	f8309d546a	[YouTube] Consolidate current regexes of the cipher decryption function ... Also add more regexes	2021-08-12 18:26:43 +02:00
magicfelix	0e16091ce0	Add invidious instance EduVid Tubus	2021-08-12 10:06:41 +02:00
gardenapple	50bba32239	Fix typos in Javadoc	2021-08-11 12:01:25 +03:00
TacoTheDank	cf9e678223	Declare common dependency versions in root gradle	2021-08-07 17:51:09 -04:00
FireMasterK	71b9fd0076	Faster iframe api based player extraction. (#694) ... * Faster iframe api based player extraction. Uses the IFrame API to reduce the required download to less than 1/50 of the size. * Remove debug code. * Extract to two methods. * Add tests for player URL extraction. * Add assertThat for tests.	2021-08-07 12:35:48 +02:00
FireMasterK	ee32317315	Update supported countries list for YouTube.	2021-08-05 23:48:38 +05:30
FireMasterK	e8e9e6cd00	Fix comment reply continuations.	2021-08-04 22:14:04 +05:30
FireMasterK	666b45958e	Fix test and update mocks.	2021-08-04 21:10:02 +05:30
FireMasterK	f6d054e5da	Add test for comments.	2021-08-04 21:03:00 +05:30
FireMasterK	ed84658055	Get it working.	2021-08-04 18:35:14 +05:30
FireMasterK	ea7e6526fd	Collect replies/page to CommentsInfoItem.	2021-08-04 18:09:42 +05:30
FireMasterK	4385cc1b7c	Add comment replies.	2021-08-04 17:56:04 +05:30
Stypox	5b38b3ae97	Merge branch 'master' into v0.21.8	2021-08-03 20:38:07 +02:00
TiA4f8R	79cc5c8d12	Fix tests of YoutubePlaylistExtractorTest and YoutubeMixPlaylistExtractorTest ... Without removing RunWith and SuiteClasses annotations (and the corresponding imports) in YoutubePlaylistExtractorTest and YoutubeMixPlaylistExtractorTest, some mocks cannot be generated, so the CI fails because of the missing mocks. Mocks of workings tests have been also updated.	2021-08-01 17:28:01 +02:00
TiA4f8R	b74a39c176	Reformat some code and don't use the clickTrackingParams in continuations of YouTube Music search results ... The clickTrackingParams of YouTube Music search results are not needed to get continuations. This commit removes their use, which may improve privacy.	2021-08-01 17:25:59 +02:00
TiA4f8R	f8197da09e	Update YouTube comments and YouTube playlist continuations mocks	2021-08-01 12:39:05 +02:00
FireMasterK	f3e4c9d689	Use the youtubei API for YouTube comments ... Migrate YouTube comments to the desktop version by using the `next` endpoint of the InnerTube internal API. With the desktop version, we are able to get the exact like count of YouTube comments (by parsing the accessibility data) (the current extraction is used as a fallback). We are also now able to get if the uploader of the comment is verified or not. Co-authored-by: TiA4f8R <74829229+TiA4f8R@users.noreply.github.com>	2021-08-01 12:39:05 +02:00
TiA4f8R	286d839a3b	Update mocks (except mocks of YoutubeCommentsExtractorTest)	2021-08-01 12:39:04 +02:00
TiA4f8R	693f654f02	Ignore a broken test of YoutubeMixPlaylistExtractorTest	2021-08-01 12:39:03 +02:00
TiA4f8R	d55109258d	Fix YoutubeStreamExtractorAgeRestrictedTest	2021-08-01 12:39:03 +02:00
FireMasterK	2eeb0a3403	Rebase + some code improvements + fix extraction of age-restricted videos + update clients version ... Here is now the requests which will be made by the `onFetchPage` method of `YoutubeStreamExtractor`: - the desktop API is fetched. If there is no streaming data, the desktop player API with the embed client screen will be fetched (and also the player code), then the Android mobile API. - if there is no streaming data, a `ContentNotAvailableException` will be thrown by using the message provided in playability status If the video is age restricted, a request to the next endpoint of the desktop player with the embed client screen will be sent. Otherwise, the next endpoint will be fetched normally, if the content is available. If the video is not age-restricted, a request to the player endpoint of the Android mobile API will be made. We can get more streams by using the Android mobile API but some streams may be not available on this API, so the streaming data of the Android mobile API will be first used to get itags and then the streaming data of the desktop internal API will be used. If the parsing of the Android mobile API went wrong, only the streams of the desktop API will be used. Other code changes: - `prepareJsonBuilder` in `YoutubeParsingHelper` was renamed to `prepareDesktopJsonBuilder` - `prepareMobileJsonBuilder` in `YoutubeParsingHelper` was renamed to `prepareAndroidMobileJsonBuilder` - two new methods in `YoutubeParsingHelper` were added: `prepareDesktopEmbedVideoJsonBuilder` and `prepareAndroidMobileEmbedVideoJsonBuilder` - `createPlayerBodyWithSts` is now public and was moved to `YoutubeParsingHelper` - a new method in `YoutubeJavaScriptExtractor` was added: `resetJavaScriptCode`, which was needed for the method `resetDebofuscationCode` of `YoutubeStreamExtractor` - `areHardcodedClientVersionAndKeyValid` in `YoutubeParsingHelper` returns now a `boolean` instead of an `Optional<Boolean>` - the `fetchVideoInfoPage` method of `YoutubeStreamExtractor` was removed because YouTube returns now 404 for every client with the `get_video_info` page - some unused objects and some warnings in `YoutubeStreamExtractor` were removed and fixed Co-authored-by: TiA4f8R <74829229+TiA4f8R@users.noreply.github.com>	2021-08-01 12:39:03 +02:00
TiA4f8R	7753556e66	Adress the last requested changes + update YoutubeCommentsExtractor mocks	2021-08-01 12:39:03 +02:00
TiA4f8R	2320aecb7c	Update mocks	2021-08-01 12:39:02 +02:00
TiA4f8R	629dcd63b2	Ignore some broken tests	2021-08-01 12:39:01 +02:00
TiA4f8R	3adac6a150	Workaround for rate limits: always use the Android mobile API	2021-08-01 12:39:01 +02:00
TiA4f8R	8aa60d7e8f	Update clients version	2021-08-01 12:39:01 +02:00
TiA4f8R	27bd797f1e	Update mocks	2021-08-01 12:39:01 +02:00
TiA4f8R	609919db59	Adress again reviews, fix some rebase issues	2021-08-01 12:39:00 +02:00
TiA4f8R	4299d806a2	Adress changes	2021-08-01 12:38:59 +02:00
TiA4f8R	1a6b8da438	Annotate YoutubeParsingHelper methods with Nonnull when needed	2021-08-01 12:38:59 +02:00
TiA4f8R	d8177b57f6	Loop in all formats to check if the stream has URLs protected by signatureCiphers	2021-08-01 12:38:59 +02:00
TiA4f8R	a6a2c6eb80	Revert the use of Collections.singletonList instead of Arrays.asList in addCookieHeader of YoutubeParsingHelper	2021-08-01 12:38:59 +02:00
TiA4f8R	81013e5a8e	Add a static method to reset the YouTube deobfuscation code and use it in tests ... This method is needed for YouTube stream tests, because when all YouTube tests are ran, the signatureTimestamp is known (the sts string) so a different body than the body present in the mocks is send by the extractor instance. As a result, running all YouTube stream tests with the MockDownloader (like the CI does) will fail if this method is not called before fetching the page of a test.	2021-08-01 12:38:58 +02:00
TiA4f8R	cc798523cd	Adress requested changes in other classes	2021-08-01 12:38:58 +02:00
TiA4f8R	accd5ddef3	Adress requested changes in YoutubeStreamExtractor	2021-08-01 12:38:58 +02:00
TiA4f8R	632772d17f	Adress requested changes in YoutubeParsingHelper	2021-08-01 12:38:58 +02:00
TiA4f8R	657f165771	Update client version and mocks	2021-08-01 12:38:44 +02:00
TiA4f8R	8c1c7281b0	Real fix of the fetch of the JS player at each the signatureTimestamp is required ... The strings playerJsUrl, sts and playerCode are now static in order to don't fetch again the JavaScript player at each time the signatureTimestamp is needed.	2021-08-01 12:38:43 +02:00
TiA4f8R	e97a685989	Format the YoutubeMusicSearchExtractor class to be in the 100 characters per line limit	2021-08-01 12:38:43 +02:00
TiA4f8R	34a9ccb0fd	Adress requested changes	2021-08-01 12:38:42 +02:00
TiA4f8R	54d4551ca6	Adress requested changes in YoutubeParsingHelper and update mobile client version	2021-08-01 12:38:42 +02:00
TiA4f8R	a59c2a3577	Catch every exception instead of only IOException and ExtractionException and add a Javadoc ... Catch every exception instead of only IOException and ExtractionException. Add JavaDoc for fetchAndroidMobileJsonPlayer method of YoutubeStreamExtractor	2021-08-01 12:38:42 +02:00
TiA4f8R	6921e80ded	Try again to don't fetch at each time the JavaScript player	2021-08-01 12:38:42 +02:00
TiA4f8R	70927ddade	Update client version and mocks	2021-08-01 12:38:40 +02:00
TiA4f8R	318bc46a8c	Readd the deleted code of views because watching count of livestreams was broken ... The number shown was the total number of views that a livestream has. In order to fix this bug, the previous code is readded.	2021-08-01 12:38:39 +02:00
TiA4f8R	947baec805	Fetch again the desktop player JSON only if the content is protected by signatureCiphers	2021-08-01 12:38:39 +02:00
TiA4f8R	7474049fd1	Update mocks	2021-08-01 12:38:39 +02:00
TiA4f8R	c32bc6e534	Try to don't fetch again the first page when requesting a channel continuation ... Try to don't fetch again the first page of a YouTube channel when requesting a continuation of it by trying to store the channel name and the channel id into the next page using the ids field of the Page class.	2021-08-01 12:38:38 +02:00
TiA4f8R	b52732a1c3	Fix the fetch of the playerCode at each time getStsFromPlayerJs is called ... storePlayerJs was called even if sts was not empty in the getStsFromPlayerJs method. This commit fixes it.	2021-08-01 12:38:37 +02:00
TiA4f8R	ae5abc0c5d	Fix a typo in YoutubeChannelExtractor	2021-08-01 12:38:37 +02:00
TiA4f8R	0f9e9b8b4b	Use the youtubei API for YouTube mixes + update the corresponding test + do some improvements ... Use the youtubei API for YouTube mixes. The corresponding has been updated because the new API breaks the tests of YoutubeMixPlaylistExtractorTest. Remove some deprecated code (the old search code with the pbj JSON) and do some other improvements.	2021-08-01 12:38:37 +02:00
TiA4f8R	14569c4aa9	Readd the return of itags 134 and 136	2021-08-01 12:38:37 +02:00
TiA4f8R	3017dde67e	Fix some typos	2021-08-01 12:38:36 +02:00
TiA4f8R	013b902535	Use the Android mobile API when there are OTF streams or the content is protected by signatureCiphers ... Use the Android mobile API to get the itag 22 (720p with audio), removed when the content is protected by signatureCiphers. Also use this API when they are OTF streams, to get the itag 17 and 36, low 3GPP quality streams but also the itag 139. Update the web client version.	2021-08-01 12:38:36 +02:00
TiA4f8R	e7d589edbf	Use the youtubei API for YouTube videos + update client version ... Update the hardcoded client version to 2.20210520.09.00 Use the player and next endpoints of the Innertube API for YouTube videos	2021-08-01 12:38:36 +02:00
TiA4f8R	f73c923f60	Don't use the youtubei.googleapis.com but the websites domains + update client version of the desktop internal API ... Use again www.youtube.com and music.youtube.com domains instead of youtubei.googleapis.com domain because it spoofs more a web client of YouTube or YouTube Music and may reduce Google's detection of NewPipe Extractor users.	2021-08-01 12:38:34 +02:00
TiA4f8R	effcdaa4f2	Update mocks, reenable a test and fix a test ... This commit updates mocks and reenables the test invalidId of the NotAvailable class of the YoutubePlaylistExtractorTest class beacuse with the youtubei/innertube API, it returns "Not found" and doesn't redirect to the YouTube homepage. The expectedMetaInfo test of the MetaInfoTest class of the YoutubeSearchExtractorTest class was broken because YouTube removes the vaccine progress link of the WHO from the meta info so this commit removes it and the test is now passing.	2021-08-01 12:38:04 +02:00
TiA4f8R	4d682834c3	Fix localization and update client version	2021-08-01 12:38:03 +02:00
TiA4f8R	f46cfb0f26	Adress reviews and do some improvements ... Adress changes requested in reviews. Do some improvements, remove unused imports and format some code to be in the 100 characters line limit.	2021-08-01 12:38:03 +02:00
TiA4f8R	e075dd5a63	Update client version, fix some tests, update mocks and do some improvements ... Add the origin and the referer headers with the https://www.youtube.com value for YouTube JSON POST requests. Don't add the consent cookie header for the requests which use the youtubei/innertube API because it's uneeded. Fix some tests and update YouTube mocks	2021-08-01 12:38:02 +02:00
TiA4f8R	b49ae547a3	Do some improvements to YoutubeStreamExtractor ... Get the real name of the uploader (for autogenerated channels and music artist channels), like before the migration to the JSON pbj. Do some other improvements, especially reformatting some code to be in the 100 characters line limit and use final where possible.	2021-08-01 12:38:01 +02:00
TiA4f8R	58ce9b04a1	Fix channel extraction when channel URL is youtube.com/c/username	2021-08-01 12:38:01 +02:00
TiA4f8R	991b2c7d73	Use lightweight requests when getting and checking YouTube API key and client version	2021-08-01 12:38:01 +02:00
TiA4f8R	9ab9c66ddf	Use the youtubei.googleapis.com domain for YouTube Music searches + change a check + update client version and mocks ... Change the domain from music.youtube.com to youtubei.googleapis.com. Use a lightweight request to check if the hardcoded YouTubeMusic keys are valid. Increase the length of the response to 500 because if the key is invalid, the length of the response returned is higher than 250 and the response when the key is valid is higher than 1500. Format the YoutubeMusicSearchExtractor file. Update YouTube web client version and mocks	2021-08-01 12:37:56 +02:00
TiA4f8R	77c031a88a	Use the youtubei API for YouTube trends	2021-08-01 12:36:28 +02:00
TiA4f8R	f461224b2b	Use the youtubei API for YouTube searches + update mocks ... Add getSearchParameter, a new method in YoutubeSearchQueryHandlerFactory class which returns the params field for a search, or an empty string if there is no one. Update mocks of YoutubeSearchExtractorTest.	2021-08-01 12:36:28 +02:00
TiA4f8R	a12c69da7d	Use the youtubei API for YouTube channels	2021-08-01 12:36:27 +02:00
TiA4f8R	5794eb2350	Use the youtubei API for YouTube playlists	2021-08-01 12:36:12 +02:00
Stypox	c97a19d719	Merge pull request #696 from XiangRongLin/decrypt_pattern ... Expand regex to match n param decrypt function	2021-08-01 10:24:22 +02:00
XiangRongLin	37df225556	Remove length check from StringUtils.matchToClosingParenthesis	2021-07-31 16:05:24 +02:00
FireMasterK	f4aad8b014	Fix tests again.	2021-07-31 14:40:51 +05:30
FireMasterK	88c11db4c5	Fix isCorrectedSearch test.	2021-07-31 02:18:18 +05:30
FireMasterK	fd19c53f65	Fix search suggestions test.	2021-07-31 02:08:50 +05:30
FireMasterK	a685941bba	Fix YouTube music search.	2021-07-31 01:33:38 +05:30
XiangRongLin	852a65ff18	Add tests for StringUtils	2021-07-30 19:56:57 +02:00
XiangRongLin	48d897e6ad	Add final and adjust utils class name	2021-07-29 21:33:45 +02:00
XiangRongLin	60794aea31	[YouTube] Add parenthesis matching as way to parse decrypt function ... From @Stypox	2021-07-29 21:23:00 +02:00
XiangRongLin	2967d1ae6a	[YouTube] Compile YoutubeThrottlingDecrypter pattern statically	2021-07-29 20:27:29 +02:00
XiangRongLin	1c78976900	[YouTube] Expand regex to match n param decrypt function ... Temporary solution	2021-07-29 20:26:36 +02:00
TobiGr	b62fe7141e	Fix JDoc 2	2021-07-29 12:34:04 +02:00
TobiGr	dbc4e01c17	Fix JDoc	2021-07-29 12:31:33 +02:00
TobiGr	8bfcb0ad59	[SoundCloud] Fix hardcoded client id	2021-07-28 20:33:21 +02:00
Tobi	027dc65434	pull request #683 from XiangRongLin/yt_throttling ... [YouTube] Fix buffering by decoding n parameter of stream urls	2021-07-28 20:32:38 +02:00
Tobi	14c179f343	Merge pull request #692 from TeamNewPipe/fix/sc/id ... [SoundCloud] Fix hardcoded client id	2021-07-28 18:28:07 +02:00
Tobi	394c02ad06	Merge pull request #683 from XiangRongLin/yt_throttling ... [YouTube] Fix buffering by decoding n parameter of stream urls	2021-07-28 18:01:57 +02:00
TobiGr	785ff4aa32	[SoundCloud] Fix hardcoded client id	2021-07-28 17:02:56 +02:00
TobiGr	79f2d74b04	Fix test ... The decryption code changes over time. Only check whether the n parameter was changed in the URL	2021-07-28 16:45:31 +02:00
TobiGr	d70adfdb8f	Add methods for cache control to YoutubeThrottlingDecrypter.	2021-07-27 20:32:22 +02:00
TiA4f8R	d13f531b6f	Use YoutubeThrottlingDecrypter also in getAudioStreams and getVideoOnlyStreams methods of YoutubeStreamExtractor ... Without this commit, the n param is only decrypted for streams extracted in getVideoStreams (so only for streams in the formats object of the player response).	2021-07-27 17:28:32 +02:00
TiA4f8R	32055147e0	Do some code improvements ... Use final where possible, annotate some methods and parameters as Nonnull and format new code to be in the 100 characters limit per line.	2021-07-27 17:24:27 +02:00
TobiGr	1c30a2725e	Cache nParams to prevent executing the JavaScript function for the same nParam multiple times. ... Closes #689	2021-07-22 19:40:12 +02:00
XiangRongLin	3a3d1d7f2b	Make YoutubeJavaScriptExtractor and JavaScript methods static ... Also address review and rewrite some comments	2021-07-20 20:48:11 +02:00
XiangRongLin	a683c8d278	Delete duplicated code to load youtube javascript file	2021-07-17 19:14:57 +02:00
XiangRongLin	a02ee2e952	Rewrite youtube throttling solution and add tests	2021-07-17 19:10:09 +02:00
litetex	0c12b396e5	Review changes	2021-07-17 16:41:12 +02:00
litetex	ecf4232ce3	Using `Collections.emptyList()`	2021-07-17 16:41:12 +02:00
litetex	0f9ed020a4	Frameset has to implement Serializable or NewPipe fails ... java.lang.RuntimeException: Parcelable encountered IOException writing serializable object (name = org.schabi.newpipe.extractor.stream.StreamInfo) ... Caused by: java.io.NotSerializableException: org.schabi.newpipe.extractor.stream.Frameset ...	2021-07-17 16:41:11 +02:00
litetex	17ccaf4b87	Update YoutubeStreamExtractor.java ... To fix ``java.lang.NullPointerException: Attempt to invoke virtual method 'java.lang.String[] java.lang.String.split(java.lang.String)' on a null object reference``	2021-07-17 16:41:10 +02:00
litetex	4e41e172ea	Update Frameset.java ... Reformatted code	2021-07-17 16:41:10 +02:00
litetex	09b670d745	Renamed frames to previewFrames (so it's better understandable)	2021-07-17 16:41:09 +02:00
litetex	b086655d54	Reverted useless change	2021-07-17 16:41:08 +02:00
litetex	93b0c5b15f	Added seekbar data	2021-07-17 16:41:08 +02:00
XiangRongLin	6956b72af7	Fix YoutubeThrottlingDecoder.replaceNParam ... Previously it replaced the parameter itself not the value of the parameter.	2021-07-17 10:05:14 +02:00
XiangRongLin	a189f685dc	cache player js code	2021-07-16 19:20:40 +02:00
XiangRongLin	fcdb9bdbeb	add url to sout	2021-07-15 20:05:26 +02:00
XiangRongLin	80cf8b3acd	Extract separate YoutubeThrottlingDecoder	2021-07-15 20:05:25 +02:00
XiangRongLin	a86a30103f	[Youtube] bare bones version to solve throttling ... Done by transforming the parameter "n" from videoplayback urls https://github.com/ytdl-org/youtube-dl/issues/29326#issuecomment-865985377	2021-07-15 20:05:25 +02:00
bopol	c38a06e8dc	Merge pull request #661 from B0pol/peertube-livestreams ... [PeerTube] Support livestreams	2021-07-13 15:48:13 +02:00
litetex	fdebf3c6cd	Added javadoc warnings	2021-07-07 20:41:59 +02:00
litetex	6860543b07	Added more doc	2021-07-06 21:16:31 +02:00
litetex	a59fead0d7	Fixed typos	2021-07-06 21:16:30 +02:00
litetex	688a1c316b	Fixed Exception when YT comments are disabled and added `commentsDisabled` field ... * Fixed code: Added missing finals (according to NewPipes Checkstyle guide) * Fixed ``findValue`` method in ``YoutubeCommentsExtractor``	2021-07-06 21:16:29 +02:00
litetex	ca33f4f60b	Improved code format (final) and added `isCommentsDisabled` method	2021-07-06 21:16:28 +02:00
TobiGr	dbdc962a07	Update mocks	2021-07-03 13:30:26 +02:00
TobiGr	9ca6dc26ed	Fix invalid CONSENT cookie value	2021-07-03 13:27:40 +02:00
XiangRongLin	b9549ace25	[Youtube] Fix lofi girl test and ignore mix with id test	2021-06-29 21:12:20 +02:00
XiangRongLin	5982431ffb	Remove IPv4 addresses from response during mock recording	2021-06-29 21:03:33 +02:00
bopol	7d7cc087e2	[PeerTube] Support livestreams	2021-06-24 11:02:30 +02:00
bopol	3966178979	Merge pull request #665 from B0pol/agerestricted ... [youtube] Fix extraction of age restricted videos	2021-06-23 21:07:14 +02:00
bopol	b475f09ba7	don't do useless concatenation	2021-06-23 20:33:32 +02:00
bopol	b96b2a6eba	Update invidious instances	2021-06-23 14:39:10 +02:00
bopol	c5e8bd368d	Update Mocks	2021-06-23 12:21:57 +02:00
bopol	e9a992b0a9	[youtube] Fix extraction of age restricted videos	2021-06-23 12:06:02 +02:00
dependabot[bot]	ad403ce73e	Bump spotbugs-annotations from 4.0.2 to 4.2.3 ... Bumps [spotbugs-annotations](https://github.com/spotbugs/spotbugs) from 4.0.2 to 4.2.3. - [Release notes](https://github.com/spotbugs/spotbugs/releases) - [Changelog](https://github.com/spotbugs/spotbugs/blob/master/CHANGELOG.md) - [Commits](https://github.com/spotbugs/spotbugs/compare/4.0.2...4.2.3) --- updated-dependencies: - dependency-name: com.github.spotbugs:spotbugs-annotations dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>	2021-06-23 09:57:42 +00:00
bopol	65fd8740aa	Merge pull request #658 from TeamNewPipe/revert-651-fix/streamType ... [YouTube] Fix getting stream type of live streams	2021-06-23 11:33:24 +02:00
bopol	361f142621	Use badge style instead of label	2021-06-23 11:25:59 +02:00
bopol	58f109ddec	adress stypox review	2021-06-23 11:15:40 +02:00
bopol	fe29c78fe1	Merge pull request #656 from TeamNewPipe/dependabot/gradle/org.mozilla-rhino-1.7.13 ... Bump rhino from 1.7.12 to 1.7.13	2021-06-20 00:46:42 +02:00
bopol	e9412fdadc	Merge pull request #654 from TeamNewPipe/dependabot/gradle/com.google.code.gson-gson-2.8.7 ... Bump gson from 2.8.6 to 2.8.7	2021-06-20 00:35:24 +02:00
TobiGr	1f9a99c931	[SoundCloud] Update HARDCODED_CLIENT_ID	2021-06-18 15:30:54 +02:00
Tobi	c998012c28	Revert "[YouTube] Fix getting stream type of live streams"	2021-06-18 09:20:32 +02:00
dependabot[bot]	38d0339753	Bump rhino from 1.7.12 to 1.7.13 ... Bumps [rhino](https://github.com/mozilla/rhino) from 1.7.12 to 1.7.13. - [Release notes](https://github.com/mozilla/rhino/releases) - [Changelog](https://github.com/mozilla/rhino/blob/master/RELEASE-NOTES.md) - [Commits](https://github.com/mozilla/rhino/commits) --- updated-dependencies: - dependency-name: org.mozilla:rhino dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>	2021-06-17 21:49:07 +00:00
dependabot[bot]	5a46878b09	Bump gson from 2.8.6 to 2.8.7 ... Bumps [gson](https://github.com/google/gson) from 2.8.6 to 2.8.7. - [Release notes](https://github.com/google/gson/releases) - [Changelog](https://github.com/google/gson/blob/master/CHANGELOG.md) - [Commits](https://github.com/google/gson/compare/gson-parent-2.8.6...gson-parent-2.8.7) --- updated-dependencies: - dependency-name: com.google.code.gson:gson dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>	2021-06-17 21:49:01 +00:00
TobiGr	6762fe3357	[YouTube] Fix getting stream type of live streams	2021-06-10 16:51:00 +02:00
TobiGr	f7f727d19c	Add constant NO_LIKE_COUNT to CommentsInfoItem	2021-06-08 10:28:44 +02:00
TobiGr	b70c0f93c7	Add streamPosition for comments ... SoundCloud is the only service which supports adding comments at a specific timestamp in the stream.	2021-06-08 10:27:52 +02:00
Tobi	d4186d100b	Merge pull request #580 from TeamNewPipe/accountTerminated ... Add AccountTerminatedException for terminated channels	2021-06-08 09:55:58 +02:00
TobiGr	89e0514d8b	Fix Javadoc generation fix	2021-06-08 09:30:05 +02:00
TobiGr	80d3052033	Fix Javadoc generation	2021-06-08 09:26:01 +02:00
TobiGr	fa444c8298	Remove assertations of exception messages	2021-06-06 11:38:43 +02:00
Stypox	2158ca4060	Remove two unneded warning suppressions	2021-06-05 14:15:48 +02:00
Stypox	13f192704d	Add utility method getService() to Info object	2021-06-05 14:14:19 +02:00
XiangRongLin	0ad51e76fb	Merge pull request #638 from TeamNewPipe/spelling ... Improve exception message thrown when not accepting an URL in LinkHandlerFactory#fromUrl(String url, String baseUrl)	2021-06-04 18:23:31 +02:00
XiangRongLin	faa503c7dd	Fix PeertubeAccountExtractorTest	2021-06-03 19:54:42 +02:00
TiA4f8R	ac31f3a883	Specify UTF-8 file encoding in RecordingDownloader and MockDownloader ... On Windows, mocks are recorded and read with the Cp1252 encoding so it breaks the mocks on non ASCII characters for Linux OS (and so the CI). The project is in Java 8, so we can't use FileReader(File, Charset) and FileReader(File, Charset) because these methods require Java 11. Instead of changing the Java version of the extractor, use OutputStreamWriter and FileOutputStream instead of FileWriter and InputStreamReader and FileInputStream instead of FileReader.	2021-06-03 18:38:25 +02:00
bopol	ff11c2df2a	Merge pull request #628 from litetex/fix-broken-yt-liked-comments ... Fix broken yt likes in comments	2021-05-29 11:04:11 +02:00
TobiGr	1ff56a85df	Update junit from 4.13.1 to 4.13.2 ... https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.13.2.md	2021-05-28 15:09:26 +02:00
TobiGr	858e007804	Improve exception message thrown when not accepting an URL in LinkHandlerFactory#fromUrl(String url, String baseUrl)	2021-05-28 14:59:44 +02:00
Tobi	519bba70b3	Merge pull request #630 from XiangRongLin/fix_tests ... Fix MetaInfoTest.expectedMetaInfo	2021-05-28 14:16:53 +02:00
litetex	289b84e68b	Removed unused code	2021-05-27 21:27:44 +02:00
litetex	e81b0e2885	Tweaked code	2021-05-27 19:48:31 +02:00
litetex	8c96545e57	Merge branch 'TeamNewPipe:dev' into fix-broken-yt-liked-comments	2021-05-24 18:06:45 +02:00
litetex	12fb18c310	Updated mock data (only the affected)	2021-05-24 18:05:26 +02:00
litetex	8e08a2aac7	Improved docs	2021-05-24 18:05:26 +02:00
litetex	020acfed71	Fixed tests and added new ones	2021-05-24 18:05:26 +02:00
litetex	b310922fc0	Better exception message	2021-05-24 18:05:25 +02:00
litetex	86bb9efb5d	Added likeCount back to tests	2021-05-24 18:05:25 +02:00
litetex	46654f5fca	Reformatted code	2021-05-24 18:05:25 +02:00
litetex	2174685c5c	Reimplemented likeCount	2021-05-24 18:05:25 +02:00
litetex	01cfb55505	Update YoutubeCommentsInfoItemExtractor.java	2021-05-24 18:05:25 +02:00
litetex	bedcd87abb	Fixed occurrences where voteCount is not set (<=0 likes)	2021-05-24 18:05:25 +02:00
litetex	545c0a6f42	Fixed comment	2021-05-24 18:05:24 +02:00
TiA4f8R	a00fdcbd3d	Format the indentation of the definition of the apiUrl string in SoundcloudParsingHelper.resolveFor method ... Co-authored-by: Tobi <TobiGr@users.noreply.github.com>	2021-05-23 20:27:45 +02:00
TiA4f8R	6b607eb38d	Update the hardcoded client_id	2021-05-23 18:08:44 +02:00
TiA4f8R	4552ea9c9f	Use the SOUNDCLOUD_API_V2_URL constant in all the SoundCloud package	2021-05-23 18:08:44 +02:00
TiA4f8R	86308d0603	Use a lightweight request to check if the client_id is valid ... Request the api-v2 host with the client_id instead of checking if the streams of a SoundCloud track are not empty: if it is valid, the API returns 404, otherwise it should return 401.	2021-05-23 18:08:43 +02:00
TiA4f8R	c5c190500c	Format the SoundCloud package of the extractor and fix some warnings ... Use final where possible in the package and format code to be in the 100 caracters per line limit. Fix some warnings generated by Android Studio and do some code improvements	2021-05-23 18:08:42 +02:00
Tobi	636e27333b	Merge pull request #603 from TeamNewPipe/peertube_hls ... Add PeerTube HLS streams	2021-05-23 11:49:40 +02:00
XiangRongLin	172394cd52	Update mocks	2021-05-22 20:33:20 +02:00
XiangRongLin	18c8cac40c	Fix MetaInfoTest.expectedMetaInfo ... Expected meta changed	2021-05-21 21:00:08 +02:00
Hassan Igbaria	e8b5364745	Update YoutubeStreamExtractor.java	2021-05-21 16:22:24 +03:00
litetex	c3b837fe3b	Deduped code; Using default values...	2021-05-20 20:22:40 +02:00
litetex	b934c7ccbb	Removed unused import	2021-05-20 20:11:35 +02:00
litetex	10cf081145	Removed likeCount and added textualVoteCount	2021-05-20 20:10:14 +02:00
TobiGr	1acc53b8db	[PeerTube] Add support for HLS streams	2021-05-15 12:39:23 +02:00
Tobi	3a3ade20f4	Merge pull request #620 from TeamNewPipe/bandcamp-comments ... Add Bandcamp comments (first page)	2021-05-02 13:43:53 +02:00
Fynn Godau	85a4270bd9	[Bandcamp] Fix comments test	2021-05-02 13:07:24 +02:00
TobiGr	ec50570053	Add mocks	2021-05-01 17:50:19 +02:00
TobiGr	f4404b5cc8	Add tests for more reasons	2021-05-01 17:50:19 +02:00
TobiGr	fc998589dc	Detect channels which have been terminated due to copyright infringement	2021-05-01 17:50:19 +02:00
TobiGr	bb3861ddce	Add AccountTerminatedException for better error handling of terminated channels	2021-05-01 17:50:19 +02:00
Tobi	284362fc15	Merge pull request #608 from TeamNewPipe/youtube-fix-segment-test ... [YouTube] fix tests	2021-05-01 17:24:41 +02:00
TobiGr	9c12dc5609	[SoundCloud] Fix SoundCloud ID extraction ... resolveIdWithEmbedPlayer() does not work anymore because the JSON data has been extracted to an API call. For this reason, replace resolveIdWithEmbedPlayer() with resolveIdWithWidgetApi)( which performs the API call.	2021-04-26 17:58:34 +02:00
Fynn Godau	9dc7a1d5ee	[Bandcamp] Add comments (first page)	2021-04-26 17:25:04 +02:00
TobiGr	8f023c1ec7	[SoundCloud] Update client id	2021-04-26 15:14:11 +02:00
Tobi	7f202db8b1	Fix Google URL detection (#617)	2021-04-26 11:49:47 +02:00
fynngodau	6db4bea8ca	Rework Bandcamp comments (#613)	2021-04-13 21:10:59 +02:00
Fynn Godau	3671876721	Fix bandcamp radio stream extractor	2021-04-13 19:30:11 +02:00
Fynn Godau	90b5c00599	Extract related items #593	2021-04-13 17:56:19 +02:00
TobiGr	1fe645704f	[YouTube] FIx testMoreRelatedItems in RandomQueryNoMorePages	2021-04-11 23:21:35 +02:00
TobiGr	9a2b814d8f	Update mocks	2021-04-11 22:59:01 +02:00
TobiGr	50005ce937	[YouTube] Replace unavailable YouTube Stream Segments test ... The stream is unavailable in many countries.	2021-04-11 22:58:48 +02:00
TobiGr	318cec7625	[Bandcamp] Add Radio stream segments	2021-04-11 21:19:44 +02:00
TobiGr	55e0f8e725	Merge branch 'master' into dev	2021-04-10 11:33:34 +02:00
TobiGr	33173eb3e6	Make CONSENT COOKIE constants private	2021-04-09 11:59:17 +02:00
TobiGr	b94b316558	Update mocks	2021-04-09 11:59:17 +02:00
TobiGr	4b8d4a84b6	Set the NumberGenerator for th remaining tests	2021-04-09 11:59:17 +02:00
XiangRongLin	db7ad9c27d	Update Mocks	2021-04-08 16:37:29 +02:00
XiangRongLin	4833df60e5	Add method to inject Random into YoutubeParsingHelper and use in tests	2021-04-08 16:36:55 +02:00
TobiGr	682ec27737	Update mocks	2021-04-08 14:22:31 +02:00
TobiGr	080d1e318d	Code improvement	2021-04-07 14:11:47 +02:00
TobiGr	4ee270fe01	Fix LiveStream Test	2021-04-07 14:11:41 +02:00
TobiGr	883f16e0ad	[YouTube] Set CONSENT cookie	2021-04-07 14:11:37 +02:00
XiangRongLin	aee3838ed5	Ignore unreliable SoundcloudGoPlusTrack.testRelatedItems	2021-04-06 18:26:45 +02:00
XiangRongLin	7b06c696e2	Use Page.body for YoutubeChannelExtractor	2021-04-06 18:14:38 +02:00
XiangRongLin	eda1b6e199	Use Page.body for YoutubePlaylistExtractor	2021-04-06 18:14:38 +02:00
XiangRongLin	3af26a2821	Add body field to Page	2021-04-04 11:39:58 +02:00
XiangRongLin	7b36469807	Fix SoundcloudGoPlusTrack ... https://soundcloud.com/martinsolveig/places#t=0 has related streams	2021-04-04 09:39:26 +02:00
XiangRongLin	3d70b6d46a	Fix a test and ignore rest	2021-04-03 11:03:10 +02:00
XiangRongLin	1925dcf4dc	Merge pull request #597 from fynngodau/related-items ... Rename getRelatedStreams to getRelatedItems and change return type	2021-04-02 19:54:22 +02:00
Fynn Godau	c877712647	Add deprecated old method calls to avoid breaking API	2021-04-01 22:32:36 +02:00
Fynn Godau	14f6f1b7c3	Generify related streams calls and rename method	2021-03-31 20:24:40 +02:00
TobiGr	1498e1905e	Reduce number of type conversions in YouTubeSearchExtractor.collectStreamsFrom by introducing new variable	2021-03-31 13:30:38 +02:00
Tobi	7ea2cd73d3	Merge pull request #589 from fynngodau/bandcamp-more-recent-pages ... Load more featured pages and fix featured cover arts not loading	2021-03-31 12:15:12 +02:00
Fynn Godau	705f6c6e33	Apply review	2021-03-31 11:02:30 +02:00
TobiGr	657b00ca11	Smaller code improvements	2021-03-30 10:10:53 +02:00
TobiGr	8fbd1a2bcf	FIx potential NullPointerException	2021-03-30 10:07:20 +02:00
TobiGr	f2e7c7ebd6	Replace Vector with ArrayList for performance improvements ... I could not figure out, why these two methods are necessary.	2021-03-30 10:07:20 +02:00
TobiGr	48fd8ca7b3	Remove redundant type declaration	2021-03-30 10:07:20 +02:00
TobiGr	22bd16fca3	Make vars private, final and static	2021-03-30 10:07:20 +02:00
TobiGr	6a0f6e846a	Fix possible bug when language.id field is not present and this the languageCode an empty String.	2021-03-30 10:07:20 +02:00
TobiGr	b9282bbe94	Remove checks which are always true	2021-03-30 10:07:20 +02:00
TobiGr	46eab1ec17	Remove declarations of exceptions which are not thrown from method signatures	2021-03-30 10:07:20 +02:00
TobiGr	c3e23559d7	Remove unused imports	2021-03-30 10:07:20 +02:00
TobiGr	e4c40ae6d1	Use List.isEmpty()	2021-03-30 10:07:20 +02:00
Fynn Godau	d663be5a78	[Bandcamp] Fix featured albums not showing thumbnails	2021-03-28 13:12:22 +02:00
Fynn Godau	dbcf61c6f7	[Bandcamp] Load more featured pages	2021-03-28 12:53:05 +02:00
Stypox	b4dee6d08f	Merge pull request #306 from B0pol/metadata ... Extract metadata for youtube, soundcloud & mediaccc	2021-03-27 08:45:47 +01:00
TobiGr	070a40e181	[YouTube] Mix Playlist - Fix getting name	2021-03-25 21:47:16 +01:00
TobiGr	b029779217	[media.cc.de] Disable unreliable tests for now	2021-03-25 19:44:44 +01:00
TobiGr	bbee15474d	[PeerTube] Fix playlist extraction	2021-03-25 19:19:19 +01:00
TiA4f8R	ae283314da	Add a comment explaining subscribers count calculation	2021-03-25 17:54:46 +01:00
TiA4f8R	abb790f465	Remove println in getSubscriberCount from PeertubeAccountExtractor class	2021-03-25 17:49:22 +01:00
TiA4f8R	772de53a66	Increase the number of subscribers in PeertubeAccountExtractorTest	2021-03-25 17:49:20 +01:00
TiA4f8R	efe2b964f0	Extract real subscribers of a Peertube account ... Apply fix provided in the PR and move all accounts/ strings to a constant, ACCOUNTS.	2021-03-25 17:47:44 +01:00
Tobi	12835bfae1	Merge pull request #551 from FireMasterK/ff-privacy ... Change UA to privacy.resistFingerprinting.	2021-03-24 17:58:34 +01:00
Tobi	cae6cea551	Merge pull request #582 from Stypox/fix-trending ... [YouTube] Fix trending getName()	2021-03-24 10:49:46 +01:00
Stypox	ce7cbbc9a0	[YouTube] Fix trending getName()	2021-03-24 09:14:09 +01:00
XiangRongLin	9009f1e277	Merge pull request #517 from XiangRongLin/flaky_test ... Add MockOnlyRule to allow skipping specific tests based on downloader	2021-03-20 18:04:14 +01:00
bopol	152221c7fb	ignore a test with encoding problems	2021-03-19 10:14:43 +01:00
bopol	8806fb4e6b	address reviews	2021-03-19 10:14:43 +01:00
bopol	c47cc54908	Extract metadata for YouTube, SoundCloud & MediaCCC	2021-03-19 10:14:43 +01:00
TiA4f8R	379d7312fa	Don't use a regular expression to find the last segment URL and do code improvements ... Apply suggestions provided in the PR and remove a redundant import.	2021-03-14 17:54:22 +01:00
TiA4f8R	0e3e420a25	Fix tests	2021-03-14 17:54:21 +01:00
TiA4f8R	d61d9d116d	Refactor getAudioStreams method of SoundcloudStreamExtractor ... Split the method into private methods, in order to have a better reading.	2021-03-14 17:54:20 +01:00
TiA4f8R	a7b15b51e6	Change t to transcodingJsonObject in SoundcloudStreamExtractor	2021-03-14 17:54:19 +01:00
TiA4f8R	0438828e36	Add a test for the number of audioStreams in CreativeCommonsPlaysWellWithOthers test ... It should be only two audio streams for track "Plays Well with Others, Ep 2: What Do an Army of Ants and an Online Encyclopedia Have in Common?" by Creative Commons (https://soundcloud.com/wearecc/plays-well-with-others-ep-2-what-do-an-army-of-ants-and-an-online-encyclopedia-have-in-common): - one which is a progressive stream, in MP3 format with a bitrate of 128 kbps - one which is an HLS stream, in OPUS format with a bitrate of 64 kbps.	2021-03-14 17:54:18 +01:00
TiA4f8R	3bd08a2880	Adress requested changes and use final where possible in SoundcloudStreamExtractor ... This commit moved the HLS parsing task to a separate method, did little performance improvements and used final where possible in the SoundcloudStreamExtractor file.	2021-03-14 17:54:17 +01:00
TiA4f8R	cbacd3c0a5	Add a check to don't show MP3 128kbps stream twice and catch IOException when fetching the HLS Manifest ... If a progressive stream is present in the transcodings, it's unnecessary to show twice an MP3 128kbps stream so if this is the case, the MP3 HLS stream will be not added to the audioStreams, else it will. This commit also catch fetching errors in HLS manifests parsing and don't add the corresponding stream if an error occurs.	2021-03-14 17:54:16 +01:00
TiA4f8R	26f1b4e7dc	Support SoundCloud HLS by using a workaround ... This commit tries to support SoundCloud HLS streams by parsing M3U manifests, get the last segment URL (in order to get track length) and request a segment URL equals to track's duration so it's a single URL.	2021-03-14 17:54:14 +01:00
Fynn Godau	2e57a8f24f	[Bandcamp] Fix link handler acceptance behaviour ... * Test for bandcamp footer instead of meta tag (which is not present on all pages) * Accept links to /music, not just /releases * Correctly handle uppercase URLs	2021-03-14 09:51:30 +01:00
TobiGr	70d9e389b9	[Bandcamp] Add tests for external bandcamp artists ... Necessary, because the external pages tested before were converted to bandcamp.com pages. See f9d06252f2	2021-03-13 20:51:59 +01:00
TobiGr	f9d06252f2	[Bandcamp] Fix tests	2021-03-13 17:42:39 +01:00
Tobi	7e6f464407	Merge pull request #509 from TiA4f8R/soundcloud-improvements ... Add new exceptions to be able to display different error messages in apps	2021-03-07 13:51:15 +01:00
Tobi	ac51134aed	Merge pull request #546 from XiangRongLin/unignore2 ... Unignore more tests	2021-03-07 13:15:24 +01:00
TobiGr	22fa131922	Merge branch 'dev' into bandcamp	2021-03-05 19:45:37 +01:00
TiA4f8R	e55284bb8f	Code style improvements ... Use final where possible in YoutubeStreamExtractor and do some other code style improvements	2021-03-05 16:38:53 +01:00
TobiGr	d0a1041afd	Prevent NullPointerException and use equalsIgnorecase	2021-03-05 16:38:52 +01:00
TiA4f8R	890cbba625	Update PaidContent exception	2021-03-05 16:38:51 +01:00
TobiGr	da3cfa967d	Handle age-restricted videos	2021-03-05 16:38:50 +01:00
TiA4f8R	448b68700c	Use final in exceptions and add AgeRestrictedException ... AgeRestrictedException will be thrown only if the reason message equals to "Sign in to confirm your age" and if the age limit is 18.	2021-03-05 16:38:49 +01:00
TobiGr	22e6f33f0a	Ignore SoundCloudGeoRestrictedTest until HLS stream extraction is added ... The ContentNotSupportedException is thrown because no supported audio streams where extracted. However, SoundCLoud does not check, whether there are any streams available. This commit should be reverted in #526	2021-03-05 16:38:47 +01:00
TobiGr	21158744b1	Add expectedDescriptionIsEmpty() to DefaultStreamExtractorTest ... Also check if related streams are empty if they are expected to be empty.	2021-03-05 16:38:46 +01:00
TiA4f8R	f15d7837a1	Fix SoundCloud tests	2021-03-05 16:38:45 +01:00
TiA4f8R	4e45aef2b3	Fix SoundCloud CreativeCommonsPlaysWellWithOthers test	2021-03-05 16:38:43 +01:00
TiA4f8R	59d6d3f04e	Fix YouTube tests	2021-03-05 16:38:42 +01:00
TiA4f8R	3925204658	Add tests and rename SoundcloudGoPlusException to SoundcloudGoPlusContentException	2021-03-05 16:38:40 +01:00
TiA4f8R	771bb1a2cb	Add three new exceptions for unavailable YouTube contents ... These exceptions are thrown on a test with the error messages text, because YouTube returns only "UNPLAYABLE" status in most error cases. Tests are based with English strings, so changing the lang used by extractor will throw the generic exception (ContentNotAvailableException).	2021-03-05 16:38:39 +01:00
TiA4f8R	35325d980d	Add GeographicRestrictionException and SoundCloudGoPlusException in NewPipe Extractor to be able to display different error messages ... This commit adds two new exceptions in NewPipe Extractor: GeographicRestrictionException and SoundCloudGoPlusException (which extend to ContentNotAvailableException). These exceptions allow showing different error messages to user when a content isn't available in his/her/its country (only used for now by SoundCloudStreamExtractor) or when the content is a SoundCloud Go+ track.	2021-03-05 16:38:37 +01:00
Tobi	bc0cda68d1	Merge pull request #570 from TiA4f8R/fix-yt-channels-playlists ... Fix YouTube channels contination	2021-03-05 15:26:49 +01:00
Tobi	a3c6fceef5	Merge pull request #573 from B0pol/comments-performance ... [youtube] improve comments extraction performance	2021-03-05 14:46:27 +01:00
bopol	ff5273b882	Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/extractors/YoutubeCommentsExtractor.java ... Co-authored-by: Tobi <TobiGr@users.noreply.github.com>	2021-03-05 14:39:01 +01:00
Tobi	ec1127dd6a	Merge pull request #571 from TeamNewPipe/soundcloud_client_id ... [SoundCloud] Update client_id and fix extraction	2021-03-05 14:21:14 +01:00
bopol	ed850d0688	[youtube] improve comments extraction performance ... - do not parse responseBody twice for continuation instead try to get commentsTokenInside with the new pattern ("sectionListRenderer") and try again with the old pattern ("commentSectionRenderer") on failure - do not unescape responseBody multiple times -> parse responseBody less times	2021-03-05 13:37:08 +01:00
TobiGr	440a808b8a	Update mocks for channels	2021-03-05 13:13:36 +01:00
golfinq	05213175cd	minor edit, made variable final	2021-03-05 02:49:15 -05:00
golfinq	8bceb57a6c	formatting fix	2021-03-05 02:46:06 -05:00
golfinq	ae48bdea4c	Updated to reflect variable name change	2021-03-05 02:40:36 -05:00
TiA4f8R	b27efdc9a4	Remove clickTrackingParams in getNextPage method of YoutubeChannelExtractor ... Because it's unused with the new request. Co-authored-by: bopol <bopol@e.email>	2021-03-04 22:50:20 +01:00
TobiGr	2b671b15ce	[YouTube] Use new continuation API for channels ... Co-authored-by: TiA4f8R <74829229+tia4f8r@users.noreply.github.com>	2021-03-04 21:47:16 +01:00
TobiGr	33594d4aff	[SoundCloud] Fix client id extraction ... SoundCloud changed their Javascript structure. That caused the client_id to be in the middle of the file (at byte ~43000 ). To have a little buffer, we now fetch the first 50000 bytes.	2021-03-04 21:32:34 +01:00
TobiGr	fb2f37d223	[SoundCloud] Update hardcoded client id	2021-03-04 21:25:41 +01:00
XiangRongLin	506cc5fbe9	regenerate mocks	2021-03-04 19:01:58 +01:00
XiangRongLin	03b00ff1d6	Move toJsonArray and toJsonObject to JsonUtils	2021-03-04 18:58:51 +01:00
FireMasterK	668b080b3a	Use dynamic key.	2021-03-04 08:45:14 +01:00
XiangRongLin	43b46bd408	Use new youtube continuations api for playlists ... Requires sending a POST request instead of GET. clientName and clientVersion, which were required as headers previously now need to be part of the request payload. continuation id also needs to be part of request body. quick and dirty solution.	2021-03-03 19:49:26 +01:00
XiangRongLin	beb05bd05c	Replace test url, because playlist was deleted	2021-03-03 19:46:14 +01:00
XiangRongLin	e13e237392	Fix typo and reword some explanations	2021-02-26 17:48:03 +01:00
TobiGr	a1688fe953	Move BandcampExtractorHelper.getJsonData(String, String) to JsonUtils	2021-02-21 13:51:12 +01:00
TobiGr	70814dcfef	Fix Utils.nonEmptyAndNullJoin ... When using the index here, it the index needs to be decremented once an element is removed. To cirecumvent this, the native Collections.removeIf() method is used.	2021-02-21 12:51:47 +01:00
TobiGr	b9e8ee8450	Rename BandcampExtractorHelper.smartConcatenate(String[], String) to Utils.nonEmptyAndNullJoin(String, String[])	2021-02-21 12:35:14 +01:00
TobiGr	c07db80ef0	Add BASE_URL and BASE_API_URL to BandcampExtractorHelper	2021-02-21 12:15:45 +01:00
TobiGr	adde4332d1	Code improvements	2021-02-20 23:58:54 +01:00
FireMasterK	42bdd8adc5	Update to fix failing mock tests.	2021-02-20 17:14:41 +05:30
FireMasterK	e8bccfaf5d	Change UA to privacy.resistFingerprinting.	2021-02-20 17:14:41 +05:30
bopol	0c831afa7b	use DownloaderFactory instead of DownloaderTestImpl for search ChannelVerified	2021-02-20 11:41:30 +01:00
TobiGr	ea120a4637	remove print stacktrace ... use interface	2021-02-20 00:23:45 +01:00
XiangRongLin	adf9d7d10f	Add reason field to MockOnly ... This enforces developers to document why a test is skipped	2021-02-19 18:54:18 +01:00
XiangRongLin	ea52030613	Add MockOnlyRule to allow skipping specific tests based on downloader	2021-02-19 18:54:17 +01:00
TobiGr	02920fafa8	Add isUploaderVerified()	2021-02-19 15:51:23 +01:00
TobiGr	91e9309486	Merge remote-tracking branch 'origin/dev' into bandcamp	2021-02-19 13:37:45 +01:00
TobiGr	fa61b864f2	Code improvements	2021-02-19 13:37:24 +01:00
TobiGr	98268e351c	Move radio URL check into a function	2021-02-19 13:35:44 +01:00
FireMasterK	5333d8a98b	Add more data.	2021-02-19 00:55:03 +05:30
bopol	1a322ad8ed	Add uploader verified by service extraction	2021-02-18 16:36:40 +01:00
FireMasterK	c24afa2cbb	Permalink URL.	2021-02-18 20:16:53 +05:30
FireMasterK	8d54401233	Add fps field as well.	2021-02-18 20:16:53 +05:30
FireMasterK	5b70645d85	Add data to muxed streams aswell.	2021-02-18 20:16:53 +05:30
FireMasterK	e975d33fbe	Change jdoc / comments.	2021-02-18 20:16:53 +05:30
FireMasterK	f31b2a68fd	use getters/setters.	2021-02-18 20:16:53 +05:30
FireMasterK	d4945ac55c	Constructor cleanup.	2021-02-18 20:16:52 +05:30
FireMasterK	3f6a601be8	Fix jdoc / comments.	2021-02-18 20:16:52 +05:30
FireMasterK	a7c9905183	Fix requested changes.	2021-02-18 20:16:52 +05:30
FireMasterK	0c0f2d74bc	Make fields private ... Co-authored-by: bopol <bopol@e.email>	2021-02-18 20:16:52 +05:30
FireMasterK	d1054338d3	Fix error with livestreams.	2021-02-18 20:16:52 +05:30
FireMasterK	11eb4932f4	Add data to respective classes.	2021-02-18 20:16:52 +05:30
FireMasterK	525e345ed8	remove duplicate field.	2021-02-18 20:16:52 +05:30
FireMasterK	e844d2aed3	Add dash fields to the ItagItem.	2021-02-18 20:16:52 +05:30
bopol	64f6b0478f	Merge pull request #545 from XiangRongLin/comment_reset ... Add comment explaining YoutubeParsingHelper.resetClientVersionAndKey	2021-02-18 15:14:03 +01:00
XiangRongLin	a64579da2f	Unignore some tests that work on Linux	2021-02-17 20:31:26 +01:00
XiangRongLin	b99cb8624f	Unignore PublicBroadcasterTest ... Tests only work on linux, mocks have to be generated on linux too	2021-02-17 20:31:26 +01:00
XiangRongLin	37366c65ca	Remove encoding of '&' to '&amp;' in test and unignore it	2021-02-17 20:31:26 +01:00
XiangRongLin	c327922cc0	Add explanations to broken tests	2021-02-17 20:31:26 +01:00
XiangRongLin	26d50aca05	Add comment explaining YoutubeParsingHelper.resetClientVersionAndKey	2021-02-17 19:21:39 +01:00
TobiGr	54aa5b3042	Use propper structure in KioskExtractors ... Made BandCampRadioExtractor a Kiosk which holds StreamInfoItems and not InfoItems.	2021-02-16 21:09:42 +01:00
TobiGr	54b8e54f80	Fix potential NPE	2021-02-16 20:59:13 +01:00
TobiGr	16973126a0	Use Collections.emptyList() instead of Collections.EMPTY_LIST or new ArrayList()	2021-02-16 20:25:48 +01:00
bopol	84380e4c3a	Merge pull request #536 from B0pol/watchendpoints ... support /watch/, /v/ and /w/ subpaths	2021-02-16 20:23:27 +01:00
bopol	41f689b099	Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/linkHandler/YoutubeStreamLinkHandlerFactory.java ... Co-authored-by: Stypox <stypox@pm.me>	2021-02-16 19:17:10 +01:00
XiangRongLin	a9594f531f	Merge pull request #543 from XiangRongLin/unignore1 ... Unignore and fix some tests	2021-02-16 18:00:16 +01:00
XiangRongLin	a28e0226c5	Merge pull request #519 from XiangRongLin/better_logging_error_handling ... Better logging, error handling, doc for mock tests	2021-02-16 15:06:06 +01:00
XiangRongLin	05cb22c654	Unignore Gronkh.testDescription ... Remove special character ü from assertion	2021-02-16 08:59:35 +01:00
XiangRongLin	d9b4c2779f	Unignore YoutubeSearchQHTest.testRegularValues	2021-02-16 08:59:35 +01:00
XiangRongLin	137272b7a4	Unignore YoutubeCommentsExtractorTest.EmptyComment ... Mock files were missing	2021-02-16 08:59:34 +01:00
TobiGr	5bf9fddba9	Code improvements	2021-02-15 22:45:17 +01:00
TobiGr	e062c8cb0d	Merge branch 'dev' into bandcamp	2021-02-15 22:09:41 +01:00
bopol	7a3d9bdb7d	Description: add .equals() and .hashCode()	2021-02-15 18:51:47 +01:00
bopol	557934cb17	use EMPTY_STRING	2021-02-15 18:51:46 +01:00
bopol	9ca52ca68a	use only one constant UTF-8	2021-02-15 18:47:54 +01:00
XiangRongLin	50e5718bd1	Merge pull request #518 from XiangRongLin/generate_mocks ... Use and generate mocks for youtube tests	2021-02-15 06:39:41 +01:00
Tobi	d7486f251a	Merge pull request #538 from B0pol/soundcloud-georestriction ... Soundcloud: fix charts extraction when georestricted	2021-02-14 23:52:27 +01:00
TobiGr	432c68cdec	Update okhttp from 3.12.11 to 3.12.13 ... Changelog https://square.github.io/okhttp/changelog_3x/#version-31213	2021-02-14 23:44:19 +01:00
XiangRongLin	a88b2e6af4	Undo formatting of YoutubeStreamExtractorDefaultTest	2021-02-14 14:57:23 +01:00
bopol	8d89c82caa	Soundcloud: fix charts extraction when georestricted	2021-02-13 22:28:31 +01:00
bopol	ee2012c948	rename folder to subpath	2021-02-13 22:25:46 +01:00
bopol	8db40db4de	use substring instead of split	2021-02-13 22:20:05 +01:00
bopol	81317cf6bc	youtube: support /w/ folder	2021-02-13 22:20:05 +01:00
bopol	f15c0fcfed	support /watch/ and /v/ folders	2021-02-13 22:20:05 +01:00
XiangRongLin	971128c464	Ignore failing tests	2021-02-13 20:28:03 +01:00
XiangRongLin	92dea0806a	Regenerate json files for youtube tests	2021-02-13 20:28:03 +01:00
XiangRongLin	d6b9930c18	Use DownloaderFactory in YoutubeCommentsExtractorTest.Pinned	2021-02-13 20:12:39 +01:00
XiangRongLin	b555f38fa5	Use real downloader for YoutubeSubscriptionExtractorTest ... It doesn't make network requests anyway	2021-02-13 20:12:39 +01:00
XiangRongLin	f352b0c7af	YoutubeMusicSearchExtractorTest can't be used with mocks ... Requires further research	2021-02-13 20:00:51 +01:00
XiangRongLin	f45b5610bf	Unignore YoutubeChannelLocalizationTest ... With mocks time is not a concern anymore	2021-02-13 20:00:51 +01:00
XiangRongLin	19737e06ee	Use DownloaderFactory in YoutubeChannelExtractorTest ... KurzgesagtAdditional needed to be extracted because it ran a duplicated request.	2021-02-13 20:00:51 +01:00
XiangRongLin	84eb4b30c9	Use DownloaderFactory in YouTube tests	2021-02-13 20:00:51 +01:00
XiangRongLin	b43f3474f5	Make resetClientVersionAndKey public	2021-02-13 20:00:49 +01:00
bopol	2859c7ba5c	add comment explaining why .substring(23)	2021-02-13 12:10:41 +01:00
bopol	73353996d7	fix more tests	2021-02-12 23:18:38 +01:00
bopol	a931e31239	fix tests	2021-02-12 22:36:18 +01:00
TobiGr	d116680ea3	Merge branch 'master' into dev	2021-02-12 22:26:35 +01:00
bopol	b0f356dd76	YT channels: fix continuation	2021-02-12 14:39:09 +01:00
bopol	fcdb50b825	make getInfo from Extractor public ... In StreamInfo and CommentsInfo	2021-02-09 14:58:43 +01:00
bopol	44c54d403a	fix Invidious channels being accepted by YoutubeStreamLinkHandlerFactory (#527) ... fixes https://github.com/TeamNewPipe/NewPipeExtractor/issues/524, see it for more details	2021-02-05 18:57:58 +01:00
TiA4f8R	1414a6f178	Fix extraction of YouTube chapters (#531) ... YouTube changed the name of the chapters in the JSON from engagement-panel-macro-markers to engagement-panel-macro-markers-description-chapters, so extracting chapters doesn't work.	2021-02-03 22:07:34 +01:00
Tobias Groza	bfa639950e	Merge pull request #528 from Stypox/ytmusic-0 ... [YouTube Music] Fix extracting search item view/subscriber count when = 0	2021-01-29 23:23:23 +01:00
TobiGr	d728c4fad0	Remove unused imports ... I searched for "StandardCharsets" to prevent possible bugs with Android KitKat. The import is unused and is therefor removed. There are other usages of the StandardCharsets class in the test implementation, but those are not relevant.	2021-01-29 23:16:40 +01:00
Stypox	a64dfd7343	[YouTube Music] Fix extracting search item view/subscriber count when = 0	2021-01-29 16:44:24 +01:00
bopol	2662ceca22	rename getHeartedByCreator and getPinned to isPinned and isHeartedByCreator	2021-01-23 13:06:07 +01:00
bopol	66e4eb2f96	extract whether the comment is pinned	2021-01-22 23:59:22 +01:00
bopol	67ddfefdc6	keep invidious instances list in the same order ... as suggested by @TobiGr	2021-01-22 20:58:14 +01:00
bopol	53d3f7989b	update invidious instances list	2021-01-22 19:30:25 +01:00
XiangRongLin	88e4c8667b	Add comments to RecordingDownloader about usage	2021-01-17 19:53:20 +01:00
XiangRongLin	eecfe09f2c	Check for non-existent folder in MockDownloader	2021-01-17 19:52:58 +01:00
TobiGr	07570554b6	Fix build and optimize imports	2021-01-15 21:55:40 +01:00
TobiGr	78c2113094	Merge remote-tracking branch 'origin/dev' into bandcamp	2021-01-15 21:49:58 +01:00
Stypox	b9ba95614b	Merge pull request #514 from XiangRongLin/test_jvm_property ... Pass on gradle JVM system property to test JVM	2021-01-15 20:44:23 +01:00
XiangRongLin	a6cfe11ae9	Pass on gradle JVM system property to test JVM	2021-01-15 20:37:14 +01:00
XiangRongLin	6888e7773f	Ignore all failing tests	2021-01-15 20:11:40 +01:00
Stypox	7c774c84ce	Merge pull request #505 from TeamNewPipe/meida.ccc.de_search_fix ... [media.ccc.de] Fix NPE in search results if they contain a future talk	2021-01-15 14:16:59 +01:00
Stypox	c2c4d97efd	Merge pull request #482 from XiangRongLin/mock_mix_pl_test ... Mock mix pl tests	2021-01-15 12:54:48 +01:00
XiangRongLin	4f81d9d689	Regenerate mock files	2021-01-15 09:51:23 +01:00
Stypox	951159f75f	Merge pull request #504 from TeamNewPipe/media.ccc.de_recent_order ... [media.ccc.de] Recent kiosk: order streams by upload date	2021-01-14 23:31:09 +01:00
Tobias Groza	cc51c5f6a3	Merge pull request #507 from vkay94/add-interval-to-stream-frames ... Add durationPerFrame to Frameset	2021-01-14 20:27:40 +01:00
XiangRongLin	6ca7123d5d	Add comment for usage of YoutubeParsingHelper.resetClientVersionAndKey	2021-01-14 20:01:52 +01:00
vkay94	11dcfe638b	Add durationPerFrame to Frameset and getFrameBoundsAt method	2021-01-14 20:01:06 +01:00
Tobias Groza	c2ff6723d7	Merge pull request #511 from B0pol/errors ... Fix PeerTube tests by changing instance	2021-01-14 19:15:22 +01:00
Tobias Groza	6773ec8699	Merge pull request #510 from B0pol/heartedbycreator ... comments: add heartedByUploader and extract it for YouTube	2021-01-12 19:00:04 +01:00
TobiGr	0c4836c800	[media.ccc.de] Recent kiosk: Add comment clarifying "length" and "duration" fields have the same value	2021-01-12 18:48:42 +01:00
XiangRongLin	4af50c6870	Use @link for downloader	2021-01-12 09:41:57 +01:00
XiangRongLin	3455f0f23c	Add documentation to testing with mocks	2021-01-12 09:37:48 +01:00
bopol	5292cc1636	[peertube] test: remove usage of peertube.mastodon.host ... It's down for months	2021-01-11 19:46:27 +01:00
bopol	f3c22da618	comments: add heartedByCreator and extract it for youtube	2021-01-11 18:48:08 +01:00
XiangRongLin	63c237db41	Only create directories if they do not exist	2021-01-11 07:57:11 +01:00
XiangRongLin	e82cb1efea	Remove dependency to commons-io	2021-01-10 20:50:38 +01:00
XiangRongLin	1ea6c6ce54	Prefix mock file from RecordingDownloader with "generated_mock_" ... Only read those files in MockDownloader	2021-01-10 20:46:42 +01:00
XiangRongLin	f91916c017	Remove unnecessary file object creation	2021-01-10 20:39:22 +01:00
XiangRongLin	255c726f20	Add resource path base to DownloaderFactory	2021-01-10 20:27:34 +01:00
XiangRongLin	35e299759e	Add method to allow resetting youtube client version and key ... This is needed so that a request is made for each test class when running multiple at once. This way RecordingDownloader records all necessary requests. This works as long as tests are run sequentially and not in parallel.	2021-01-10 20:24:50 +01:00
XiangRongLin	f447a7a450	fix import due to downloader package move	2021-01-10 19:49:14 +01:00
XiangRongLin	f8aa989c42	Add generated json and manually copy client_version.json	2021-01-10 19:38:47 +01:00
XiangRongLin	285c26eafa	Adjust YoutubeMixPlaylistExtractorTest to use DownloaderFactory	2021-01-10 19:38:41 +01:00
XiangRongLin	e6e8e39def	Add DownloaderFactory to return a specific downloader based on 2 variables. ... If the system property 'downloader' is set that use that specific downloader. This is used from gradle by appending `-Ddownloader=ABCD to the command. ABCD is one of DownloaderType. The other variable is the static property `DEFAULT_DOWNLOADER` in DownloaderFactory, which can be easily changed as needed inside the IDE according to development needs`. Normal workflow would be to first use the recording downloader and afterwards only use mocks, if the requests are always staying the same.	2021-01-10 19:36:24 +01:00
XiangRongLin	7c40fb8bf7	Add additional downloader implementations ... RecordingDownloader relies on the real downloader and saves the request/response pair into a json file. MockDownloader uses json files from above and mocks responses for specific requests.	2021-01-10 19:36:24 +01:00
XiangRongLin	1bcb9c76a7	Generate equals and hashCode for Request	2021-01-10 19:36:24 +01:00
XiangRongLin	4dad3d60d6	Move DownloaderTestImpl into downloader package	2021-01-10 19:36:21 +01:00
XiangRongLin	fe3902c669	Un-comment out 360p webm	2021-01-08 19:43:03 +01:00
TobiGr	2d93b23723	[media.ccc.de] Show conference title in search results instead of slug	2021-01-08 14:15:25 +01:00
TobiGr	52f3bd15f2	[media.ccc.de] Add conference title to "recent" kiosk ... The "conference_title" field was added upstream recently (see https://github.com/voc/voctoweb/issues/495)	2021-01-08 12:04:39 +01:00
Tobias Groza	ab62464e86	Merge pull request #495 from B0pol/sepiasearch ... [PeerTube] Implement sepia search	2021-01-06 11:06:36 +01:00
bopol	0bcea9c2cd	add JDoc	2021-01-01 14:37:32 +01:00
bopol	0e45c25ea9	Update extractor/src/main/java/org/schabi/newpipe/extractor/services/peertube/extractors/PeertubeStreamInfoItemExtractor.java ... Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>	2021-01-01 14:20:47 +01:00
TobiGr	fae67fbd45	Merge remote-tracking branch 'origin/master' into dev	2021-01-01 13:06:11 +01:00
TobiGr	d63d20f943	Fix JDoc	2020-12-31 18:05:40 +01:00
TobiGr	2cbc3ccb1a	[media.ccc.de] Fix live stream test if there are no current live streams	2020-12-31 17:03:23 +01:00
TobiGr	a896ec41b9	[media.ccc.de] Use Pattern for getting the ive stream id	2020-12-31 17:03:23 +01:00
TobiGr	5dc9a76e3c	[media.ccc.de] Recent kiosk: order streams by upload date	2020-12-30 18:38:09 +01:00
TobiGr	edf8dd0e92	[media.ccc.de] Fix NPE in search results if they contain a future talk	2020-12-30 18:34:28 +01:00
Tobias Groza	3c8c8e7307	Merge pull request #497 from TiA4f8R/fix-player-js-url-redirect ... Fix YouTube player JavaScript URL redirection	2020-12-27 18:04:40 +01:00
TiA4f8R	d30caeb091	Fix YouTube player JavaScript URL redirection	2020-12-27 15:55:03 +01:00
TobiGr	deb9af7bf5	[media.ccc.de] live kiosk - display info of the current talk	2020-12-27 15:52:09 +01:00
TobiGr	b4e1913971	[media.ccc.de] Play live streams	2020-12-27 13:25:48 +01:00
TobiGr	80f4d42226	[Draft] Add live extractor	2020-12-27 11:01:22 +01:00
TobiGr	ed9402c002	[media.ccc.de] Escape "." in id pattern	2020-12-27 11:01:10 +01:00
TobiGr	674f2227f3	make url pattern static	2020-12-27 01:27:37 +01:00
TobiGr	090dade5b3	[media.ccc.de] Add short test for recent kiosk	2020-12-27 01:03:08 +01:00
TobiGr	0cfefe222a	Revert "[media.ccc.de] Get conference name from API URL" ... This reverts commit d1a0686d2f.	2020-12-26 16:55:44 +01:00
TobiGr	d1a0686d2f	[media.ccc.de] Get conference name from API URL	2020-12-26 13:15:03 +01:00
TobiGr	41699798fe	[media.ccc.de] Use web URL for uploaderUrl instead of API URL in recent	2020-12-26 12:47:32 +01:00
TobiGr	b2154c6e87	[media.ccc.de] Fix conferences kiosk test	2020-12-25 23:55:45 +01:00
TobiGr	ed6ae329b3	[media.ccc.de] Add "recent" kiosk	2020-12-25 22:26:58 +01:00
Tobias Groza	b5e50cc9fb	Merge pull request #484 from TeamNewPipe/yt_,music_search ... Fix YouTube Music search	2020-12-25 18:30:25 +01:00
TobiGr	d9e2da53c3	Replace explicit string checks whether a playlist íd is a certain YouTube Mix type with calling the dedicated methods	2020-12-25 15:00:34 +01:00
TobiGr	abee0a8df1	Fix typos in YoutubePlaylistLinkHandlerFactory	2020-12-24 10:34:21 +01:00
TobiGr	3033c0b993	[YouTube] Music Mix: Fix playlist not being accepted ... Regression introduced by YouTube Mix support (#280)	2020-12-23 21:08:36 +01:00
TobiGr	b62144b49d	[YouTube] Music: Fix search extraction caused by changed JSON response	2020-12-23 21:08:36 +01:00
TobiGr	c9d9bd1e24	Fix typo in ListLinkHandlerFactory#getUrl	2020-12-23 21:05:58 +01:00
bopol	3ae924a7f1	Implement sepia search	2020-12-23 15:07:55 +01:00
TobiGr	942d840624	Fix typo in AffiliateService: rename "NO_AFILIATE" to "NO_AFFILIATE"	2020-12-23 12:50:45 +01:00
TobiGr	865c42e273	Fix line break	2020-12-23 12:49:50 +01:00
Fynn Godau	c9e9953bb0	[Bandcamp] Fix channel link handler factory	2020-12-22 20:02:53 +01:00
TobiGr	41a8ed625d	Make some vars final and add annotations to methods	2020-12-22 18:19:26 +01:00
TobiGr	bc6de14952	Extract stream and search meta info for YouTube ... Add method to extract Google webcache URLs.	2020-12-22 18:19:26 +01:00
TobiGr	50903730b1	[Bandcamp] Fix accepting HTTP URLs	2020-12-22 08:53:10 +01:00
TobiGr	838dc1ab54	Fix vulnerability in junit	2020-12-21 23:22:16 +01:00
bopol	8d3436565f	[peertube] fix related streams api url when no tags ... fixes #489	2020-12-19 23:25:37 +01:00
Tobias Groza	22a415156f	Merge pull request #479 from vkay94/stream-segments ... Extract stream segments for YouTube	2020-12-19 11:12:22 +01:00
TobiGr	74b46fed2d	[Bandcamp] Fix deprecation in parseDate helper function	2020-12-19 10:40:21 +01:00
TobiGr	116e921d6c	Merge remote-tracking branch 'origin/dev' into bandcamp	2020-12-19 00:26:11 +01:00
TobiGr	df16a8646f	[Bandcamp] Improve radio stream extractor test	2020-12-18 23:15:59 +01:00
XiangRongLin	cdcb66b93e	Clean up stuff from mix pl ... Coming from unclean merge and missing hints from IDE	2020-12-16 08:35:28 +01:00
XiangRongLin	ba8782a9ed	Use lowercase string for extracting cookies ... The map in a real response happens to ignore upper-/lowercase differences. Other maps used in unit test may not have that behaviour.	2020-12-16 08:32:04 +01:00
Stypox	85fa006214	Merge pull request #280 from XiangRongLin/mixPL ... Extractor for youtube mix (auto-generated playlist)	2020-12-14 18:11:07 +01:00
XiangRongLin	f90f6fcf92	[YouTube] Don't escape & in getUrlFromNavigationEndpoint for playlists	2020-12-12 20:40:13 +01:00
Xiang Rong Lin	a338e4e08e	[Youtube] Apply review suggestions and avoid channel mix edge case	2020-12-12 20:32:43 +01:00
Xiang Rong Lin	22d2f7e400	[Youtube] Add cookies to youtube mix request ... This way youtube wont return duplicates when getting more items of the mix (but youtube can also track us)	2020-12-12 20:32:40 +01:00
Stypox	421935401f	[Youtube] Add subchannel functions to mix and fix imports	2020-12-12 20:30:57 +01:00
Xiang Rong Lin	3ff8619bcc	[Youtube] apply wb9688 suggestion (mix) ... Channel mix adjusments and test Don't accept youtube music mix urls as playlist Don't override playlistData to keep getInitialPage() Remove json constants Indentation	2020-12-12 20:30:57 +01:00
Xiang Rong Lin	822cf307f7	[Youtube] Add _ITEMS constants and improve code style ... Move thumbnail id exctraction code to getThumbnailUrlFromId Add test for "My mix" detection to service tests Use ITEM_COUNT_UNKNOWN everywhere instead of -1 and add some tests	2020-12-12 20:30:57 +01:00
Xiang Rong Lin	df38b1926c	[Youtube] Add tests and take thumbnail image always from first video of mix ... Also fix getThumbnailUrl for "My Mix"	2020-12-12 20:30:57 +01:00
Xiang Rong Lin	68a3948af6	[Youtube] Fix get banner url	2020-12-12 20:30:57 +01:00
Xiang Rong Lin	ec6b99c082	[Youtube] Adjust mix extractor to new user agent ... Also extract continuation now	2020-12-12 20:30:57 +01:00
Xiang Rong Lin	0ff054acb4	[Youtube] Extract initial playlist info	2020-12-12 20:30:57 +01:00
Xiang Rong Lin	a376792a5d	[Youtube] Handle case where url is in "youtube.com/playlist?list=listID" format. ... This occurs when sharing a mix from the official youtube app.	2020-12-12 20:30:57 +01:00
Xiang Rong Lin	327a5730a8	[Youtube] Add some comments to mix	2020-12-12 20:30:57 +01:00
Xiang Rong Lin	d74265c846	[Youtube] Extract getThumbnailUrl into method and change getUploaderName	2020-12-12 20:30:57 +01:00
Xiang Rong Lin	0efb854d27	[Youtube] Implement mix extractor for auto-generated playlists. ... -New YoutubeMixPlaylistExtractor, that extracts from a mix (auto-generated playlist). -The url has the format of "youtube.com/watch?v=videoID&playlistID", where playlistID always starts with "RD" and usually followed by the videoID. -Change YoutubePlaylistLinkHandlerFactory to create a linkhandler with the given url if it is a mix. -Change YoutubeService to return YoutubeMixPlaylistExtractor if the url is a mix.	2020-12-12 20:30:57 +01:00
vkay94	2ba27b39af	Stream segments: Remove replaceAll from setPreviewUrl	2020-12-12 15:00:45 +01:00
vkay94	9a7a224a54	Add tests for stream segments	2020-12-12 10:24:55 +01:00
vkay94	d3eea4f4be	Add stream segments to StreamInfo	2020-12-12 10:24:29 +01:00
Isira Seneviratne	57be1f1123	Use Objects.requireNonNull().	2020-12-11 06:09:57 +05:30
Isira Seneviratne	4c19a88612	Use Objects.toString().	2020-12-11 06:09:50 +05:30
Isira Seneviratne	b8f64595a2	Use Objects' static equals() and hashCode() methods.	2020-12-11 06:09:44 +05:30
vkay94	9dbacbc618	Fix comments parsing	2020-12-09 14:57:38 +01:00
Fynn Godau	04dd3d4d32	Rework link handlers to correctly accept external websites	2020-12-05 15:08:26 +01:00
TheAssassin	ba3e2302bc	Fix name of media.ccc.de service	2020-12-01 23:54:27 +01:00
Tobias Groza	b7a995187f	Merge pull request #463 from opusforlife2/update_invidious_URLs ... Update Invidious URL List	2020-11-28 13:30:32 +01:00
Fynn Godau	be562b8436	Change tests	2020-11-27 16:48:40 +01:00
TobiGr	c91e21b37c	[Bandcamp] Tests - Add finals and improve code formatting	2020-11-24 14:13:31 +01:00
TobiGr	99e7ef013e	[Bandcamp] Apply small changes to code formatting and style ... Make variables final when possible Remove unused imports Improve code formatting	2020-11-24 14:13:31 +01:00
fynngodau	8c369b0f79	Rephrase link in javadoc ... Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>	2020-11-24 14:01:31 +01:00
Tobias Groza	650f0920fe	Merge pull request #465 from XiangRongLin/playlist_continuation ... [YouTube] Fix playlist continuations extraction	2020-11-20 20:48:20 +01:00
TobiGr	334e1e9b53	Fix YouTube comments test	2020-11-20 19:53:53 +01:00
Fynn Godau	186936d041	Various changes regarding tests	2020-11-19 22:33:52 +01:00
Fynn Godau	6bc7e3420e	Merge TNP/dev into fynngodau/dev	2020-11-19 21:32:08 +01:00
XiangRongLin	5bceff0083	[YouTube] Fix extraction of next page url for the last page of playlist	2020-11-18 19:03:12 +01:00
XiangRongLin	8347e14952	[YouTube] Fix playlist continuations extraction	2020-11-17 21:04:53 +01:00
opusforlife2	2174055ef8	Update Invidious URL list in Parsing Helper	2020-11-11 15:12:31 +00:00
opusforlife2	c0afd5213a	Update Invidious URL list in Link Handler Factory	2020-11-11 14:54:16 +00:00
Stypox	89a77ae74a	[YouTube] Fix detection of ended livestreams and parse livestream upload date	2020-11-04 16:03:08 +01:00
Stypox	827f7bd137	[YouTube] Cache deobfuscation and improve requests made ... Fix age restriction extraction Automatically fixes more things	2020-11-04 16:02:12 +01:00
Isira Seneviratne	4fe28d7e3a	Fix YouTube parse error when only a date is present.	2020-11-04 05:49:42 +05:30
Isira Seneviratne	9cf9e7e980	Call existing constructor in DateWrapper.	2020-11-04 05:49:41 +05:30
Isira Seneviratne	fe31a90cb3	Remove DateTimeFormatter.ISO_OFFSET_DATE_TIME usage.	2020-11-04 05:49:41 +05:30
Tobias Groza	b13c7e1c1e	Merge pull request #452 from Stypox/yt-import ... Implement YouTube subscription import from Google takeout	2020-11-03 20:32:17 +01:00
bopol	345e136f6c	create YouTubeCommentsLinkHandlerFactoryTest ... and remove invidious test from YouTubeCommentsExtractorTest, because it was just testing if the URL is accepted, then the extractor does the same thing, we don't need to test the same thing twice	2020-11-03 19:10:10 +01:00
Stypox	501ec30152	Implement youtube subscription import from Google takeout	2020-11-02 11:04:48 +01:00
Bri@n	82746d172f	Fix typo in DonationLinkHelper and rewrote swtich statement	2020-11-01 17:34:34 -05:00
Stypox	2f02c0e6a4	Merge pull request #450 from B0pol/fileutils ... create FileUtils	2020-11-01 18:13:57 +01:00
bopol	f69b3ef05d	create FileUtils	2020-11-01 17:15:34 +01:00
TobiGr	be9e160333	Fix build	2020-11-01 17:04:02 +01:00
Isira Seneviratne	4f04cfccca	Switch from Calendar to OffsetDateTime in DateWrapper.	2020-11-01 06:28:54 +05:30
Isira Seneviratne	ee3af63c04	Switch to ChronoUnit.	2020-11-01 06:28:53 +05:30
Stypox	6cc50b57e3	Merge pull request #410 from Scrxtchy/sets-track-title ... Fix parsing Soundcloud tracks that contain the term 'sets'	2020-10-31 21:43:21 +01:00
Stypox	b242e1d113	Merge pull request #362 from Stypox/google-search-urls ... Add support for Google search redirect url	2020-10-31 21:03:48 +01:00
Stypox	9e53cf0b56	Fix parameter reassignment and other style issues ... Also remove left-behind debug statement	2020-10-31 20:58:28 +01:00
Stypox	3fe55b30ba	Add support for Google search redirect url	2020-10-31 20:58:28 +01:00
Stypox	c190a3029b	Consider protocol as base url when it is a custom one (e.g. vnd.youtube)	2020-10-31 20:58:28 +01:00
bopol	b21e59925d	[PeerTube] fix account and channel extractors	2020-10-29 19:52:29 +01:00
Stypox	30ed4f2d63	Remove any reference to decrypt and improve error message	2020-10-28 20:26:00 +01:00
TobiGr	62912ee834	Merge branch 'master' into dev	2020-10-28 14:28:06 +01:00
Scratch	947ce3ee10	Fix parsing Soundcloud tracks that contain the term 'sets'	2020-10-28 07:55:02 +11:00
bopol	6dc5ab4015	find playerUrl in another place when assetsPattern regex fails	2020-10-27 13:48:58 +01:00
bopol	db0ef83d6b	fix youtube decryption and three attemps bug ... fixes teamnewpipe/newpipe#4572 fixes #439	2020-10-27 13:40:24 +01:00
bopol	c1e9857960	fix subscriber count when subscribe is disabled ... fixes #305	2020-10-26 19:06:01 +01:00
bopol	a39a2cca82	fix redirect channels	2020-10-26 19:05:19 +01:00
bopol	0a12300c5e	polish tests	2020-10-26 16:57:37 +01:00
bopol	01f49e8f66	polish strings	2020-10-26 16:32:39 +01:00
Tobias Groza	14c0c37d64	Merge pull request #309 from Stypox/improve-stream-tests ... Improve stream extractor tests & various fixes	2020-10-25 18:49:53 +01:00
Kiril Isakov	366ed71523	[YouTube] Ignore leading characters in video id	2020-10-25 14:34:20 +01:00
Stypox	8026304a0a	Merge pull request #409 from Bartoshr/bugfix/short-channel-urls ... Support short custom YouTube channel URLs	2020-10-25 11:43:50 +01:00
Stypox	57e7994c9e	Add some missing finals, nullables and comments	2020-10-25 08:12:41 +01:00
Stypox	f11fe87688	[YouTube] Replace outdated PewDiePie video test with another one ... The old video was made private, and this video (wedding) is probably never going to be removed.	2020-10-24 18:41:41 +02:00
Stypox	3c55ea9321	[PeerTube] Change age restricted video in tests ... The old one wasn't available anymore	2020-10-24 18:41:41 +02:00
Stypox	880ff04a5c	[Test] Add stream dash mpd url test	2020-10-24 18:41:41 +02:00
Stypox	d0b14644bb	[YouTube/MediaCCC] Consider dates as GMT and not as local	2020-10-24 18:41:41 +02:00
Stypox	8ce711f40f	[Test] Add sub channel name, url and thumbnail tests	2020-10-24 18:41:41 +02:00
Stypox	6127826571	[Test] Add stream metadata tests	2020-10-24 18:41:41 +02:00
Stypox	a087b092b4	[Test] Improve code style and add final	2020-10-24 18:39:55 +02:00
Stypox	a4097d8d01	[MediaCCC] Return empty list of video-only streams instead of null	2020-10-24 18:39:55 +02:00
Stypox	8dc3f28618	[PeerTube] Test one channel url with api and one without	2020-10-24 18:39:55 +02:00
Stypox	68d23defba	[YouTube] Do not catch every exception on getErrorMessage ... @B0pol suggestion	2020-10-24 18:39:55 +02:00
Stypox	3191bd6c70	[YouTube] Use final when possible	2020-10-24 18:39:55 +02:00
Stypox	55bc01d1ce	[SoundCloud] Use final when possible, ide refactorings	2020-10-24 18:39:55 +02:00
Stypox	06430c4749	[PeerTube] Use final when possible, ide refactorings	2020-10-24 18:39:06 +02:00
Stypox	fcb9b6f855	[MediaCCC] Use final when possible, ide refactorings ... Refactorings on `throws` clause	2020-10-24 18:36:22 +02:00
Stypox	af5b8b1915	[MediaCCC] Add tests for stream and conference link handlers	2020-10-24 18:36:21 +02:00
Stypox	07a90d116a	[MediaCCC] Use regex to parse stream and conference urls	2020-10-24 18:36:21 +02:00
Stypox	0c980b2d64	[PeerTube] Improve channel and stream link handler tests	2020-10-24 18:36:21 +02:00
Stypox	d130fd79c3	[PeerTube] Prepend "accounts/" to channel id for backward compatibility	2020-10-24 18:36:21 +02:00
Stypox	492db83ccf	[MediaCCC] Return null instead of empty items collector ... As per the documentation in the base getRelatedStreams()	2020-10-24 18:34:57 +02:00
Stypox	3b2a1829d4	[MediaCCC] Extract tags	2020-10-24 18:34:57 +02:00
Stypox	b461da792f	[MediaCCC] Fix link handler inconsistency providing API links	2020-10-24 18:34:34 +02:00
Stypox	aeeae87641	[PeerTube] Parse timestamp from url (previously unimplemented)	2020-10-24 18:33:42 +02:00
Stypox	7ae3cb6d07	[PeerTube] Fix link handler inconsistency providing API links	2020-10-24 18:33:11 +02:00
Stypox	4349be13af	[PeerTube] Return empty audio stream list instead of null	2020-10-24 18:25:56 +02:00
Stypox	3b2cfb4ca2	[SoundCloud] Return empty video stream list instead of null ... Also replace every instance of `return new ArrayList<>();` with `return Collections.emptyList();`	2020-10-24 18:23:46 +02:00
Stypox	072bae321f	[YouTube] Fix frame extraction for livestreams ... Use saved playerResponse instead of parsing json every time	2020-10-24 18:23:46 +02:00
Stypox	7cd410f3fc	[YouTube] Return 0 when there is no timestamp, not -2, as per javadoc	2020-10-24 18:23:45 +02:00
Stypox	7fb867c166	[YouTube] Fix error message obtaining when there is none	2020-10-24 18:22:44 +02:00
Stypox	1d7a86e664	[Test] Add base classes for stream extractor tests ... Refactor all stream extractor tests to use new base class. Remove check if upload date is in the past: this does not have to hold true: youtube premieres turn up in search results even though they are in the future	2020-10-24 18:22:44 +02:00
Bartosz Rumiński	29695aed0a	Small field refactor	2020-10-23 16:42:30 +02:00
Bartosz Rumiński	0e67d820bc	Use static regex pattern for excluded path segments	2020-10-22 20:13:29 +02:00
Bartosz Rumiński	d3f80d1538	Exlude links which are not channels	2020-10-20 20:06:06 +02:00
Stypox	6887d59570	[YouTube] Handle urls for Shorts	2020-10-18 12:03:01 +02:00
Scratch	e945f711c1	Fix SoundCloud test artist account name (#416)	2020-10-16 23:42:46 +02:00
wb9688	be9a6f931c	Fix parsing new ytInitialData	2020-10-16 20:27:40 +02:00
Tobias Groza	1a9bc8caf1	Merge pull request #414 from Scrxtchy/soundcloud-slash ... Remove tailing slash from SoundCloud URLs	2020-10-16 09:48:44 +02:00
Scratch	6a70cb9d50	Remove tailing slash from SoundCloud URLs ... Fixes #412	2020-10-16 17:17:40 +11:00
Tobias Groza	9a785f9a7e	Merge pull request #413 from Scrxtchy/soundcloud-test-update ... Fix SoundCloud test artist account name	2020-10-16 08:05:44 +02:00
Scratch	538f5d3973	Fix SoundCloud test artist account name	2020-10-16 13:45:05 +11:00
Stypox	19e862657a	[YouTube] Fix some decryption exceptions by retrying	2020-10-15 20:16:44 +02:00
Bartosz Rumiński	e3f996e014	Exlude links which are not channels	2020-10-12 20:59:56 +02:00
Bartosz Rumiński	5ab1b053d2	Update youtube channel link handler tests ... Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>	2020-10-12 20:11:28 +02:00
Bartosz Rumiński	7abb4b3713	Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/linkHandler/YoutubeChannelLinkHandlerFactory.java ... Fix typos Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>	2020-10-12 19:57:45 +02:00
Bartosz Rumiński	9d63211a66	Fix typos ... Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>	2020-10-12 19:56:53 +02:00
Bartosz Rumiński	8c38a5509e	Prevent attribution_link urls to be accepted by channel links handler	2020-10-09 19:07:38 +02:00
Bartosz Rumiński	4e04991762	Support short custom youtube channel urls	2020-10-09 17:06:59 +02:00
Fynn Godau	81b5e7cf3d	Fix extractor	2020-10-08 17:56:03 +02:00
wb9688	ebbfe7f6d4	Skip YouTube's OTF streams	2020-09-29 10:48:02 +02:00
Tobias Groza	6633f26ec5	Merge pull request #386 from mhmdanas/remove-subtitles-stream-get-url ... Remove SubtitlesStream#getURL()	2020-08-26 21:32:36 +02:00
wb9688	7657c2ed1a	Use initSafeStandardsObjects()	2020-08-15 17:08:07 +02:00
mhmdanas	1a63dcb355	Remove overriden field url	2020-08-12 22:37:36 +03:00
mhmdanas	f0f1c009b2	Remove SubtitlesStream#getURL()	2020-08-10 19:28:56 +03:00
Fynn Godau	cfe88a74c1	Throw ContentNotSupportedException when opening radio uploader channel	2020-08-09 10:07:49 +02:00
TobiGr	0fb73301e3	[YouTube] Fix crash on empty comment ... Closes #380	2020-08-05 18:25:40 +02:00
Fynn Godau	932d094d6a	Upgrade featured playlist urls to https	2020-08-02 16:55:18 +02:00
Fynn Godau	c12ef3a02d	Merge TNP/dev into fynngodau/dev	2020-08-02 16:43:04 +02:00
TobiGr	5ac80624a4	Fix detection of YouTube's decryption function ... Closes TeamNewPipe/NewPipe#3951	2020-07-28 01:01:10 +02:00
Tobias Groza	32dff1541f	Merge pull request #370 from wb9688/fix-yt-age-restricted ... Fix age restricted YouTube videos	2020-07-26 14:17:19 +02:00
TobiGr	ec778200d9	Add comments to the code	2020-07-26 14:15:13 +02:00
wb9688	667dce034c	Only use fallback methods for clientVersion when clientVersion hasn't been successfully extracted yet	2020-07-26 13:14:25 +02:00
wb9688	f345f667e2	Extract YouTube's key	2020-07-26 12:01:59 +02:00
wb9688	4c987a5302	Support YouTube's new continuations for search	2020-07-26 10:01:03 +02:00
wb9688	81459e289f	Fix age restricted YouTube videos	2020-07-18 19:52:54 +02:00
wb9688	fc3a63fec5	Update Javadoc for getRelatedStreams()	2020-07-15 18:48:37 +02:00
wb9688	ff560e907a	Remove getNextStream() from StreamExtractor	2020-07-15 18:48:37 +02:00
Tobias Groza	a70cb0283f	Merge pull request #314 from wb9688/remove-getnextpageurl ... Next page stuff	2020-07-07 20:49:38 +02:00
wb9688	0a5a905bc7	Add final at more places	2020-07-07 20:45:47 +02:00
wb9688	17ba8a57fa	Clean up the code	2020-07-06 20:19:31 +02:00
wb9688	9b6fe1dea6	Throw IllegalArgumentException when Page is invalid	2020-07-06 20:19:31 +02:00
wb9688	4cc312086a	Introduce Page class	2020-07-06 20:19:31 +02:00
wb9688	e3bfdba135	Remove getNextPageUrl() function from ListExtractor	2020-07-06 20:11:40 +02:00
wb9688	156a26b64b	Support start= timestamps for YouTube	2020-07-06 14:08:40 +02:00
bopol	261471e7f4	remove invidiou.sh from tests ... dead instance	2020-07-02 21:33:45 +02:00
bopol	eb3901acee	update invidious instances	2020-07-02 21:31:05 +02:00
Tobias Groza	c0ceb5cb27	Merge pull request #340 from wb9688/peertube-deleted-comments ... Handle isDeleted for PeerTube comments	2020-06-28 10:21:37 +02:00
Tobias Groza	339f1d9c43	Merge pull request #349 from wb9688/okhttp ... Use OkHttp for tests like in NewPipe	2020-06-27 20:08:02 +02:00
Tobias Groza	970bc7f69d	Merge pull request #359 from mauriciocolli/fix-client-id-bug ... [SoundCloud] Fix concurrency issue when getting the client id	2020-06-27 17:36:24 +02:00
Robin	576754982e	Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/YoutubeParsingHelper.java ... Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>	2020-06-27 11:35:01 +02:00
Mauricio Colli	7505e828cc	[SoundCloud] Update client id	2020-06-24 01:50:09 -03:00
Mauricio Colli	d2f1c0f40d	[SoundCloud] Fix concurrency issue when getting the client id	2020-06-24 01:50:08 -03:00
Robin	b63ae93495	Added a comment ... and fixed a "typo" in the method parseDurationString replacing Long with Integer. These are not the droids you are looking for wb9688 😁	2020-06-15 11:27:44 +02:00
Tobias Groza	92f6754f0f	Merge pull request #338 from wb9688/parse-duration-string-test ... Add tests for parseDurationString()	2020-06-15 11:01:21 +02:00
Stypox	5808439c3f	[PeerTube] Change age restricted video in tests ... The old one wasn't available anymore	2020-06-14 19:57:39 +02:00
wb9688	3239aa84f2	Handle isDeleted for PeerTube comments	2020-06-14 19:27:20 +02:00
wb9688	a65e46e4b1	Use OkHttp for tests like in NewPipe	2020-06-11 15:06:08 +02:00
Fynn Godau	96de834b67	Bandcamp: parse date from radio info item	2020-06-04 19:36:58 +02:00
Fynn Godau	39b55b5b42	Bandcamp radio stream extractor: test uploader url	2020-06-04 19:32:19 +02:00
Fynn Godau	9a555d97e5	Remove useless bandcamp tests	2020-06-04 19:31:01 +02:00
Fynn Godau	e13f341a45	Bandcamp stream extractor: test service id	2020-06-04 19:29:37 +02:00
Fynn Godau	4dd9540782	Bandcamp license switch order	2020-06-04 19:27:39 +02:00
Fynn Godau	52103ac61f	Remove "useless" comments	2020-06-04 19:27:13 +02:00
Fynn Godau	8fa81537c0	Better bandcamp stream extractor description test	2020-06-04 19:17:51 +02:00
Fynn Godau	0950a95577	Fix for testing wrong object for null or empty	2020-06-04 19:16:08 +02:00
Fynn Godau	e6ecd91929	Bandcamp: Test full date	2020-06-04 19:12:10 +02:00
Fynn Godau	27e7aad159	Bandcamp: Improve code style	2020-06-04 19:04:25 +02:00
fynngodau	ea49202f64	Don't display internal license id ... Co-authored-by: bopol <bopol@e.email>	2020-06-04 18:26:58 +02:00
bopol	063f4be766	For #342: support PeerTube embeds	2020-06-04 14:26:42 +02:00
Fynn Godau	34b6928124	Don't print stacktrace before throwing new exception	2020-06-04 14:25:38 +02:00
Fynn Godau	1be20ceeec	Bandcamp: parse date	2020-06-04 14:09:21 +02:00
Fynn Godau	6822fe3dc8	Bandcamp: Get Stream upload date	2020-06-04 13:03:16 +02:00
Fynn Godau	e98bff8ae9	Improved Bandcamp radio stream extractor tests	2020-06-04 12:30:53 +02:00
Fynn Godau	8f6c00f8d8	Use default test in BandcampSearchExtractorTest	2020-06-04 12:12:20 +02:00
Fynn Godau	de776561b0	Bandcamp: fix loading uploader from streams in search	2020-06-04 12:02:18 +02:00
Fynn Godau	3940138fc5	Fix Bandcamp capitalization in one more spot	2020-06-03 22:41:27 +02:00
Fynn Godau	9fa9d920a9	Bandcamp: Implement new methods required due to interface changes	2020-06-03 21:35:11 +02:00
Fynn Godau	dd955c7f0c	Merge branch 'origin/dev' into dev	2020-06-03 21:13:40 +02:00
wb9688	33b1121fc7	Add tests for parseDurationString()	2020-05-30 17:20:54 +02:00
wb9688	e39147202c	Fix YouTube video duration parsing	2020-05-30 09:31:08 +02:00
wb9688	98055a3c3c	Revert "Add more decryption function regexes" ... This reverts commit 384d6acd5d.	2020-05-29 18:41:40 +02:00
Tobias Groza	bdb0f2ae6b	Merge pull request #332 from wb9688/learning-playlist ... Support YouTube's learning playlists	2020-05-29 10:11:08 +02:00
Tobias Groza	cf18cdb2f5	Merge pull request #329 from wb9688/decryption-regexes ... Add more decryption function regexes	2020-05-29 10:10:50 +02:00
wb9688	ab77961b0a	Extract trailer thumbnail in YT learning playlists	2020-05-29 09:55:24 +02:00
wb9688	4d683e7655	Support YouTube's learning playlists	2020-05-29 09:17:03 +02:00
Tobias Groza	0b4977bb0c	Merge pull request #333 from wb9688/playerresponse-dashmanifesturl ... Extract dashManifestUrl in playerResponse	2020-05-28 21:17:07 +02:00
wb9688	384d6acd5d	Add more decryption function regexes	2020-05-28 11:40:38 +02:00
wb9688	8f71bde776	Fix bug for YT videos longer than 1000h	2020-05-28 11:34:42 +02:00
wb9688	aa277e1b6a	Upgrade other dependencies	2020-05-28 11:34:42 +02:00
wb9688	036cc7e26e	Upgrade jsoup	2020-05-28 11:34:41 +02:00
Fynn Godau	89b0639faa	New Bandcamp tests for channel extractor	2020-05-25 20:22:23 +02:00
Fynn Godau	e08256ebc6	Remove "url cleanup" in Bandcamp link handlers	2020-05-25 19:17:27 +02:00
Fynn Godau	692b2d06f4	Bandcamp: read info item data in place and not in advance	2020-05-25 18:53:15 +02:00
Fynn Godau	fff21caa87	Fix Bandcamp no avatar test ... I own the bandcamp artist account called "NewPipeExtractorTest", so it shouldn't break again.	2020-05-25 18:53:10 +02:00
wb9688	7ef3e80493	Extract dashManifestUrl in playerResponse	2020-05-15 12:07:26 +02:00
wb9688	efafbbb88a	Merge pull request #331 from wb9688/playlist-subchannel ... Add sub-channel stuff to PlaylistInfo	2020-05-12 10:49:13 +02:00
wb9688	baf5dd3e8f	Merge pull request #311 from B0pol/fixSearchSuggestion ... Search: add isCorrectedSearch() and fix YoutubeSearchExtractor#getSea…	2020-05-12 09:29:15 +02:00