Commit Graph

1955 Commits

Author SHA1 Message Date
AudricV
56ab35423e
[YouTube] Fix potential NullPointerException in YoutubeSearchExtractor.getSearchSuggestion 2023-12-08 21:46:48 +01:00
AudricV
9dc1eab28c
[YouTube] Fix expected upload dates of StreamExtractor tests
YouTube is rolling out or A/B testing a new date format returned inside player
responses, which are precise to the second instead of the day.

This commit makes the StreamExtractor tests use these more precise dates.
2023-12-08 21:46:48 +01:00
AudricV
ad3d187ac7
[YouTube] Fix testRelatedItems method of YoutubeStreamExtractorRelatedMixTest
This commit fixes the testRelatedItems test method by:
- accepting consent in the test class, in order to extract mixes in
recommendations;
- removing assertion of a music mix inside the recommendations, as YouTube
doesn't seem to return such mixes anymore, at least for the video used in the
test class.
2023-12-08 21:46:47 +01:00
AudricV
e111814401
[YouTube] Fix YoutubeStreamExtractorDefaultTest.RatingsDisabledTest
Replace the video used in this test class with another one publicly available
and update the corresponding expected test values.

The test class's mocks will be updated in a different commit.
2023-12-08 21:46:47 +01:00
AudricV
fc45941ead
[YouTube] Fix YoutubeChannelExtractorTest tests
- Change CarouselHeader test channel to Sports system one, as the Coachella one
doesn't return this channel header anymore;
- Fix InteractiveTabbedHeader test by checking whether the test's channel
description is not empty instead of containing some words, as it is changing
frequently.
2023-12-08 21:46:47 +01:00
AudricV
0bcb241c38
[YouTube] Fix expected data in YoutubeStreamExtractorRelatedMixTest
Video's title and tags have been changed by its uploader, so they have to be
updated.

Also make some package-private constants private, as they are not used outside
of the class, and remove unneeded test overrides.
2023-12-08 21:46:47 +01:00
AudricV
6ba8251be1
[YouTube] Bypass crisis resources blocking search results
These crisis resources are preventing search results to be returned. See
https://support.google.com/youtube/answer/10726080?hl=en for more info on them.

This commit changes search parameters to include the property allowing to show
search results.
2023-12-08 21:46:47 +01:00
AudricV
7dea2d0d27
[YouTube] Remove Channels channel tab support
This tab has been removed by YouTube.
2023-12-08 21:46:47 +01:00
AudricV
3782d9a02a
[YouTube] Support new A/B tested like data and avoid like count conversion from integer to long
Also make minor improvements to current like data extraction and remove
previous like count data support, as it is not returned anymore.
2023-12-08 21:46:46 +01:00
AudricV
b71ce1123f
[YouTube] Extract only search results corresponding to a search type
YouTube returns sometimes videos inside channel search results. As we only want
results corresponding to the type we requested, this commits makes
YoutubeSearchExtractor ignoring non-requested search results we get, using the
extractor LinkHandler's first content filter value.

Also remove an unneeded exception throwing declaration in
YoutubeSearchExtractor.
2023-12-08 21:46:46 +01:00
AudricV
ff8ed7247f
[YouTube] Switch to new consent cookie
Also move the documentation of the consent in its setter method in order to be
accessible publicly and improve it.
2023-12-08 21:46:46 +01:00
AudricV
ec838d7421
[YouTube] Add missing prettyPrint query parameter to some test InnerTube requests
This query parameter for which its value is set to false was not added to two
requests made in test classes of YoutubeMixPlaylistExtractorTest.

Also remove an unneeded ParsingException exception throwing declaration in a
test method.
2023-12-08 21:46:46 +01:00
AudricV
2c941794c0
[YouTube] Add utcOffsetMinutes to all InnerTube payloads
This should make returned dates consistent between timezones and countries on
which the extractor is ran.

It was previously only set on YouTube Music search continuations.
2023-12-08 21:46:46 +01:00
AudricV
d97c9e0db1
[YouTube] Improve payloads and URLs of InnerTube requests
For every InnerTube request:
- Always add a `request` object with the following properties:
  - "internalExperimentFlags" set to an empty array;
  - "useSsl" set to "true";
  - "lockedSafetyMode" set to "false".
- Use proper TODO comment to provide a way to enable restricted mode on every
request and add it on requests on which it wasn't present.

For YouTube Music:
- Remove alt query parameter, as it is not used anymore by the website;
- Add prettyPrint query parameter with false value on YouTube Music search
continuations.
2023-12-08 21:46:45 +01:00
AudricV
8a9ebcc373
[YouTube] Update InnerTube clients' version and devices' OS version and model 2023-12-08 21:46:45 +01:00
Tobi
eac850ca10
Merge pull request #1114 from FineFindus/feat/comment-author-is-owner
[YouTube] Add channel owner to comments
2023-10-25 09:51:12 +02:00
dependabot[bot]
9d7bcba050
Bump org.jsoup:jsoup from 1.16.1 to 1.16.2
Bumps [org.jsoup:jsoup](https://github.com/jhy/jsoup) from 1.16.1 to 1.16.2.
- [Release notes](https://github.com/jhy/jsoup/releases)
- [Changelog](https://github.com/jhy/jsoup/blob/master/CHANGES)
- [Commits](https://github.com/jhy/jsoup/compare/jsoup-1.16.1...jsoup-1.16.2)

---
updated-dependencies:
- dependency-name: org.jsoup:jsoup
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-20 09:13:21 +00:00
FineFindus
34b05a0dda
feat(youtube/comments): support creator replies 2023-10-09 16:33:43 +02:00
TobiGr
0821f09114
Add missing mocks 2023-10-09 16:33:43 +02:00
FineFindus
c1784a4bdb
[YouTube] Add channel owner to comments 2023-10-09 16:33:43 +02:00
TobiGr
f9846352ea Fix wrong @Nullable annotation 2023-10-09 16:02:57 +02:00
Tobi
d6f5cba6e2
Merge pull request #1111 from FineFindus/feat/creator-reply
Add `hasCreatorReply()` to CommentsInfoItem
2023-10-09 12:45:56 +02:00
TobiGr
9d63c75623 Add missing mocks 2023-10-09 11:24:39 +02:00
TobiGr
d49f8411d7 [PeerTube] Implement CommentsInfoItemExtractor.hasCreatorReply() 2023-10-09 02:47:12 +02:00
AudricV
c98695fcea
[SoundCloud] Fix extraction of non-JPG images
Default image qualities were removed in image URLs with the jpg extension,
causing the addition of the image suffix to full non-JPG images URLs and so to
invalid image URLs.

Only the image quality name with its leading "-" character and the "."
character after the name is now removed and replaced by a string format
replaced itself with the image quality name for each quality.

As the image suffixes do not contain the image extension, the name of image
qualities lists has been adapted with these changes and some related comments
have been also improved.
2023-10-01 20:33:25 +02:00
AudricV
ac00459c1a
Change requirement of image extensions in ImageSuffix class' Javadoc to a possibility
Some services may provide different image formats using the same suffix,
without we know what format the service provide. Enforcing an image extension
could so lead to provide invalid image URLs, like for SoundCloud PNG images
currently.

With this documentation change, it is now clear that users of this class decide
of whether they want to include image extensions in the suffix. The previous
behavior described in the Javadoc was not enforced.
2023-09-30 21:11:09 +02:00
FineFindus
dd7b2d9798
feat(youtube/comments): support creator replies 2023-09-25 10:40:45 +02:00
Youssif Shaaban Alsager
917554acc4
[YouTube] Add support for ultralow audio formats (#1063) 2023-09-24 19:04:34 +02:00
Christian
fc67d49f59 Update copyright notices
Update copyright notices to comply to GPLv3 and change NewPipe to NewPipe Extractor on some notices that were not updated.
2023-09-22 19:10:15 -03:00
AudricV
6ed22099a2
[YouTube] Update stream mocks 2023-09-21 21:59:34 +02:00
AudricV
714b141ecb
[YouTube] Catch any exception when extracting something from JavaScript's base player 2023-09-21 21:59:33 +02:00
AudricV
588c6a8422
[YouTube] Quote signature deobfuscation function name and add semicolon only where needed 2023-09-21 21:59:33 +02:00
AudricV
1fa85ec6ca
[YouTube] Add tests for signature timestamp extraction and signature deobfuscation function extraction and execution 2023-09-21 21:59:33 +02:00
AudricV
a04bc320de
[YouTube] Convert signature timestamp to integer
The signature timestamp is used as a number by HTML5 clients, so it should be
used in the same way by the extractor too instead of being a string.

As the timestamp doesn't seem to exceed 5 digits, an integer is used to store
its value.
2023-09-21 21:59:32 +02:00
AudricV
7de3753a81
[YouTube] Refactor JavaScript player management API
This commit is introducing breaking changes.

For clients, everything is managed in a new class called
YoutubeJavaScriptPlayerManager:
- caching JavaScript base player code and its extracted code (functions and
variables);
- getting player signature timestamp;
- getting deobfuscated signatures of streaming URLs;
- getting streaming URLs with a throttling parameter deobfuscated, if
applicable.

The class delegates the extraction parts to external package-private classes:
- YoutubeJavaScriptExtractor, to extract and download YouTube's JavaScript base
player code: it always already present before and has been edited to mainly
remove the previous caching system and made it package-private;
- YoutubeSignatureUtils, for player signature timestamp and signature
deobfuscation function of streaming URLs, added in a recent commit;
- YoutubeThrottlingParameterUtils, which was originally
YoutubeThrottlingDecrypter, for throttling parameter of streaming URLs
deobfuscation function and checking whether this parameter is in a streaming
URL.

YoutubeJavaScriptPlayerManager caches and then runs the extracted code if it
has been executed successfully. The cache system of throttling parameters
deobfuscated values has been kept, its size can be get using the
getThrottlingParametersCacheSize method and can be cleared independently using
the clearThrottlingParametersCache method.

If an exception occurs during the extraction or the parsing of a function
property which is not related to JavaScript base player code fetching, it is
stored until caches are cleared, making subsequent failing extraction calls of
the requested function or property faster and consuming less resources, as the
result should be the same until the base player code changes.

All caches can be reset using the clearAllCaches method of
YoutubeJavaScriptPlayerManager.

Classes using JavaScript base player code and utilities directly (in the code
and its tests) have been also updated in this commit.
2023-09-21 21:59:32 +02:00
AudricV
6884d191cd
[YouTube] Add utility class around signatures and fix signature deobfuscation function extraction
The goal of this class is to decouple the extraction of signature timestamp and
signature deobfuscation function from YoutubeStreamExtractor.

The extraction of the signature deobfuscation function has been also adapted to
support the latest YouTube player versions.

This new class, YoutubeSignatureUtils, doens't store anything temporary such as
a copy of the player code, which has to be passed where required. It is not
public, as it will be used by a JavaScript player manager class in the future,
in order to handle in a better way fetching, caching and resetting cache of the
player code.
2023-09-21 21:59:26 +02:00
TobiGr
17790328cd Improve doc 2023-09-18 16:44:51 +02:00
Isira Seneviratne
4bc8ae7812 Use Locale.forLanguageTag() in tests 2023-09-18 08:59:13 +05:30
TobiGr
cf49f4a31c [Badcamp] Fix StreamExtractorTest
The song was renamed and the URL changed
2023-09-17 23:58:07 +02:00
AudricV
e8bfd20170
[MediaCCC] Apply changes in extractor tests
Also remove some public test methods modifiers.
2023-08-12 22:56:33 +02:00
AudricV
0292c4f3e8
[Bandcamp] Apply changes in extractor tests
Also remove some public test methods modifiers, add missing Test annotations on
old Junit 4 tests (and update them if needed), and use final in some places
where it was possible.

BandcampChannelExtractorTest.testLength has been removed as the test is always
true.
2023-08-12 22:56:32 +02:00
AudricV
2578f22054
[Bandcamp] Add utility test method to test images
This method, testImages(Collection<Image>), will use first the default image
collection test in DefaultTests and then will check that each image URL
contains f4.bcbits.com/img and ends with .jpg or .png.

To do so, a new non-instantiable final class has been added: BandcampTestUtils.
2023-08-12 22:56:32 +02:00
AudricV
ba5315c72d
[PeerTube] Apply changes in extractor tests
Also remove some public test methods modifiers, add missing Test annotations on
old Junit 4 tests (and update them if needed), and improve some code.
2023-08-12 22:56:32 +02:00
AudricV
1d72bac53d
[SoundCloud] Apply changes in extractor tests 2023-08-12 22:56:32 +02:00
AudricV
93a210394d
[YouTube] Apply changes in extractor tests
Also remove some public test methods modifiers, add missing Test annotations on
old Junit 4 tests (and update them if needed), and use final in some places
where it was possible.
2023-08-12 22:56:31 +02:00
AudricV
2c436d428c
[YouTube] Add utility test method to test images in YoutubeTestsUtils
This method, testImages(Collection<Image>), will use first the default image
collection test in DefaultTests and then will check that each image URL
contains the string yt.

The JavaDoc of the class has been also updated to reflect the changes made in
it (it is now more general).
2023-08-12 22:56:31 +02:00
AudricV
d381f3b70b
Update avatar, banners and thumbnail methods' name and apply changes in DefaultStreamExtractorTest 2023-08-12 22:56:31 +02:00
AudricV
434e885708
Add utility methods in ExtractorAsserts to check whether a collection is empty and to test image collections
Two new methods have been added in ExtractorAsserts to check if a collection is
empty:

- assertNotEmpty(String, Collection<?>), checking:
  - the non nullity of the collection;
  - its non emptiness (if that's not case, an exception will be thrown using
    the provided message).

- assertNotEmpty(Collection<?>), calling assertNotEmpty(String, Collection<?>)
  with null as the value of the string argument.

A new one has been added to this assertion class to check the contrary:
assertEmpty(Collection<?>), checking emptiness of the collection only if it is
not null.

Three new methods have been added in ExtractorAsserts as utility test methods
for image collections:

- assertContainsImageUrlInImageCollection(String, Collection<Image>), checking
that:
  - the provided URL and image collection are not null;
  - the image collection contains at least one image which has the provided
    string value as its URL (which is a string) property.

- assertContainsOnlyEquivalentImages(Collection<Image>, Collection<Image>),
  checking that:
  - both collections are not null;
  - they have the same size;
  - each image of the first collection has its equivalent in the second one.
    This means that the properties of an image in the first collection must be
    equal in an image of the second one.

- assertNotOnlyContainsEquivalentImages(Collection<Image>, Collection<Image>),
  checking that:
  - both collections are not null;
  - one of the following conditions is met:
    - they have different sizes;
    - an image of the first collection has not its equivalent in the second one.
      This means that the properties of an image in the first collection must
      be not equal in an image of the second one.

These methods will be used by services extractors tests (and default ones) to
test image collections.
2023-08-12 22:56:31 +02:00
AudricV
5158472852
Apply changes in DefaultTests and add utility method to test image lists
This new method, defaultTestImageList(List<Image), will check that the image
list is not null.

For each image, it will test that its URL is secure and its height and width
are more than or equal to their relevant unknown constants in the Image class
(HEIGHT_UNKNOWN and WIDTH_UNKNOWN).
2023-08-12 22:56:31 +02:00
AudricV
70fb3aa38e
Update BaseExtractorTests image methods' name
Also suppress unused warnings in BaseStreamExtractorTest, like it is done on
other BaseExtractorTests interfaces.
2023-08-12 22:56:30 +02:00
AudricV
e16d521b7b
[MediaCCC] Apply changes in Extractors
Also remove usage of the conference logo as the banner of a conference, as it
is a logo and not a banner.
2023-08-12 22:56:30 +02:00
AudricV
306068a63b
[MediaCCC] Apply changes in InfoItemExtractors 2023-08-12 22:56:30 +02:00
AudricV
2f40861428
[MediaCCC] Add utility methods to get image lists from conference logos and streams
These three new methods, added in MediaCCCParsingHelper,
getImageListFromImageUrl(String), getThumbnailsFromStreamItem(JsonObject) and
getThumbnailsFromLiveStreamItem(JsonObject) (the last two are based on a common
method, getThumbnailsFromObject(JsonObject, String, String)), return an empty
list if the case no image URL could be extracted.

Images returned have their height and width unknown and a resolution level
depending on the image key of the JSON API response.
2023-08-12 22:56:30 +02:00
AudricV
71cda03c4c
[Bandcamp] Apply changes in Extractors 2023-08-12 22:56:29 +02:00
AudricV
7e01eaac33
[Bandcamp] Apply changes in InfoItemExtractors 2023-08-12 22:56:29 +02:00
AudricV
4b80d737a4
[Bandcamp] Add utility methods to get multiple images
Bandcamp images work with image IDs, which provide different resolutions.

Images on Bandcamp are not always squares, and some IDs respect aspect ratios
where some others not.

The extractor will only use the ones which preserve aspect ratio and will not
provide original images, for performance and size purposes.

Because of this aspect ratio preservation constraint, only one dimension will
be known at a time.

The image IDs with their respective dimension used are:

- 10: 1200w;
- 101: 90h;
- 170: 422h;
- 171: 646h;
- 20: 1024w;
- 200: 420h;
- 201: 280h;
- 202: 140h;
- 204: 360h;
- 205: 240h;
- 206: 180h;
- 207: 120h;
- 43: 100h;
- 44: 200h.

(Where w represents the width of the image and h the height of the image)

Note that these dimensions are theoretical because if the image size is less
than the dimensions of the image ID, it will be not upscaled but kept to its
original size.

All these resolutions are stored in a private static list of ThumbnailSuffixes
in BandcampExtractorHelper, in which the methods to get mutliple images have
been added:

- getImagesFromImageUrl(String): public method to get images from an image URL;
- getImagesFromImageId(long, boolean): public method to get images from an
  image ID;
- getImagesFromImageBaseUrl(String): private utility method to get images from
  the static list of ThumbnailSuffixes from a given image base URL, containing
  the path to the image, a "a" letter if it comes from an album, its ID and an
  underscore.

Some existing methods have been also edited:

- the documentation of getImageUrl(long, boolean) has been changed to reflect
  the Bandcamp images findings;
- getThumbnailUrlFromSearchResult has been renamed to
  getImagesFromSearchResult, and a documentation has been added to this method.

The method replaceHttpWithHttps of the Utils class has been also used in
BandcampExtractorHelper instead of doing manually what the method does.
2023-08-12 22:56:29 +02:00
AudricV
4e6fb368bc
[PeerTube] Apply changes in Extractors and remove usages of default avatar picture
The default avatar picture was used when no profile picture was found, but it
was removed and split in multiple images.

Thumbnails' size is not known, as this data is not provided by the API.
2023-08-12 22:56:29 +02:00
AudricV
0a6011a50e
[PeerTube] Apply changes in InfoItemExtractors
Also lower the visibility of attributes of channels and playlists InfoItems to
private.
2023-08-12 22:56:29 +02:00
AudricV
6f8331524b
[PeerTube] Add utility method to get thumbnails of playlists and videos
This method, getThumbnailsFromPlaylistOrVideoItem, has been added in
PeertubeParsingHelper and returns the two image variants for playlists and
videos.
2023-08-12 22:56:28 +02:00
AudricV
81c0d80a54
[PeerTube] Add utility methods to get avatars and banners of accounts and channels
Four new static methods have been added in PeertubeParsingHelper to do so:
- two public methods to get the corresponding image type:
  getAvatarsFromOwnerAccountOrVideoChannelObject(String, JsonObject) and
  getBannersFromAccountOrVideoChannelObject(String, JsonObject);
- two private methods as helper methods: getImagesFromAvatarsOrBanners(String,
  JsonObject, String, String) and getImagesFromAvatarOrBannerArray(String,
  JsonArray).
2023-08-12 22:56:28 +02:00
AudricV
31da5beb51
[SoundCloud] Apply changes in Extractors 2023-08-12 22:56:28 +02:00
AudricV
a3a74cd566
[SoundCloud] Apply changes in InfoItemExtractors and return track user avatars as uploader avatars in SoundcloudStreamInfoItemExtractor 2023-08-12 22:56:28 +02:00
AudricV
7f818217d2
[SoundCloud] Add utility methods to get images from track JSON objects and image URLs
These new public and static methods, added in SoundcloudParsingHelper,
getAllImagesFromArtworkOrAvatarUrl(String) and
getAllImagesFromVisualUrl(String) (which call a common private method,
getAllImagesFromImageUrlReturned(String, List<ImageSuffix>, List<Image>)),
return an unmodifiable list of JPEG images containing almost every image
resolution provided by SoundCloud except the original size and the tiny
resolution (for artworks and avatars, as the image size is 20x20 for artworks
and 18x18 for avatars, so very close to or equal to the t20x20 resolution):

- for artworks and avatars:
  - mini: 16x16;
  - t20x20: 20x20;
  - small: 32x32;
  - badge: 47x47;
  - t50x50: 50x50;
  - t60x60: 60x60;
  - t67x67: 67x67;
  - large: 100x100;
  - t120x120: 120x120;
  - t200x200: 200x200;
  - t240x240: 240x240;
  - t250x250: 250x250;
  - t300x300: 300x300;
  - t500x500: 500x500.

- for visuals/user banners:
  - t1240x260: 1240x260;
  - t2480x520: 2480x520.

Duplicated code in two methods of SoundcloudParsingHelper
(getUsersFromApi(ChannelInfoItemsCollector, String) and
getStreamsFromApi(StreamInfoItemsCollector, String, boolean)) has been merged
into one common private method, getNextPageUrlFromResponseObject(JsonObject).
2023-08-12 22:56:28 +02:00
AudricV
266cd1f76b
[YouTube] Apply changes in YoutubeMusicSearchExtractor and split its InfoItemExtractors into separate classes
Splitting YoutubeMusicSearchExtractor's InfoItemExtractors into separate
classes (YoutubeMusicSongOrVideoInfoItemExtractor,
YoutubeMusicAlbumOrPlaylistInfoItemExtractor and
YoutubeMusicArtistInfoItemExtractor) allows to simplify
YoutubeMusicSearchExtractor,improves reading and applying changes to InfoItems
(no more losing at least quarter of a line due to indentations).

These InfoItems, in which the image changes have been applied, don't extend the
YouTube ones anymore, as most methods were overridden and the few ones that are
not don't apply in YouTube Music items responses, so it was useless to extend
them.

The code of YoutubeMusicSearchExtractor have been also improved a bit.
2023-08-12 22:56:27 +02:00
AudricV
c1981ed54f
[YouTube] Apply changes in Extractors except YoutubeMusicSearchExtractor
Also improve a bit some code related to the changes.
2023-08-12 22:56:27 +02:00
AudricV
4cc99f9ce1
[YouTube] Apply changes in InfoItemExtractors except YouTube Music ones 2023-08-12 22:56:27 +02:00
AudricV
adfad086ac
[YouTube] Add utility methods to get images from InfoItems and thumbnails arrays
Unmodifiable lists of Images are returned, parsed from a given YouTube
"thumbnails" JSON array.

These methods will be used in all YouTube extractors and InfoItems, as the
structures between content types (videos, channels, playlists, ...) are common.
2023-08-12 22:56:27 +02:00
AudricV
d56b880cae
Replace avatar and thumbnail URLs attributes and methods to List<Image> in Infos 2023-08-12 22:56:26 +02:00
AudricV
9d8098576e
Replace avatar and thumbnail URLs attributes and methods to List<Image> in Extractors 2023-08-12 22:56:26 +02:00
AudricV
0f4a5a8184
Replace avatar and thumbnail URLs attributes and methods to List<Image> in InfoItemsCollectors 2023-08-12 22:56:26 +02:00
AudricV
ca1d4a6fa4
Replace avatar and thumbnail URLs attributes and methods to List<Image> in InfoItemExtractors 2023-08-12 22:56:26 +02:00
AudricV
2f3ee8a3f2
Replace avatar and thumbnail URLs attributes and methods to List<Image> in InfoItems 2023-08-12 22:56:25 +02:00
AudricV
78ce65769f
Add an ImageSuffix class to the extractor
The goal of this utility class is to simply store suffixes which need to be
appended to image URLs, in order to get images at the suffix resolution.

This class contains four properties: the suffix (as a string), the height,
the width (as integers) and the estimated resolution level of the image
corresponding to the one represented by the suffix.
2023-08-12 22:56:25 +02:00
AudricV
d85454186a
Add an Image class to the extractor
Objects of this serializable class contains four properties: a URL (as a
string), a width, a height (represented as integers) and an estimated
resolution level, which can be constructed from a given height.

Possible resolution levels are:
- UNKNOWN: for unknown heights or heights <= 0;
- LOW: for heights > 0 & < 175;
- MEDIUM: for heights >= 175 & < 720;
- HIGH: for heights >= 720.

Getters of these properties are available and the constructor needs these four
properties.
2023-08-12 22:56:25 +02:00
Stypox
7294675aea
Merge pull request #1093 from AudricV/yt_support-shorts-ui-playlists
[YouTube] Support Shorts UI in playlists
2023-08-12 11:11:36 +02:00
Stypox
44b664af15
[YouTube] Simplify Optional chains in channel 2023-08-12 11:02:51 +02:00
AudricV
2f7bfd3e7f
[YouTube] Add mocks of interactiveTabbedHeaderRenderer channel header test 2023-08-08 19:12:29 +02:00
AudricV
b147904571
[YouTube] Add test for interactiveTabbedHeaderRenderer channel header
This test uses the Minecraft game topic channel.
2023-08-08 19:12:28 +02:00
AudricV
1852031a0b
[YouTube] Support pageHeaderRenderer and interactiveTabbedHeaderRenderer channel headers
The addition of this support required to turn the isCarouselHeader boolean into
an enum containing all supported channel headers named HeaderType.

Also assert that the page has been fetched where needed to avoid
NullPointerExceptions when the channel page has been not fetched and remove the
getChannelHeaderJson method in YoutubeChannelExtractor, method for which its
code has been moved to its sole usage after the new headers support changes.
2023-08-08 19:12:27 +02:00
AudricV
698c710685
Do not require knowledge of uploader in default StreamInfoItems tests
This change is required as some services can return no uploader info, such as
YouTube for playlists with a Shorts UI.
2023-08-07 19:43:15 +02:00
AudricV
8237052ef5
Fix wrong assertion in assertNotEmpty
The non-null assertion was made on the exception message instead of the string
to check, causing a NullPointerException if the string to check was null.
2023-08-07 19:43:09 +02:00
AudricV
162c261577
[YouTube] Add mocks of the playlist with Shorts UI test 2023-08-07 19:07:53 +02:00
AudricV
e2f4ee47b9
[YouTube] Add a playlist with Shorts UI test
The system Shorts videos uploads playlist of the YouTube official channel has
been chosen for this test.
2023-08-07 19:06:09 +02:00
AudricV
e6f371fb94
[YouTube] Support Shorts UI in playlists
Also remove an outdated A/B test comment.
2023-08-07 19:01:08 +02:00
Stypox
6d2227111f
[YouTube] Assert that videos tab is ready after channel fetching 2023-08-06 21:14:57 +02:00
Stypox
ee625c325c
Inherit from DefaultListExtractorTest in channel tab tests 2023-08-06 21:14:56 +02:00
Stypox
276c293889
Rename assertTabsContain 2023-08-06 21:14:56 +02:00
Stypox
9d3761a371
[YouTube] Directly use playlist collector in channel tabs wrapper
Note that this introduces a "Raw use of parameterized class 'InfoItemsPage'" warning, but it can be ignored since the type missing would be <InfoItem>, and StreamInfoItem extends InfoItem
2023-08-06 21:13:25 +02:00
Stypox
e34b4f1978
[YouTube] Avoid using Consumer 2023-08-06 13:02:31 +02:00
Stypox
ef67c7cd74
[YouTube] Simplify usage of channel header json 2023-08-06 13:02:31 +02:00
Stypox
a104cf3227
[YouTube] Fix docs in channel helper 2023-08-06 13:02:31 +02:00
Stypox
468bcc045d
[YouTube] Update mocks after #1087 2023-08-06 12:33:04 +02:00
AudricV
e7d64099a7
[YouTube] Update channel mocks and add channel tabs mocks 2023-08-06 12:15:06 +02:00
AudricV
684101c47d
[YouTube] Implement age-restricted channels support, link handlers and channels tabs and tags changes on tests
Co-authored-by: ThetaDev <t.testboy@gmail.com>
2023-08-06 12:15:06 +02:00
AudricV
eaf2600ce0
[SoundCloud] Implement link handlers and channels tabs and tags changes on tests
Co-authored-by: ThetaDev <t.testboy@gmail.com>
2023-08-06 12:15:06 +02:00
AudricV
0ee2072de5
[PeerTube] Implement link handlers and channels tabs and tags changes on tests
Co-authored-by: ThetaDev <t.testboy@gmail.com>
2023-08-06 12:15:06 +02:00
AudricV
d3801dd0e9
[MediaCCC] Implement link handlers and channels tabs and tags changes on tests
Co-authored-by: ThetaDev <t.testboy@gmail.com>
2023-08-06 12:15:06 +02:00
AudricV
8baec04611
[Bandcamp] Implement link handlers and channels tabs and tags changes on tests
Tests in BandcampChannelExtractorTest and BandcampChannelLinkHandlerFactoryTest
have been also fixed.

Co-authored-by: ThetaDev <t.testboy@gmail.com>
2023-08-06 12:15:06 +02:00
AudricV
e0ba29cd19
Add utility method to assert that given channel tabs are in the ones returned by a channel extractor
Only the first content filter of the ListLinkHandler instances provided is
used when collecting all channel tabs of the ListLinkHandler list, as channel
tabs implementations only use one content filter per ListLinkHandler instance.

Co-authored-by: ThetaDev <t.testboy@gmail.com>
2023-08-06 12:15:06 +02:00
AudricV
18846baba7
Add tabs and tags methods in tests interfaces and annotate all methods with the Test JUnit annotation
These changes should help to detect tests as tests, when running a subset of
tests or all tests.
They should be also implemented in these interfaces' implementations (new and
existing ones).

Co-authored-by: ThetaDev <t.testboy@gmail.com>
2023-08-06 12:15:06 +02:00