Commit Graph

2433 Commits

Author SHA1 Message Date
petlyh
e6aad117e7
[Bandcamp] Throw PaidContentException on paywalled albums 2023-04-03 19:27:09 +02:00
TobiGr
8495ad619e Bump version to 0.22.6 2023-04-02 22:35:02 +02:00
fynngodau
69705138e4
[Bandcamp] Fix extraction of related playlist items URL (#1047)
Small change in HTML structure
2023-04-02 22:24:29 +02:00
Björn Sigurbergsson
1b6fe5edd6
[YouTube] Fix ParsingException when comments are unavailable in a video (#1040)
Co-authored-by: bjs <bjs@elect-it.com>
Co-authored-by: Audric V. <74829229+AudricV@users.noreply.github.com>
Co-authored-by: Kavin <20838718+FireMasterK@users.noreply.github.com>
2023-03-30 19:58:06 +02:00
ThetaDev
8d1303e18f
Add track types to audio streams (#1041) 2023-03-28 00:02:20 +02:00
Tobi
44ae139d33
Merge pull request #1034 from AudricV/peertube-fix-commentsinfo-test
[PeerTube] Fix comments info test
2023-03-20 19:30:35 +01:00
Tobi
d5e9df5381
Merge pull request #1038 from TeamNewPipe/docs-jdk-11
Update JDK to 11 in JDoc workflow
2023-03-20 19:25:21 +01:00
Tobi
0486ad7a08
Update JDK to 11 in JDoc workflow 2023-03-06 18:45:11 +01:00
AudricV
80a6fc2c63
[PeerTube] Fix testGetCommentsFromCommentsInfo test of PeertubeCommentsExtractorTest.Default
The tested comment has been removed, so it couldn't be found in the comments
list.

This comment has been replaced by a new one from the current comments of the
video.

Also, in the parent class PeertubeCommentsExtractorTest, final has been used as
much as possible and for-each loops of lists have been replaced by their
forEach method or the Stream API, in order to simplify code.
2023-03-04 16:49:10 +01:00
petlyh
5a9b6ed2e3
[Bandcamp] Support loading additional comments (#1030) 2023-03-04 14:01:06 +01:00
Stypox
6bdd698c25
Merge pull request #1026 from AudricV/audio-streams-descriptive-and-locale-properties
Add descriptive and locale properties to audio streams
2023-03-01 11:15:46 +01:00
Stypox
19e4b216c9
Merge pull request #1032 from AudricV/yt_fix-comments-hashtags-links-extraction
[YouTube] Fix hashtags links extraction and escape HTML links
2023-03-01 10:47:37 +01:00
Stypox
b1298490c0
Merge pull request #1029 from AudricV/yt_fix-no-views-extraction-playlist-items
[YouTube] Fix partial non-extraction of "No views" string in stream items
2023-03-01 10:46:52 +01:00
petlyh
9dc1832733
[Bandcamp] Handle paywalled tracks 2023-02-28 17:51:30 +01:00
fynngodau
3fdb6ee476
Merge pull request #1031 from petlyh/bandcamp-fix-radio-comments
[Bandcamp] Show comments as disabled on radio streams
2023-02-27 20:50:12 +01:00
AudricV
bd79b921e8
[YouTube] Refactor the code to get stream items' view count
This refactoring avoids code duplication as much as possible.
2023-02-27 10:25:46 +01:00
AudricV
51f9b39953
[YouTube] Fix partial non-extraction of no views string in stream items
As the "No views" string is returned in the case there is no view on a video, a
number cannot be parsed in this case, so -1 was returned.

This string is now detected in all methods to get the view count of a stream.
2023-02-27 10:18:45 +01:00
AudricV
95b3f5e391
[MediaCCC] Test audio language property extraction 2023-02-26 19:06:18 +01:00
AudricV
30a0f8c510
[MediaCCC] Extract audio language property for single language audio tracks 2023-02-26 19:06:18 +01:00
AudricV
7f0269c4c7
[YouTube] Edit YoutubeStreamExtractorDefaultTest.AudioTrackLanguage to test audio locale property
The Hindi audio track language presence test has been changed from audio track
label to audio locale.
2023-02-26 19:06:18 +01:00
AudricV
034f82dae7
[YouTube] Test language and descriptive audio in YoutubeDashManifestCreatorsTest 2023-02-26 19:06:17 +01:00
AudricV
05e8cb39f7
[YouTube] Add language and descriptive audio properties to DASH manifests 2023-02-26 19:06:17 +01:00
AudricV
bf30d70152
[YouTube] Add descriptive audio test
This test uses video TjxC-evzxdk.

Also improve a bit YoutubeStreamExtractorDefaultTest.AudioTrackLanguage test.
2023-02-26 19:06:17 +01:00
AudricV
76b7c19c5d
[YouTube] Extract whether a track is a descriptive audio and audio locale when available
Also use audio track setters only for audio itags.
2023-02-26 19:06:17 +01:00
AudricV
3bb5eeef30
[YouTube] Add descriptive and locale audio support in ItagItem 2023-02-26 19:06:16 +01:00
AudricV
14bf3fb05b
Add ability to know the locale of an audio stream
Getting audio tracks locales by parsing their ID or their label, should not be
done by clients, but by the extractor.

This commit adds the ability to store the Locale of an AudioStream, which is
used to compare similar AudioStreams (in the equalStats method).
2023-02-26 19:06:16 +01:00
AudricV
f92426560c
Add descriptive audio properties
Also improve AudioStream's audio language documentation
2023-02-26 19:06:16 +01:00
AudricV
a63f289667
[YouTube] Update mocks of YoutubeCommentsExtractorTest.FormattingTest 2023-02-26 18:50:07 +01:00
AudricV
9483dcd9fa
[YouTube] Update mocks of YoutubeCommentsExtractorTest.RepliesTest 2023-02-26 18:43:36 +01:00
AudricV
1556adbb2d
[YouTube] Fix hashtags links extraction and escape text in attribute descriptions + HTML links
webCommandMetadata object is contained inside a commandMetadata one, so it is
not accessible from the root of the navigationEndpoint object.

The corresponding statement has been moved at the bottom of the specific
endpoints parsing, as the webCommandMetadata object is present almost
everywhere, otherwise URLs of some endpoints would have be changed, such as
uploader URLs (from channel IDs to handles).

As no ParsingException is now thrown by getUrlFromNavigationEndpoint, and so by
getTextFromObject, getUrlFromObject and getTextAtKey, the methods which were
catching ParsingExceptions thrown by these methods had to be updated.

URLs got in the HTML version of getTextFromObject are now escaped properly to
provide valid HTML to clients. This has been also done for attribute
descriptions, with the description text for this type of descriptions.

As YouTube descriptions are in HTML format (except for the fallback on the JSON
player response, which is plain text and only happens when there is no visual
metadata or a breaking change), all URLs returned are escaped, so tests which
are testing presence of URLs with escaped characters had to be updated (it was
only the case for YoutubeStreamExtractorDefaultTest.DescriptionTestUnboxing).
2023-02-26 18:43:36 +01:00
Stypox
99ab9777ad
Release 0.22.5 2023-02-26 16:33:25 +01:00
Stypox
39c2e1fda0
Merge pull request #1028 from TeamNewPipe/dependabot/gradle/org.jsoup-jsoup-1.15.4
Bump org.jsoup:jsoup from 1.15.3 to 1.15.4
2023-02-26 16:27:31 +01:00
petlyh
f7a7a236fb
[Bandcamp] Show comments as disabled on radio streams 2023-02-23 18:42:43 +01:00
dependabot[bot]
f5599ff08d
Bump org.jsoup:jsoup from 1.15.3 to 1.15.4
Bumps [org.jsoup:jsoup](https://github.com/jhy/jsoup) from 1.15.3 to 1.15.4.
- [Release notes](https://github.com/jhy/jsoup/releases)
- [Changelog](https://github.com/jhy/jsoup/blob/master/CHANGES)
- [Commits](https://github.com/jhy/jsoup/compare/jsoup-1.15.3...jsoup-1.15.4)

---
updated-dependencies:
- dependency-name: org.jsoup:jsoup
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-20 10:05:26 +00:00
Stypox
7e793c11ae
Empty commit to make JitPack happy
The build for e920ab3f5f succeeded, but for some reason it is considered as failed...
2023-02-08 22:35:26 +01:00
Stypox
e920ab3f5f
Merge pull request #1021 from TeamNewPipe/fix/yt-comment-text-hashtag
[YouTube] Fix getting the comment text if the comment contains a hashtag
2023-01-29 21:47:59 +01:00
TobiGr
3f7df9536e [YouTube] Fix getting the comment text if the comment contains a hashtag 2023-01-29 20:33:51 +01:00
Stypox
999fb7f812
Merge pull request #1024 from AudricV/snd_fix-tracks-like-count
[SoundCloud] Fix extraction of tracks like count
2023-01-29 10:52:54 +01:00
Stypox
3519d4c367
Merge pull request #1015 from AudricV/yt_fix-channel-id-rss-feeds
[YouTube] Fix channel ID extraction of YouTube channels RSS feeds
2023-01-29 10:41:38 +01:00
Stypox
9aca710e86
Merge pull request #1013 from Stypox/fix-music-mixes
[YouTube] Now music mixes can be treated as normal mixes
2023-01-29 09:48:51 +01:00
Stypox
76eeabac45
Merge pull request #1020 from TeamNewPipe/fix/yt-subscriber-count
[YouTube] Fix NPE in search when getting channel items without subscriber count
2023-01-29 09:44:22 +01:00
AudricV
676622f6df
[SoundCloud] Fix expectedLikeCountAtLeast tests of SoundcloudStreamExtractorTest test classes
As like count is now returned by the extractor, we need to assert a positive
minimum like count, which is close to the actual value, in order to avoid test
failures due to lower like counts than the ones excepted.
2023-01-29 01:08:02 +01:00
AudricV
2a24d407d5
[SoundCloud] Fix extraction of tracks like count
SoundCloud is using likes_count to return the like count of a track, like it
was the case before they switched to favoritings_count.
2023-01-29 01:00:49 +01:00
Tobi
0e4e6a9bac
Merge pull request #1022 from AudricV/yt_support-live-urls
[YouTube] Support live URLs
2023-01-28 21:52:26 +01:00
AudricV
ba24976e41
[YouTube] Add live URLs test and do minor improvements to YoutubeStreamLinkHandlerFactoryTest
- Remove unused imports;
- Replace wildcard imports by single class imports;
- Suppress "HTTP links are not secured" warnings from IDEA IDEs;
- Replace removed video jZViOEv90dI by an existing video, 9Dpqou5cI08 (the
corresponding test has been of course renamed).
2023-01-28 19:36:21 +01:00
AudricV
57f850bc2d
[YouTube] Support live URLs and do minor improvements to YoutubeStreamLinkHandlerFactory
- Move license header at the top;
- Use an unmodifiable set for the subpaths instead of a modifiable list;
- Add missing Nonnull and Nullable annotations;
- Improve exception messages.
2023-01-28 19:36:20 +01:00
AudricV
1f4ed9dce9
[YouTube] Fix channel ID extraction of YouTube channel RSS feeds
The yt:channelId element doesn't provide the channel ID anymore and is empty,
like the id element, so we need now to extract it from the channel URL provided
in two elements: author -> uri and feed -> link.

Also avoid a NullPointerException in getUrl and getName methods.
2023-01-28 11:53:33 +01:00
Tobi
c589a2c1a2
Merge pull request #1014 from TeamNewPipe/fix/yt-comments
[YouTube] Fix getting next comments pages
2023-01-27 11:14:55 +01:00
Tobi
c2b36fd405
Merge pull request #1016 from lonewolf2208/bug-fix
[YouTube] Remove topStandaloneBadge check for view count of stream items
2023-01-25 08:30:36 +01:00
TobiGr
72573932cf [YouTube] Fix NPE in search when getting channel items without subscriber count 2023-01-24 23:03:45 +01:00