2746 Commits

Author SHA1 Message Date
AudricV
6533a333fa
[YouTube] Add ability to get a visitorData from InnerTube 2025-02-05 09:56:52 +01:00
AudricV
9e45c8001a
[YouTube] Do not send a visitorData for every InnerTube request
As YouTube is disabling ability to use a random visitor ID in a
visitorData on player requests and BotGuard challenges, it
shouldn't matter if we use a random one or not for other
request types.
2025-02-05 09:56:52 +01:00
AudricV
94541d2d94
[YouTube] Add utility data class to store client and device info
This new class, InnertubeClientRequestInfo, composed of two mutable
subclasses, ClientInfo and DeviceInfo, allows to store client and
device info in a better way, without requiring to pass more than 10
method parameters like in YoutubeStreamHelper currently.

Mutability has been added in order to allow changing some fields
easily, especially visitorData.
2025-02-05 09:56:52 +01:00
AudricV
9333d7fcdc
[YouTube] Update DASH manifest creation clients' handling
- Use POST requests with the same body as official clients do;
- Update methods checking the client streaming URLs come from:
  - Replace TVHTML5_SIMPLY_EMBEDDED_PLAYER by TVHTML5;
  - Add WEB_EMBEDDED_PLAYER.
2025-02-05 09:56:52 +01:00
AudricV
38e2b67cb7
[YouTube] Remove unused methods and constants in YoutubeParsingHelper 2025-02-05 09:56:52 +01:00
AudricV
3878696b2c
[YouTube] Add support for poTokens, refactor player clients' fetching
Also improve detection of age-restricted statuses in playability error
messages returned by the service and provide version 7 of DASH
manifests.
2025-02-05 09:56:52 +01:00
AudricV
9d2b840cfb
[YouTube] Add utility class to handle player requests fetching
This internal class, YoutubeStreamHelper, has to be public in order to
be used by subpackages of the service's root one.

It supports poTokens, HTML5 signatureTimestamp property, embed context
and multiple InnerTube clients. It is meant to replace the
corresponding methods in YoutubeParsingHelper, in order to reduce the
class' size, code duplicates and improve its readability.

It adds a new way to get age-restricted videos' streams, the only ones
which are playable in YouTube embeds, which is not very common.
2025-02-05 09:56:52 +01:00
AudricV
1df0267440
[YouTube] Make client and origin HTTP headers methods package-private
This allows their internal usage in an upcoming new class to be used on
other InnerTube clients than the WEB one.
2025-02-05 09:56:52 +01:00
AudricV
3691fc22c6
[YouTube] Add an interface and a class to fetch and provide poTokens 2025-02-05 09:56:52 +01:00
AudricV
fd82ec585b
[YouTube] Move to their own file and update clients' constants
Also update client version and device info and add TVHTML5 client and
WEB_EMBEDDED_PLAYER constants, these will be used in the future.

TVHTML5_SIMPLY_EMBED_CLIENT_VERSION has been kept in its place as
the corresponding client does not work anonymously anymore, so it will
be removed soon.
2025-02-05 09:56:52 +01:00
Stypox
186e32c9db
Merge pull request #1269 from AudricV/snd_no_drm_streams
[Soundcloud] Remove DRM-protected and downloadable formats extraction
2025-02-03 20:35:20 +01:00
AudricV
a336a8ca97
[Soundcloud] Update SoundcloudStreamExtractorTest with latest changes 2025-01-28 21:52:29 +01:00
AudricV
7607688cb0
[Soundcloud] Remove DRM-protected and downloadable formats extraction
DRM-protected streams have been added to some tracks, mostly from
major music companies. We do not support DRM streams in the extractor,
so they shouldn't be extracted and so waste time, energy and resources.

Extracting downloadable format requires login for a pretty long time,
so there is no point again to do requests to extract this stream to get
an unauthorized response.

Also send the track_authorization property returned in the track info,
like the website does and allow duplicate MP3 formats in progressive and
HLS delivery methods to be returned.
2025-01-28 21:52:28 +01:00
litetex
3a33cefbc0
Merge pull request #1246 from Isira-Seneviratne/BufferedReader-lines
Use BufferedReader#lines()
2025-01-27 22:35:03 +01:00
litetex
222d869efc
Polish: Use ternary instead of if 2025-01-27 22:32:01 +01:00
litetex
625e9f96d2
Remove public modifier from tests 2025-01-27 22:24:15 +01:00
Isira Seneviratne
e475b2971b
Restore URL check 2025-01-27 22:21:47 +01:00
Isira Seneviratne
0352659e2b
Use BufferedReader#lines() 2025-01-27 22:21:47 +01:00
litetex
d5ade940fc
Merge pull request #1265 from gechoto/update-ios-client
[YouTube] Update IOS client
2025-01-27 22:17:53 +01:00
Stypox
f7adeeff90
Fix checkstyle 2025-01-26 16:09:22 +01:00
Stypox
1ca8275419
Merge pull request #1253 from gechoto/fix-n-func
[YouTube] Fixes for n param deobfuscation function
2025-01-26 15:15:00 +01:00
gechoto
4b16b7ee41
YoutubeParsingHelper: update IOS client 2025-01-26 13:47:30 +00:00
Stypox
b74a8d0df8
Remove manually writing cardinals of regexes 2025-01-26 13:07:23 +01:00
Stypox
2b70b5f737
Improve new regex to account for more context 2025-01-26 13:07:23 +01:00
gechoto
ca9f1935d3
YoutubeThrottlingParameterUtils: make third regex longer to ensure it is a function which takes one parameter 2025-01-26 13:07:23 +01:00
gechoto
c7d598869c
YoutubeThrottlingParameterUtils: swap regex positions again 2025-01-26 13:07:23 +01:00
gechoto
14f7ef1093
YoutubeThrottlingParameterUtils: add back second regex and swap it with the third one 2025-01-26 13:07:23 +01:00
gechoto
d007e368f3
don't commit testing code kids 2025-01-26 13:07:23 +01:00
gechoto
4181625c2c
add comment to throttling parameter deobfuscation function fixup 2025-01-26 13:07:23 +01:00
gechoto
e0d39b606c
correct comment 2025-01-26 13:07:23 +01:00
gechoto
1393644f44
formatting again 2025-01-26 13:07:23 +01:00
gechoto
68465b27fe
fix formatting 2025-01-26 13:07:23 +01:00
gechoto
56595bd9d5
[YouTube] Fix extraction of n param deobfuscation function name and fixup function to prevent early return 2025-01-26 13:07:23 +01:00
Stypox
e0359ab872
Merge pull request #1263 from litetex/followup-1262
Partial followup for #1262
2025-01-26 12:31:40 +01:00
litetex
1c45d4e240
Fix incorrect iPhone model 2025-01-23 22:08:59 +01:00
litetex
cca3e3ffc0
Use more specific exception 2025-01-23 22:08:59 +01:00
litetex
ea401638f4
Merge pull request #1249 from TeamNewPipe/dependabot/gradle/org.junit-junit-bom-5.11.4
Bump org.junit:junit-bom from 5.11.3 to 5.11.4
2025-01-23 21:56:50 +01:00
litetex
485a77d885
Merge pull request #1212 from Isira-Seneviratne/TimeAgoParser-unused
Remove unused method in TimeAgoParser
2025-01-23 21:54:55 +01:00
litetex
17f0b0d7d3
Fix checkstyle 2025-01-23 21:49:27 +01:00
litetex
746b36e80d
Improve tests
* Dedup using parameterized tests
* Make ``now`` controllable
2025-01-23 21:48:29 +01:00
Isira Seneviratne
ae07ba3c72
Remove unused method in TimeAgoParser 2025-01-23 20:56:16 +01:00
Stypox
a7154c3087
Release v0.24.4 v0.24.4 2025-01-21 23:14:31 +01:00
Stypox
4c720328ae
Merge pull request #1262 from Theta-Dev/fix/ios-client-vdata
[YouTube] update iOS client, add visitor data to requests
2025-01-21 23:12:58 +01:00
ThetaDev
936bf2d71b
Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/YoutubeParsingHelper.java
Co-authored-by: Stypox <stypox@pm.me>
2025-01-21 23:09:40 +01:00
Stypox
5b31ff20e5
iPhone 15 Pro Max instead of 16 in comments 2025-01-21 23:09:00 +01:00
ThetaDev
45645b043b
update comment 2025-01-21 13:12:51 +01:00
ThetaDev
65d888f4cf
move ProtoBuilder, add tests, fix incompatible rand call 2025-01-21 03:05:54 +01:00
ThetaDev
40059ed687
fix: update iOS client, add visitor data to YouTube requests 2025-01-19 19:59:10 +01:00
dependabot[bot]
a346f76704
Bump org.junit:junit-bom from 5.11.3 to 5.11.4
Bumps [org.junit:junit-bom](https://github.com/junit-team/junit5) from 5.11.3 to 5.11.4.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.11.3...r5.11.4)

---
updated-dependencies:
- dependency-name: org.junit:junit-bom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-17 09:49:42 +00:00
TobiGr
8e92227b2e Fix JDoc v0.24.3 2024-11-24 17:15:36 +01:00