2087 Commits

Author SHA1 Message Date
gechoto
4181625c2c
add comment to throttling parameter deobfuscation function fixup 2025-01-26 13:07:23 +01:00
gechoto
e0d39b606c
correct comment 2025-01-26 13:07:23 +01:00
gechoto
1393644f44
formatting again 2025-01-26 13:07:23 +01:00
gechoto
68465b27fe
fix formatting 2025-01-26 13:07:23 +01:00
gechoto
56595bd9d5
[YouTube] Fix extraction of n param deobfuscation function name and fixup function to prevent early return 2025-01-26 13:07:23 +01:00
litetex
1c45d4e240
Fix incorrect iPhone model 2025-01-23 22:08:59 +01:00
litetex
cca3e3ffc0
Use more specific exception 2025-01-23 22:08:59 +01:00
litetex
17f0b0d7d3
Fix checkstyle 2025-01-23 21:49:27 +01:00
litetex
746b36e80d
Improve tests
* Dedup using parameterized tests
* Make ``now`` controllable
2025-01-23 21:48:29 +01:00
Isira Seneviratne
ae07ba3c72
Remove unused method in TimeAgoParser 2025-01-23 20:56:16 +01:00
ThetaDev
936bf2d71b
Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/YoutubeParsingHelper.java
Co-authored-by: Stypox <stypox@pm.me>
2025-01-21 23:09:40 +01:00
Stypox
5b31ff20e5
iPhone 15 Pro Max instead of 16 in comments 2025-01-21 23:09:00 +01:00
ThetaDev
45645b043b
update comment 2025-01-21 13:12:51 +01:00
ThetaDev
65d888f4cf
move ProtoBuilder, add tests, fix incompatible rand call 2025-01-21 03:05:54 +01:00
ThetaDev
40059ed687
fix: update iOS client, add visitor data to YouTube requests 2025-01-19 19:59:10 +01:00
TobiGr
8e92227b2e Fix JDoc 2024-11-24 17:15:36 +01:00
Stypox
9fb03f6c87
Merge pull request #1192 from TeamNewPipe/user-agent
[tests] Update user agent
2024-11-18 17:13:55 +01:00
TobiGr
e4a1a6ecd8 Fix tests 2024-11-17 21:38:47 +01:00
TobiGr
727e791602 [YouTube] Update mocks 2024-11-17 21:38:46 +01:00
TobiGr
d635d4db2a Make RecordingDownloader more resillient against ReCaptchaExceptions
Implement a max number of requests per minute to prevent hitting reate limits and triggering ReCaptchaExceptions. This slows down the RecordingDownloader significantly and can be adjusted if needed. A request ist retried once when facing a ReCaptchaException.
2024-11-17 21:38:46 +01:00
Stypox
c00d0a7028
Revert "Refactored Identifiers (#1205)"
This reverts commit 0de224124be656b5792235303b7ce4dd0063165f.
2024-11-16 14:00:38 +01:00
Stypox
d3d5f2b3f0
Merge pull request #1240 from AudricV/yt_fix-playlists-items-extraction
[YouTube] Add support for new playlist items data structure
2024-11-14 16:37:35 +01:00
congyuluo
0de224124b
Refactored Identifiers (#1205)
Extractor.pageFetched -> Extractor.isPageFetched
Stream.equalStats(Stream) is renamed to Stream.areStatsEqual(Stream)
Stream.getBitrate() is renamed to Stream.getBitRate()
2024-11-13 10:01:20 +01:00
AudricV
183563cc9e
[YouTube] Add support for playlists lockupViewModels
This new data type, A/B tested or rolled out at the time the changes
are commited, is present on multiple surfaces.
2024-11-10 21:44:06 +01:00
AudricV
f52d2269fc
[YouTube] Move channel verified status check from badges to a method
This method will be used in more places such as the new playlist item
data.

Support for a new icon used in artist channels has been also added.
2024-11-10 20:18:04 +01:00
TobiGr
667c867ad8 Update user agent to Firefox ESR 128 2024-11-10 17:27:22 +01:00
AudricV
06b2c8e2aa
[YouTube] Fix Shorts' thumbnails extraction in their channel tab
Wrong methods were used to access and extract the thumbnails' data.
This has been fixed with this commit.
2024-11-06 09:31:42 +01:00
Tobi
c343e31ed2
Merge pull request #1236 from Thompson3142/fix_scrubbing_seekbar_preview_crash
Add documentation for faulty framesets
2024-10-27 14:42:06 +01:00
TobiGr
1f26c12098 Use JDoc and inherit doc 2024-10-27 09:45:15 +01:00
AudricV
8a3350f79d
[YouTube] Add support for automatic dubbed and secondary tracks 2024-10-26 20:32:39 +02:00
Thompson3142
542867ff4d Add documentation for faulty framesets 2024-10-24 19:36:35 +02:00
Tobi
5431069588
Merge pull request #1231 from AudricV/yt_fix-n-param-decode-function-extraction
[YouTube] Fix extraction of n param deobfuscation function name
2024-10-05 20:16:15 +02:00
Tobi
743a4000b8
Merge pull request #1207 from TeamNewPipe/fix/peertube-certain-domains
[PeerTube] Fix parsing ID for instances whose domain ends with a or c
2024-10-05 19:20:42 +02:00
AudricV
69ff271be1
[YouTube] Fix extraction of n param deobfuscation function name
This commit adds two new regular expressions to parse the n parameter function.

It also improves existing regular expressions by using the constant representing
multiple characters instead of adding the one or multiple characters
token manually in each regex for everything and not only function names.
2024-09-29 16:01:10 +02:00
AudricV
42c1afaf87
[YouTube] Fix serialization of Videos channel tab when already fetched
Also remove usage of Optional as fields as it is not a good practice. This
simplifies in some places channel info extraction code.
2024-09-29 15:35:21 +02:00
AudricV
f9ffdd91d5
[YouTube] Update YoutubeChannelTabExtractorTest.Shorts test class mocks 2024-09-08 17:51:08 +02:00
AudricV
34f28fc1f0
[YouTube] Remove visitorData usage for shorts continuations
It isn't required anymore and not used by extractor anymore since commit
5a6da5f43ebb0a612e6ffdd3feb90d78b3dd383e, as the wrong page ID is used as a
visitor data (the VerifiedStatus value as a string).
2024-09-08 17:41:23 +02:00
AudricV
f926fbcf35
[YouTube] Add support for shortsLockupViewModels
This new UI data type is replacing the reelItemRenderer one.
2024-09-08 17:21:40 +02:00
TobiGr
6e3a4a6d9d [SoundCloud] Fix test: title changed 2024-08-15 12:22:37 +02:00
TobiGr
bcacfc53c5 [PeerTube] Fix parsing id for instances whose domain ends with a or c
The pattern to detect the channel ID was faulty and e.g. the ID detected for "https://kolektiva.media/video-channels/documentary_channel" was "a/video-channels" which is wrong.  A new pattern is added to distinguish between URLs and potential IDs because IDs must not start with a "/" while IDs inside an URL must.

Fixes TeamNewPipe/NewPipe#11369
2024-08-02 18:19:45 +02:00
AudricV
8db724943d
[YouTube] Update mocks of YoutubeChannelTabExtractorTest.Shorts test
For some reason, mocks of the continuation were not parsed. All mocks of the
test have been updated to fix the issue.
2024-07-25 17:51:43 +02:00
AudricV
76956ec95f
[YouTube] Fix VSauce test of YoutubeChannelExtractorTest test class
The channel description has been changed and some expected words have been
removed.
2024-07-25 17:51:43 +02:00
AudricV
10704dfc94
[YouTube] Fix NPE when getting channel header for videos channel tab 2024-07-25 17:51:43 +02:00
AudricV
8be64574e4
[YouTube] Support pageHeader on user channels
Also move duplicate strings into constants and add a missing default switch
case.
2024-07-25 17:51:42 +02:00
AudricV
df26badd4a
[YouTube] Add common methods to get ID, name and age gate object of channels
Also move duplicate strings into constants and support pageHeader channel
header in user channels on YoutubeChannelHelper methods.
2024-07-24 19:51:58 +02:00
AudricV
5a6da5f43e
[YouTube] Support shows in channels and provide verified status to items
Also fix naming of info items' collection methods.
2024-07-24 19:51:58 +02:00
AudricV
9d5201f40e
[YouTube] Add support for showRenderers in search results 2024-07-24 19:51:58 +02:00
AudricV
37178bd007
[YouTube] Add base implementation for show InfoItems
As there are multiple show UI elements which share a lot of common data, a base
implementation, an abstract class named YoutubeBaseShowInfoItemExtractor, has
been created to handle common cases.
2024-07-24 19:51:57 +02:00
AudricV
5879190ada
[YouTube] Move channel header's verified status code to YoutubeChannelHelper
Also throw an exception when we cannot get the verified status of a channel in
YoutubeChannelExtractor due to a missing channelHeader, if the channel has no
channelAgeGateRenderer.
2024-07-24 19:51:56 +02:00
AudricV
9fa8d4c0b4
[YouTube] Support playlists as URL navigation endpoints 2024-07-24 18:47:38 +02:00