me/offline-twitter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
935 Commits 1 Branch 0 Tags
Commit Graph

209 Commits

Author SHA1 Message Date
Alessio
041af0f91d Move common type definitions (Tweet, User, etc) from 'scraper' package to 'persistence' 2025-02-14 16:02:40 -08:00
Alessio
4abbb93c63 Update link preview image integration test (old one broke again) 2025-02-14 14:46:56 -08:00
Alessio
4132eb4bca REFACTOR: SaveTweetTrove no longer calls scraper.GetUserByID, removing another scraper dependency from persistence package 
- it was calling it to handle conflicting user handles
- now user handle conflicts are handled by the callee
- Add a utility function in `main` and `webserver` packages to rescrape the conflicting users
 2025-02-10 12:53:31 -08:00
Alessio
fc66d1d8a6 Update 2 integration tests that got out-of-date 2025-02-10 12:31:46 -08:00
Alessio
b2f697a8a6 REFACTOR: make Session file storage functions more generic (not hardcoded for scraper.API) 2025-02-04 14:46:13 -08:00
Alessio
cb0b478c08 REFACTOR: apply dependency injection change (whole API object -> just the downloader function) to 'SaveTweetTrove' method 2025-02-03 21:58:53 -08:00
Alessio
0c1d853f55 REFACTOR: change media downloader dependency injection to take just the download function rather than a whole API object 2025-02-03 21:24:53 -08:00
Alessio
a3ddfa72b7 Fix a typo in the windows build script 2025-01-18 21:44:42 -08:00
Alessio
af21249e67 Fix a broken integration test 2025-01-18 20:38:32 -08:00
Alessio
23f023837b Try to add an icon to the compiled application on Windows 2025-01-12 20:26:48 -08:00
Alessio
7102530661 Update integration test for user feed to use a user who retweets stuff more often 2024-12-22 08:04:18 -08:00
Alessio
e27cd12cdc REFACTOR: change 'fetch_tweet_only' to use the regular fetch-tweet scraper function, and extract the main tweet in the 'main' package instead of having a scraper function for it 2024-12-21 09:21:23 -08:00
Alessio
b5df0f3b5e Update integration test to account for @Offline_Twatter unfollowing @POTUS 2024-12-09 15:03:48 -08:00
Alessio
72edf20890 Actually fully remove the test properly from the previous commit 2024-12-09 13:08:48 -08:00
Alessio
683c05c996 Disable integration test for guy who got banned 2024-12-09 12:59:46 -08:00
Alessio
178e38555f Reactivate integration tests that were temporarily deactivated 2024-12-09 11:53:02 -08:00
Alessio
ebf8ad5947 Fix an integration test due to twitter changing the way block works 2024-11-06 22:11:19 -08:00
Alessio
418187f925 temporarily disable an integration test because some schmuck unprivated his account 2024-11-06 21:57:07 -08:00
Alessio
faac7e9b16 Add marking notifications as read 2024-11-06 21:54:21 -08:00
Alessio
c7a6b5b321 Add back in the GetUserByID unauthenticated operation since apparently it does actually work 2024-11-06 18:58:18 -08:00
Alessio
2b7a636720 Remove the unauthenticated GetUser requests as they no longer work 2024-11-06 18:36:06 -08:00
Alessio
631d07c6fc Fix integration test (user changed their handle) 2024-11-03 18:38:16 -08:00
Alessio
3539d9418d SaveTweetTrove now re-fetches users with conflicting handles as well 2024-09-23 00:23:07 -07:00
Alessio
11b03a7927 Add integration test for resolving conflicting user handles 2024-09-22 22:58:32 -07:00
Alessio
f7cbe17deb Fix lint errors 2024-09-22 22:24:56 -07:00
Alessio
381e9e0706 'fetch_user' and 'fetch_user_by_id' subcommands now check for conflicting user handle errors and rescrape if needed 2024-09-22 22:11:08 -07:00
Alessio
79033cfc79 Make user handles only unique among active users 2024-09-17 18:33:58 -07:00
Alessio
8c390444d6 Temporarily remove the follow/unfollow integration tests 2024-09-17 18:25:25 -07:00
Alessio
51eaa2a0c4 Remove some useless functions ('list_followed', superceded by Lists; 'UserExists') 2024-09-17 18:13:54 -07:00
Alessio
4ea15f10af Add scraper function to get user by ID 2024-09-15 22:56:59 -07:00
Alessio
210328bcc9 Kick CI 2024-09-15 19:26:14 -07:00
Alessio
a418fc99bf Kick CI 2024-09-15 19:20:02 -07:00
Alessio
9c0f9504f6 "ConvertToAPIUser() now returns an error indicating a "not-found" response, which propagates through "GetUser" API calls 2024-09-15 17:59:25 -07:00
Alessio
a13f47e508 Make the data directory if needed 2024-09-15 15:22:07 -07:00
Alessio
67f2852ac7 Put pkg/persistence and cmd/data in tmpfs 2024-09-15 15:18:54 -07:00
Alessio
b9e025589e Update CHANGELOG.txt for v0.6.16 2024-09-03 18:36:23 -07:00
Alessio
9c7174a84a Add way to scrape unread count; add db query to count unread notifs since a given sort index 2024-09-02 15:02:27 -07:00
Alessio
0c620621a6 Add scraping notiication detail 2024-08-28 19:22:09 -07:00
Alessio
0a1cdc17af Fix integration test (link thumbnail image went down again) 2024-08-26 12:21:46 -07:00
Alessio
2c7c35e23f Add pagination to notifications 
- handle notifications end-of-feed
- add support for "quoted" and "mentioned" notifications
 2024-08-26 00:07:10 -07:00
Alessio
04991ad554 Add 'get_notifications' subcommand 2024-08-25 22:57:40 -07:00
Alessio
9a985539c5 Add some debug info to the integration test 2024-08-23 13:24:11 -07:00
Alessio
3f7a357b17 Fix lint error 2024-08-19 18:23:46 -07:00
Alessio
c45b8e7ad8 Add --delay flag to force a delay between requests in a large paginated scrape 2024-08-19 18:20:12 -07:00
Alessio
9c8c7cdf65 'webserver' subcommand now respects the '--session' flag 2024-08-19 17:49:36 -07:00
Alessio
e2ca9a975a Remove scraper singleton pattern entirely 2024-08-19 17:27:54 -07:00
Alessio
08e5a726a8 Remove most uses of the singleton pattern in 'scraper' package 2024-08-19 16:45:32 -07:00
Alessio
f8988abef1 Enable optionally using a stored session file in integration test, bypassing login 2024-08-19 14:15:57 -07:00
Alessio
8321d65f17 Fix broken integration test (link thumbnail disappeared, as usual) 2024-08-18 15:50:04 -07:00
Alessio
24129c4852 REFACTOR: reduce technical debt, particularly that caused by singleton pattern in pkg/scraper 
- ensure all scraper functions have a `api.XYZ` version and a package-level convenience function
	- isolate `the_api` to top-level convenience functions, in preparation for removal
- move a bunch of scraper functions around to be nearby their related functions
- new ErrLoginRequired
- remove obsolete APIv1 stuff (Feed, TweetDetail)
- rename scraper function GetUserFeedGraphqlFor => GetUserFeed
- fix go.mod Go version incorrectly claiming it's compatible with Go 1.16 (should be Go 1.17)
 2024-08-09 19:48:50 -07:00