me/offline-twitter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
846 Commits 1 Branch 0 Tags
Commit Graph

202 Commits

Author SHA1 Message Date
Alessio
90f453c207 Fix crashing on non-timeline-module composer button entries 2024-10-26 22:11:23 -07:00
Alessio
e319dc1988 Fix 'user followed you' notifications sometimes parsing as 'new recommended post' notifications 2024-10-23 22:40:43 -07:00
Alessio
859431f012 Initialize database version number in schema.sql, instead of Go code 2024-10-23 22:00:38 -07:00
Alessio
74db0f5797 Remove sqlx mapper function; just map all fields manually 2024-10-23 21:50:54 -07:00
Alessio
3539d9418d SaveTweetTrove now re-fetches users with conflicting handles as well 2024-09-23 00:23:07 -07:00
Alessio
f7cbe17deb Fix lint errors 2024-09-22 22:24:56 -07:00
Alessio
381e9e0706 'fetch_user' and 'fetch_user_by_id' subcommands now check for conflicting user handle errors and rescrape if needed 2024-09-22 22:11:08 -07:00
Alessio
6ed8beae26 Update SaveUser logic to account for duplicate handles, with better handling of deleted / banned users 
- BUGFIX: deleted / banned users will no longer have all their other info wiped
	- e.g., follower counts, profile image URLs, bio, etc.
 2024-09-22 16:22:33 -07:00
Alessio
79033cfc79 Make user handles only unique among active users 2024-09-17 18:33:58 -07:00
Alessio
51eaa2a0c4 Remove some useless functions ('list_followed', superceded by Lists; 'UserExists') 2024-09-17 18:13:54 -07:00
Alessio
4ea15f10af Add scraper function to get user by ID 2024-09-15 22:56:59 -07:00
Alessio
9c0f9504f6 "ConvertToAPIUser() now returns an error indicating a "not-found" response, which propagates through "GetUser" API calls 2024-09-15 17:59:25 -07:00
Alessio
2d35c37e17 Simply ErrNotInDatabase implementation 2024-09-15 17:47:51 -07:00
Alessio
526582d90c Fix lint error 2024-09-14 16:28:00 -07:00
Alessio
2731d787c2 Prevent failed SaveNotification transaction from locking the database 2024-09-14 16:01:28 -07:00
Alessio
b9e025589e Update CHANGELOG.txt for v0.6.16 2024-09-03 18:36:23 -07:00
Alessio
72b547f6aa Unread notifications count should be specific to a single user 2024-09-02 16:25:54 -07:00
Alessio
9c7174a84a Add way to scrape unread count; add db query to count unread notifs since a given sort index 2024-09-02 15:02:27 -07:00
Alessio
1f392f5240 Fix lint error 2024-08-31 23:06:54 -07:00
Alessio
f554846355 Add sample notification data, and compound query to retrieve notifications and associated data 2024-08-31 23:04:13 -07:00
Alessio
0c620621a6 Add scraping notiication detail 2024-08-28 19:22:09 -07:00
Alessio
dc816c6f28 Track retweet lists in notifications alongside tweets 2024-08-28 19:20:29 -07:00
Alessio
b8db01c0ab Add test file for previous commit 2024-08-26 00:19:24 -07:00
Alessio
2c7c35e23f Add pagination to notifications 
- handle notifications end-of-feed
- add support for "quoted" and "mentioned" notifications
 2024-08-26 00:07:10 -07:00
Alessio
b77612c66f Add SQL queries for notifications 2024-08-25 22:54:18 -07:00
Alessio
e94327b153 Add current user ID to notifications' UserID field (notification recient) when parsing 
- also make an enum for notification types instead of magic numbers in the code
 2024-08-25 22:48:09 -07:00
Alessio
fae23cc243 Fix lint errors 2024-08-25 19:01:52 -07:00
Alessio
eafdbf5ffb Add Notifications parser 2024-08-25 16:27:49 -07:00
Alessio
4853969b4d Fix lint error 2024-08-23 12:57:27 -07:00
Alessio
f3f0b4868d Update CHANGELOG.txt for v0.6.15 2024-08-23 12:52:16 -07:00
Alessio
c45b8e7ad8 Add --delay flag to force a delay between requests in a large paginated scrape 2024-08-19 18:20:12 -07:00
Alessio
4990e7913d Fix lint error 2024-08-19 17:29:54 -07:00
Alessio
e2ca9a975a Remove scraper singleton pattern entirely 2024-08-19 17:27:54 -07:00
Alessio
08e5a726a8 Remove most uses of the singleton pattern in 'scraper' package 2024-08-19 16:45:32 -07:00
Alessio
adbab97ca1 Reset the CI pipeline step conditions (don't force everything to run every time anymore) 2024-08-19 16:39:22 -07:00
Alessio
91f722b7fa Scraper requests now report invalidated or expired sessions 2024-08-18 16:22:37 -07:00
Alessio
24129c4852 REFACTOR: reduce technical debt, particularly that caused by singleton pattern in pkg/scraper 
- ensure all scraper functions have a `api.XYZ` version and a package-level convenience function
	- isolate `the_api` to top-level convenience functions, in preparation for removal
- move a bunch of scraper functions around to be nearby their related functions
- new ErrLoginRequired
- remove obsolete APIv1 stuff (Feed, TweetDetail)
- rename scraper function GetUserFeedGraphqlFor => GetUserFeed
- fix go.mod Go version incorrectly claiming it's compatible with Go 1.16 (should be Go 1.17)
 2024-08-09 19:48:50 -07:00
Alessio
b4384f9cc4 Enable sending emoji reactions in DMs 2024-07-28 14:09:41 -07:00
Alessio
c52960759e Fix bug caused by DMTrove refactor 2024-07-28 13:23:10 -07:00
Alessio
cf58793b33 Fix error caused by previous commit in banner image download logic 2024-07-28 13:07:14 -07:00
Alessio
8aca12695b Handle media download 404s gracefully 2024-07-28 12:50:00 -07:00
Alessio
d0c23b392b Don't crash when receiving a Grok card 2024-07-28 08:07:33 -07:00
Alessio
98015ddf0c REFACTOR: remove DMTrove type, merge it into TweetTrove 2024-07-28 08:01:58 -07:00
Alessio
a9a7af2195 Band-aid fix for spaces with null fields in them 2024-07-14 13:24:44 -07:00
Alessio
ef15e8a306 Handle guest token / session initialization when not connected to internet 2024-07-14 13:20:44 -07:00
Alessio
0fd17f1af0 Fix getting tweets from deleted accounts 2024-07-14 12:52:07 -07:00
Alessio
d2045028a7 Improve closing message when a scrape ends due to rate-limiting 2024-06-11 12:32:58 -07:00
Alessio
a1faacaf6b Don't throw away all previous results if you get rate-limited, just save them 2024-06-10 19:59:08 -07:00
Alessio
cac87ec973 Save tweets with unknown user as <UNKNOWN USER> instead of UserID 0, which causes foreign key errors 2024-06-10 15:47:13 -07:00
Alessio
14a7f88cdb Add scraping and saving of bookmarks 2024-05-28 21:55:42 -07:00