me/offline-twitter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
894 Commits 1 Branch 0 Tags
Commit Graph

17 Commits

Author SHA1 Message Date
Alessio
e27cd12cdc REFACTOR: change 'fetch_tweet_only' to use the regular fetch-tweet scraper function, and extract the main tweet in the 'main' package instead of having a scraper function for it 2024-12-21 09:21:23 -08:00
Alessio
850662c3cb REFACTOR: move as much API code out of the types files and into api_types_... files as possible 
- also remove a few useless functions
 2024-12-21 09:15:02 -08:00
Alessio
08e5a726a8 Remove most uses of the singleton pattern in 'scraper' package 2024-08-19 16:45:32 -07:00
Alessio
24129c4852 REFACTOR: reduce technical debt, particularly that caused by singleton pattern in pkg/scraper 
- ensure all scraper functions have a `api.XYZ` version and a package-level convenience function
	- isolate `the_api` to top-level convenience functions, in preparation for removal
- move a bunch of scraper functions around to be nearby their related functions
- new ErrLoginRequired
- remove obsolete APIv1 stuff (Feed, TweetDetail)
- rename scraper function GetUserFeedGraphqlFor => GetUserFeed
- fix go.mod Go version incorrectly claiming it's compatible with Go 1.16 (should be Go 1.17)
 2024-08-09 19:48:50 -07:00
Alessio
0fd17f1af0 Fix getting tweets from deleted accounts 2024-07-14 12:52:07 -07:00
Alessio
0d12166966 Add parsing of DM images, videos and URLs 2024-03-08 17:22:52 -08:00
Alessio
a7ce15ebd2 Don't clobber interaction stats when fetching a deleted tweet 2024-02-18 14:52:28 -08:00
Alessio
92b166a4eb REFACTOR: create 'GetPaginatedQuery' function to encapsulate queries with cursors 2023-12-26 15:54:41 -06:00
Alessio
f3f2550d85 REFACTOR: create 'GetMore' function for paginated queries with cursors 2023-12-26 11:42:02 -06:00
Alessio
9448a84b28 Fix detection of deleted tweets 2023-10-23 13:48:38 -03:00
Alessio
9b85d81b89 Fix new video format and new user suspended format 2023-10-14 18:59:10 -03:00
Alessio
63ddaaeafb Scraper now handles implicit tombstones like '"quoted_status_result": {}' 2023-10-14 18:02:31 -03:00
Alessio
14b9143f76 Compound db queries now fetch whether the tweet is liked by the current logged-in user 2023-09-04 13:11:58 -03:00
Alessio
b2df94f041 Show tombstones 2023-08-29 12:27:53 -03:00
Alessio
eaa01a2360 Fix fetching users and search 
- Add is_deleted field on Users
- Fix fetching of tombstoned users including deleted users
- Fix "verified" bluechecks not being scraped anymore
- Fix search to use new graphql endpoint (old one got taken down)
 2023-08-22 20:07:32 -03:00
Alessio
107d21b7e6 Add retweet indicator on tweets 2023-08-13 15:57:58 -03:00
Alessio
a061decd0f REFACTOR: Rename go module to 'gitlab.com/offline-twitter/twitter_offline_engine' in accordance with 'go get' conventions 
- also restructure project to use a 'pkg' directory for reusable packages
 2023-07-30 14:20:07 -03:00