10 Commits

Author SHA1 Message Date
Alessio
041af0f91d Move common type definitions (Tweet, User, etc) from 'scraper' package to 'persistence' 2025-02-14 16:02:40 -08:00
Alessio
4ea15f10af Add scraper function to get user by ID 2024-09-15 22:56:59 -07:00
Alessio
91f722b7fa Scraper requests now report invalidated or expired sessions 2024-08-18 16:22:37 -07:00
Alessio
24129c4852 REFACTOR: reduce technical debt, particularly that caused by singleton pattern in pkg/scraper
- ensure all scraper functions have a `api.XYZ` version and a package-level convenience function
	- isolate `the_api` to top-level convenience functions, in preparation for removal
- move a bunch of scraper functions around to be nearby their related functions
- new ErrLoginRequired
- remove obsolete APIv1 stuff (Feed, TweetDetail)
- rename scraper function GetUserFeedGraphqlFor => GetUserFeed
- fix go.mod Go version incorrectly claiming it's compatible with Go 1.16 (should be Go 1.17)
2024-08-09 19:48:50 -07:00
Alessio
8aca12695b Handle media download 404s gracefully 2024-07-28 12:50:00 -07:00
Alessio
ef15e8a306 Handle guest token / session initialization when not connected to internet 2024-07-14 13:20:44 -07:00
69e0a35e57 Handle HTTP request timeouts 2024-03-16 19:55:05 -07:00
aa05708e20 Move media downloader from persistence to scraper package; add 429 Rate Limited error type 2024-03-11 12:57:58 -07:00
92b166a4eb REFACTOR: create 'GetPaginatedQuery' function to encapsulate queries with cursors 2023-12-26 15:54:41 -06:00
a061decd0f REFACTOR: Rename go module to 'gitlab.com/offline-twitter/twitter_offline_engine' in accordance with 'go get' conventions
- also restructure project to use a 'pkg' directory for reusable packages
2023-07-30 14:20:07 -03:00