22 Commits

Author SHA1 Message Date
069ab83fba Refactor scraper tests
- use Stretchr assert package for assertions
- use "dot imports" to remove "scraper.XYZ" from everything
2022-01-31 19:14:14 -08:00
c15ac9d751 Add persistence for polls 2021-12-12 16:42:32 -08:00
0de9b25dfe Change 'in_reply_to' and 'quoted_tweet' ID fields to say '_id' (hungarian notation) 2021-11-22 14:56:57 -08:00
470dce1d27 Persist tombstone data 2021-11-06 13:37:46 -07:00
3b56973355 Remove reply-mentions for threads 2021-10-10 15:22:48 -07:00
100fe6c718 Fix gifs 2021-10-04 21:06:53 -07:00
18722a6b1b Refactor tweet_test: make a tweet loader helper function 2021-09-27 18:43:24 -07:00
159084006d Fix handling of reply-mentions and tweet text normalization 2021-09-27 18:12:28 -07:00
9ae6213025 Clean up the tests some more 2021-09-27 18:08:14 -07:00
89179f2447 Tidy up the scraper tests a bit 2021-09-27 15:30:05 -07:00
c865df8aca Add support for tweets with multiple URLs 2021-09-17 19:45:31 -07:00
05c3f2289b Add persistence for new Url type 2021-09-17 18:04:57 -07:00
9c9100d874 Add RemoteURL and LocalFilename fields to Video, remove Filename 2021-08-05 14:10:46 -07:00
c803794a0f Change TweetID to be an int64 instead of a string 2021-08-04 14:49:20 -07:00
d81fae0013 Make UserID an alias for int64 rather than string 2021-08-04 02:00:58 -07:00
81392996bb Add video and image types
- in addition to `Filename` (string), includes IsDownloaded (bool)
2021-07-26 17:26:39 -07:00
0c87428c44 Make videos their own table; store as a slice on Tweet rather than text field 2021-07-25 15:42:43 -07:00
647dd8aa6b Add scraping of videos from tweets 2021-07-25 14:51:17 -07:00
90e5fdb8b1 Add parser for a TweetResponse 2021-06-15 15:18:09 -07:00
02a2a0be33 Add parsing for user API type 2021-06-13 14:34:20 -07:00
e5337ef0d0 Change mentions to track usernames rather than user ids 2021-05-24 13:17:08 -04:00
2c8fe25e78 Add tweet parsing 2021-05-22 18:20:18 -04:00