|
|
f22f15f3d9
|
Prevent quoted-tweet URLs from being added as URLs (they should just be quote-tweets)
|
2022-03-02 14:34:42 -08:00 |
|
|
|
e0f05cbeea
|
Refactor all scraper areas to use TweetTroves where applicable
|
2022-02-12 20:39:30 -08:00 |
|
|
|
8892785aad
|
Add fetching of tombstoned users for APIv2
|
2022-02-02 14:49:46 -08:00 |
|
|
|
b2199e9b59
|
Add Url#ShortText field
|
2022-02-01 15:48:43 -08:00 |
|
|
|
c253486580
|
Fix lint error
|
2022-01-07 13:52:33 -05:00 |
|
|
|
d95f183e45
|
Add parsing of tombstoned quoted-tweets
|
2022-01-07 13:42:00 -05:00 |
|
|
|
d91e76f664
|
Remove print statements
|
2021-12-20 18:17:39 -05:00 |
|
|
|
8e13e30ac5
|
Scraping now updates last_scraped_at and sets is_conversation_scraped
|
2021-12-20 15:27:59 -05:00 |
|
|
|
c1bcd54a11
|
Add Tweet#last_scraped_at and Tweet#is_conversation_scraped fields
|
2021-12-20 14:07:20 -05:00 |
|
|
|
c15ac9d751
|
Add persistence for polls
|
2021-12-12 16:42:32 -08:00 |
|
|
|
0de9b25dfe
|
Change 'in_reply_to' and 'quoted_tweet' ID fields to say '_id' (hungarian notation)
|
2021-11-22 14:56:57 -08:00 |
|
|
|
470dce1d27
|
Persist tombstone data
|
2021-11-06 13:37:46 -07:00 |
|
|
|
100fe6c718
|
Fix gifs
|
2021-10-04 21:06:53 -07:00 |
|
|
|
159084006d
|
Fix handling of reply-mentions and tweet text normalization
|
2021-09-27 18:12:28 -07:00 |
|
|
|
9ae6213025
|
Clean up the tests some more
|
2021-09-27 18:08:14 -07:00 |
|
|
|
c865df8aca
|
Add support for tweets with multiple URLs
|
2021-09-17 19:45:31 -07:00 |
|
|
|
05c3f2289b
|
Add persistence for new Url type
|
2021-09-17 18:04:57 -07:00 |
|
|
|
4a3c12b04a
|
Update some docstrings, rename a few methods
|
2021-08-16 20:35:15 -07:00 |
|
|
|
9734c09426
|
Add downloading for media in tweets
|
2021-08-07 16:51:38 -07:00 |
|
|
|
58dc223f84
|
Use VideoID given by the Twitter API instead of a rowid
|
2021-08-04 23:41:58 -07:00 |
|
|
|
c803794a0f
|
Change TweetID to be an int64 instead of a string
|
2021-08-04 14:49:20 -07:00 |
|
|
|
d81fae0013
|
Make UserID an alias for int64 rather than string
|
2021-08-04 02:00:58 -07:00 |
|
|
|
494ca25dc4
|
Change method of scraping images
- break `Filename` field on Image into `RemoteURL` and `LocalFilename`
- new parsing function for an APIMedia item
|
2021-08-03 17:34:44 -07:00 |
|
|
|
81392996bb
|
Add video and image types
- in addition to `Filename` (string), includes IsDownloaded (bool)
|
2021-07-26 17:26:39 -07:00 |
|
|
|
0c87428c44
|
Make videos their own table; store as a slice on Tweet rather than text field
|
2021-07-25 15:42:43 -07:00 |
|
|
|
647dd8aa6b
|
Add scraping of videos from tweets
|
2021-07-25 14:51:17 -07:00 |
|
|
|
2086a5e983
|
FIXUP: rename Video field in Tweet
|
2021-07-24 21:52:39 -07:00 |
|
|
|
d386ba2cb4
|
Make String() methods more useful for terminal output
|
2021-07-24 10:55:44 -07:00 |
|
|
|
96e2edfb8a
|
ID fields are now Hungarian-notated
- tweet.User is now a pointer to a User object; tweet.UserID holds the user id
|
2021-07-24 10:19:05 -07:00 |
|
|
|
b5c33c2ad0
|
Fix some types to be more clear
|
2021-07-22 14:16:40 -07:00 |
|
|
|
28b11548bf
|
Add actual scraping
|
2021-06-16 19:34:06 -07:00 |
|
|
|
90e5fdb8b1
|
Add parser for a TweetResponse
|
2021-06-15 15:18:09 -07:00 |
|
|
|
e5337ef0d0
|
Change mentions to track usernames rather than user ids
|
2021-05-24 13:17:08 -04:00 |
|
|
|
5449b16038
|
Use new helper methods in tweet re-parser
|
2021-05-23 21:03:55 -04:00 |
|
|
|
2c8fe25e78
|
Add tweet parsing
|
2021-05-22 18:20:18 -04:00 |
|
