|
|
6df313558d
|
Fix lint errors
|
2023-06-03 09:17:12 -03:00 |
|
|
|
815a8180da
|
BUGFIX: don't clobber num_likes, num_retweets etc. if tweet is a stub
- Convert Tweet queries to sqlx, including new CommaSeparatedList type to convert Hashtags, Mentions and ReplyMentions
|
2023-06-03 08:30:49 -03:00 |
|
|
|
af93f44ed2
|
Two bug fixes
- Search is now authenticated only
- `fetch_tweet` and `search` subcommands now respect the `-n [number]` flag
|
2023-06-03 07:38:17 -03:00 |
|
|
|
c09b77f497
|
Add ability to use saved sessions
|
2023-02-17 13:07:12 -05:00 |
|
|
|
fe57763568
|
Change entrypoint functions to use NewGuestSession constructor for API
|
2022-12-10 10:58:32 -05:00 |
|
|
|
55c4db32de
|
Fix lint
|
2022-11-24 19:01:30 -05:00 |
|
|
|
0293f11b9b
|
Fix parsing of empty Space responses
|
2022-11-24 18:57:42 -05:00 |
|
|
|
d54e77b169
|
Some housekeeping
|
2022-11-13 12:07:36 -05:00 |
|
|
|
6de2d670e8
|
Fix lint error
|
2022-10-15 12:47:41 -04:00 |
|
|
|
d647307097
|
Support multiple videos / videos+images
|
2022-10-15 12:40:40 -04:00 |
|
|
|
54857f40cd
|
Add parsing of tweets with Space links in them
|
2022-05-14 15:02:58 -07:00 |
|
|
|
d1d80a91cd
|
Add 'gofmt' linter
|
2022-03-13 17:13:40 -07:00 |
|
|
|
f08da27cc2
|
Enable 'wrapcheck' linter
|
2022-03-13 16:13:16 -07:00 |
|
|
|
f41d072573
|
Improve error handling some more
|
2022-03-06 20:31:04 -08:00 |
|
|
|
15d7cd77a1
|
Do slightly more proper error handling
|
2022-03-06 19:27:30 -08:00 |
|
|
|
26ab9ec937
|
Create new Timestamp helper type
|
2022-03-06 18:09:43 -08:00 |
|
|
|
f22f15f3d9
|
Prevent quoted-tweet URLs from being added as URLs (they should just be quote-tweets)
|
2022-03-02 14:34:42 -08:00 |
|
|
|
e0f05cbeea
|
Refactor all scraper areas to use TweetTroves where applicable
|
2022-02-12 20:39:30 -08:00 |
|
|
|
8892785aad
|
Add fetching of tombstoned users for APIv2
|
2022-02-02 14:49:46 -08:00 |
|
|
|
b2199e9b59
|
Add Url#ShortText field
|
2022-02-01 15:48:43 -08:00 |
|
|
|
c253486580
|
Fix lint error
|
2022-01-07 13:52:33 -05:00 |
|
|
|
d95f183e45
|
Add parsing of tombstoned quoted-tweets
|
2022-01-07 13:42:00 -05:00 |
|
|
|
d91e76f664
|
Remove print statements
|
2021-12-20 18:17:39 -05:00 |
|
|
|
8e13e30ac5
|
Scraping now updates last_scraped_at and sets is_conversation_scraped
|
2021-12-20 15:27:59 -05:00 |
|
|
|
c1bcd54a11
|
Add Tweet#last_scraped_at and Tweet#is_conversation_scraped fields
|
2021-12-20 14:07:20 -05:00 |
|
|
|
c15ac9d751
|
Add persistence for polls
|
2021-12-12 16:42:32 -08:00 |
|
|
|
0de9b25dfe
|
Change 'in_reply_to' and 'quoted_tweet' ID fields to say '_id' (hungarian notation)
|
2021-11-22 14:56:57 -08:00 |
|
|
|
470dce1d27
|
Persist tombstone data
|
2021-11-06 13:37:46 -07:00 |
|
|
|
100fe6c718
|
Fix gifs
|
2021-10-04 21:06:53 -07:00 |
|
|
|
159084006d
|
Fix handling of reply-mentions and tweet text normalization
|
2021-09-27 18:12:28 -07:00 |
|
|
|
9ae6213025
|
Clean up the tests some more
|
2021-09-27 18:08:14 -07:00 |
|
|
|
c865df8aca
|
Add support for tweets with multiple URLs
|
2021-09-17 19:45:31 -07:00 |
|
|
|
05c3f2289b
|
Add persistence for new Url type
|
2021-09-17 18:04:57 -07:00 |
|
|
|
4a3c12b04a
|
Update some docstrings, rename a few methods
|
2021-08-16 20:35:15 -07:00 |
|
|
|
9734c09426
|
Add downloading for media in tweets
|
2021-08-07 16:51:38 -07:00 |
|
|
|
58dc223f84
|
Use VideoID given by the Twitter API instead of a rowid
|
2021-08-04 23:41:58 -07:00 |
|
|
|
c803794a0f
|
Change TweetID to be an int64 instead of a string
|
2021-08-04 14:49:20 -07:00 |
|
|
|
d81fae0013
|
Make UserID an alias for int64 rather than string
|
2021-08-04 02:00:58 -07:00 |
|
|
|
494ca25dc4
|
Change method of scraping images
- break `Filename` field on Image into `RemoteURL` and `LocalFilename`
- new parsing function for an APIMedia item
|
2021-08-03 17:34:44 -07:00 |
|
|
|
81392996bb
|
Add video and image types
- in addition to `Filename` (string), includes IsDownloaded (bool)
|
2021-07-26 17:26:39 -07:00 |
|
|
|
0c87428c44
|
Make videos their own table; store as a slice on Tweet rather than text field
|
2021-07-25 15:42:43 -07:00 |
|
|
|
647dd8aa6b
|
Add scraping of videos from tweets
|
2021-07-25 14:51:17 -07:00 |
|
|
|
2086a5e983
|
FIXUP: rename Video field in Tweet
|
2021-07-24 21:52:39 -07:00 |
|
|
|
d386ba2cb4
|
Make String() methods more useful for terminal output
|
2021-07-24 10:55:44 -07:00 |
|
|
|
96e2edfb8a
|
ID fields are now Hungarian-notated
- tweet.User is now a pointer to a User object; tweet.UserID holds the user id
|
2021-07-24 10:19:05 -07:00 |
|
|
|
b5c33c2ad0
|
Fix some types to be more clear
|
2021-07-22 14:16:40 -07:00 |
|
|
|
28b11548bf
|
Add actual scraping
|
2021-06-16 19:34:06 -07:00 |
|
|
|
90e5fdb8b1
|
Add parser for a TweetResponse
|
2021-06-15 15:18:09 -07:00 |
|
|
|
e5337ef0d0
|
Change mentions to track usernames rather than user ids
|
2021-05-24 13:17:08 -04:00 |
|
|
|
5449b16038
|
Use new helper methods in tweet re-parser
|
2021-05-23 21:03:55 -04:00 |
|
