|
|
d95f183e45
|
Add parsing of tombstoned quoted-tweets
|
2022-01-07 13:42:00 -05:00 |
|
|
|
6eac56183d
|
Allow fetching of banned users
|
2022-01-07 13:40:22 -05:00 |
|
|
|
f25f1ae024
|
Change profile image downloading to use tiny profile image if appropriate
|
2022-01-05 12:37:11 -05:00 |
|
|
|
e06b0bff30
|
Add --version flag to the executable
|
2022-01-04 12:57:43 -05:00 |
|
|
|
fbbb6834de
|
Add video thumbnails
|
2021-12-23 15:12:01 -05:00 |
|
|
|
692d3de564
|
Don't double-download tweet contents
|
2021-12-21 21:57:00 -05:00 |
|
|
|
469d846db4
|
Add another section of avoiding double-downloading user content
|
2021-12-21 21:37:04 -05:00 |
|
|
|
5a4da58572
|
Add conditional downloading for user content
|
2021-12-21 21:29:27 -05:00 |
|
|
|
8e13e30ac5
|
Scraping now updates last_scraped_at and sets is_conversation_scraped
|
2021-12-20 15:27:59 -05:00 |
|
|
|
28e44897b5
|
Add poll updating
|
2021-12-12 18:42:27 -08:00 |
|
|
|
ddb16c9e64
|
Remove integration test for tombstones because the guy un-privated for some reason
|
2021-12-12 16:14:11 -08:00 |
|
|
|
abbc525df9
|
Create first release attempt
|
2021-11-29 15:23:16 -08:00 |
|
|
|
6d7d940df4
|
Add search
|
2021-11-06 14:50:39 -07:00 |
|
|
|
470dce1d27
|
Persist tombstone data
|
2021-11-06 13:37:46 -07:00 |
|
|
|
e5b4b43358
|
Temporarily remove integration tests user who got banned ('suspended' handling doesn't work yet)
|
2021-11-01 16:04:53 -07:00 |
|
|
|
c219652dcc
|
Make fetching very aggressive by default
|
2021-10-10 15:13:32 -07:00 |
|
|
|
66daacec34
|
Fix the constantly-breaking thumbnail url test
|
2021-10-10 15:06:39 -07:00 |
|
|
|
1c2f356ee9
|
Fix re-downloading making users get marked as content-not-downloaded
|
2021-10-09 18:58:52 -07:00 |
|
|
|
b117cbec7b
|
Update some of the integration tests
|
2021-10-07 14:58:59 -07:00 |
|
|
|
100fe6c718
|
Fix gifs
|
2021-10-04 21:06:53 -07:00 |
|
|
|
6aa14c32e2
|
Add persistence for reply-mentions
|
2021-09-27 18:29:55 -07:00 |
|
|
|
f278010201
|
Fix linting error
|
2021-09-27 13:52:51 -07:00 |
|
|
|
b17e0821bb
|
Make command line accept either URL or just ID to find a tweet
|
2021-09-27 13:49:17 -07:00 |
|
|
|
1dd02ff390
|
Fetching a user now downloads its content automatically
|
2021-09-27 12:54:18 -07:00 |
|
|
|
0bb9ff6c6b
|
Add handling for URL cards with no link preview thumbnail image
|
2021-09-17 20:50:28 -07:00 |
|
|
|
f9ddc0f102
|
Add downloading for link previews
|
2021-09-17 20:11:36 -07:00 |
|
|
|
05c3f2289b
|
Add persistence for new Url type
|
2021-09-17 18:04:57 -07:00 |
|
|
|
e2ff9436fa
|
Add player type of URL card
|
2021-09-17 17:35:55 -07:00 |
|
|
|
ef9a98b79c
|
Fix integration test (sample user got banned)
|
2021-09-17 15:03:23 -07:00 |
|
|
|
06831dfb52
|
Update help message
|
2021-08-22 18:43:33 -07:00 |
|
|
|
1f5ebf9f8d
|
Enable fetching tweets to the limit of scrollback
|
2021-08-22 18:22:06 -07:00 |
|
|
|
9c944cdc5e
|
Remove useless transaction block
|
2021-08-22 18:20:42 -07:00 |
|
|
|
a2554e9293
|
Enable 'fetch_tweet' full scrape
|
2021-08-22 17:55:21 -07:00 |
|
|
|
9eee64cca1
|
Catch random API 403 errors
|
2021-08-22 16:41:59 -07:00 |
|
|
|
08d703a26f
|
Add command to get a user feed
|
2021-08-19 12:54:08 -07:00 |
|
|
|
4a3c12b04a
|
Update some docstrings, rename a few methods
|
2021-08-16 20:35:15 -07:00 |
|
|
|
2b23fcbc53
|
twitter command now uses current directory as the profile dir by default
|
2021-08-11 06:56:42 -07:00 |
|
|
|
917c3fc7a7
|
Add integration test for downloading user contents
|
2021-08-10 22:06:03 -07:00 |
|
|
|
0a1fa92cdd
|
Add to database: LocalPath version of BannerImage and ProfileImage on Users
|
2021-08-10 20:51:42 -07:00 |
|
|
|
3de47f6bde
|
Add twitter command download_tweet_content
|
2021-08-07 17:06:39 -07:00 |
|
|
|
9c9100d874
|
Add RemoteURL and LocalFilename fields to Video, remove Filename
|
2021-08-05 14:10:46 -07:00 |
|
|
|
c803794a0f
|
Change TweetID to be an int64 instead of a string
|
2021-08-04 14:49:20 -07:00 |
|
|
|
494ca25dc4
|
Change method of scraping images
- break `Filename` field on Image into `RemoteURL` and `LocalFilename`
- new parsing function for an APIMedia item
|
2021-08-03 17:34:44 -07:00 |
|
|
|
c4c2ef8979
|
Fix a test
|
2021-08-02 14:55:59 -07:00 |
|
|
|
18a63cd2eb
|
Create twitter main package
|
2021-08-02 14:47:32 -07:00 |
|
|
|
6568259d63
|
Fix some other IDs that I missed
|
2021-07-24 10:29:35 -07:00 |
|
|
|
96e2edfb8a
|
ID fields are now Hungarian-notated
- tweet.User is now a pointer to a User object; tweet.UserID holds the user id
|
2021-07-24 10:19:05 -07:00 |
|
|
|
b5c33c2ad0
|
Fix some types to be more clear
|
2021-07-22 14:16:40 -07:00 |
|
|
|
12b1aeaccd
|
Remove redundant copies of cmd files
|
2021-06-16 19:36:03 -07:00 |
|
|
|
28b11548bf
|
Add actual scraping
|
2021-06-16 19:34:06 -07:00 |
|
