28 Commits

Author SHA1 Message Date
b17e0821bb Make command line accept either URL or just ID to find a tweet 2021-09-27 13:49:17 -07:00
1dd02ff390 Fetching a user now downloads its content automatically 2021-09-27 12:54:18 -07:00
0bb9ff6c6b Add handling for URL cards with no link preview thumbnail image 2021-09-17 20:50:28 -07:00
f9ddc0f102 Add downloading for link previews 2021-09-17 20:11:36 -07:00
05c3f2289b Add persistence for new Url type 2021-09-17 18:04:57 -07:00
e2ff9436fa Add player type of URL card 2021-09-17 17:35:55 -07:00
ef9a98b79c Fix integration test (sample user got banned) 2021-09-17 15:03:23 -07:00
06831dfb52 Update help message 2021-08-22 18:43:33 -07:00
1f5ebf9f8d Enable fetching tweets to the limit of scrollback 2021-08-22 18:22:06 -07:00
9c944cdc5e Remove useless transaction block 2021-08-22 18:20:42 -07:00
a2554e9293 Enable 'fetch_tweet' full scrape 2021-08-22 17:55:21 -07:00
9eee64cca1 Catch random API 403 errors 2021-08-22 16:41:59 -07:00
08d703a26f Add command to get a user feed 2021-08-19 12:54:08 -07:00
4a3c12b04a Update some docstrings, rename a few methods 2021-08-16 20:35:15 -07:00
2b23fcbc53 twitter command now uses current directory as the profile dir by default 2021-08-11 06:56:42 -07:00
917c3fc7a7 Add integration test for downloading user contents 2021-08-10 22:06:03 -07:00
0a1fa92cdd Add to database: LocalPath version of BannerImage and ProfileImage on Users 2021-08-10 20:51:42 -07:00
3de47f6bde Add twitter command download_tweet_content 2021-08-07 17:06:39 -07:00
9c9100d874 Add RemoteURL and LocalFilename fields to Video, remove Filename 2021-08-05 14:10:46 -07:00
c803794a0f Change TweetID to be an int64 instead of a string 2021-08-04 14:49:20 -07:00
494ca25dc4 Change method of scraping images
- break `Filename` field on Image into `RemoteURL` and `LocalFilename`
- new parsing function for an APIMedia item
2021-08-03 17:34:44 -07:00
c4c2ef8979 Fix a test 2021-08-02 14:55:59 -07:00
18a63cd2eb Create twitter main package 2021-08-02 14:47:32 -07:00
6568259d63 Fix some other IDs that I missed 2021-07-24 10:29:35 -07:00
96e2edfb8a ID fields are now Hungarian-notated
- tweet.User is now a pointer to a User object; tweet.UserID holds the user id
2021-07-24 10:19:05 -07:00
b5c33c2ad0 Fix some types to be more clear 2021-07-22 14:16:40 -07:00
12b1aeaccd Remove redundant copies of cmd files 2021-06-16 19:36:03 -07:00
28b11548bf Add actual scraping 2021-06-16 19:34:06 -07:00