89 Commits

Author SHA1 Message Date
7edc8ad5d3 Some whitespace changes :V 2022-03-06 17:07:05 -08:00
1d990e8a40 Remove users.yaml file 2022-03-06 16:11:45 -08:00
1582dfe44e Fix some lint problems 2022-03-06 15:36:07 -08:00
06167eef18 Fix over-enthusiastic commit 2022-03-06 15:26:27 -08:00
18c9f79429 REFACTOR: LINT: add .golangci.yaml lint config file
- Add linting for line lengths
- Fix some linting errors
2022-03-06 15:06:06 -08:00
524060a79a Fix a manual git-editing typo 2022-03-06 13:45:24 -08:00
5fb6a4aa9a REFACTOR: update persistence tests to use assert and require 2022-03-06 13:30:17 -08:00
68d4062820 Add a data migration to remove URLs that just link to a quoted tweet 2022-03-02 16:09:41 -08:00
508670248e Add list_followed subcommand 2022-02-28 16:06:58 -08:00
5f3013a49b Update tweet UserIDs in a tweet trove with Unknown Users in it (i.e., deleted users(?)) 2022-02-27 23:14:30 -08:00
772fa247f7 Add creation of unknown users with known handles (i.e., deleted accounts) 2022-02-26 22:09:27 -08:00
857b9c5255 Fix linting error 2022-02-26 16:06:51 -08:00
373b9ce16e Add IsFollowed field on Users 2022-02-26 15:58:30 -08:00
8a70f55e1b Update some more stuff to match API changes
- set X-Twitter-Client-Language header to "en"
- Add a new tombstone type (age-restricted)
2022-02-14 17:18:23 -08:00
f9a8841c07 REFACTOR: create SaveTweetTrove convenience method to reduce code in main.go 2022-02-14 13:39:59 -08:00
3d489a8e95 REFACTOR: eliminate 'download-if-needed' functions, add the 'if-needed' check to the regular downloading functions 2022-02-14 13:10:12 -08:00
b2199e9b59 Add Url#ShortText field 2022-02-01 15:48:43 -08:00
56870173e9 Fix bug with users with 600x200 banner images
When downloading these users' content, they weren't getting saved afterward (to update 'is_content_downloaded').
2022-01-18 17:48:56 -08:00
12f1cc6728 Fix some problems associated with banned users 2022-01-07 16:23:02 -05:00
6eac56183d Allow fetching of banned users 2022-01-07 13:40:22 -05:00
fb174df363 Add persistence for is_banned field 2022-01-07 12:38:56 -05:00
f25f1ae024 Change profile image downloading to use tiny profile image if appropriate 2022-01-05 12:37:11 -05:00
2210b61b3c Create a 'Follow' type 2022-01-04 14:40:51 -05:00
28e139a3e4 Add video durations and view counts 2021-12-24 16:26:34 -05:00
f73ef57cf2 Remove unique keyword from migration since that's apparently not allowed 2021-12-23 15:36:14 -05:00
fbbb6834de Add video thumbnails 2021-12-23 15:12:01 -05:00
692d3de564 Don't double-download tweet contents 2021-12-21 21:57:00 -05:00
5d44186f91 Add conditional content downloading for tweets 2021-12-21 21:49:42 -05:00
5a4da58572 Add conditional downloading for user content 2021-12-21 21:29:27 -05:00
c173f7e64e Fix missing trailing comma 2021-12-21 20:31:33 -05:00
5d054b5096 Add check for whether user needs content downloaded 2021-12-21 20:27:38 -05:00
13299233c0 Add 'suspended' tombstone text; add 'violated twitter rules' and 'account no longer exists' tombstone types 2021-12-21 16:05:51 -05:00
c1bcd54a11 Add Tweet#last_scraped_at and Tweet#is_conversation_scraped fields 2021-12-20 14:07:20 -05:00
c95ed6c918 Fix directory error in version bump test task 2021-12-12 20:17:05 -08:00
81a361d87e Add color to version upgrading messages 2021-12-12 20:10:08 -08:00
9357974a29 Make preparations for first upgrade release
- Create `CHANGELOG.txt` file
- Bump ENGINE_DATABASE_VERSION
- Add an upgrade-testing step to the CI build
2021-12-12 19:38:28 -08:00
28e44897b5 Add poll updating 2021-12-12 18:42:27 -08:00
c15ac9d751 Add persistence for polls 2021-12-12 16:42:32 -08:00
7f02d6acec Fix linting error 2021-11-22 17:03:57 -08:00
5f97014bdc Add database versioning 2021-11-22 16:55:27 -08:00
0de9b25dfe Change 'in_reply_to' and 'quoted_tweet' ID fields to say '_id' (hungarian notation) 2021-11-22 14:56:57 -08:00
4793cf8a4d Change users text file into a yaml file 2021-11-07 12:37:21 -08:00
470dce1d27 Persist tombstone data 2021-11-06 13:37:46 -07:00
4fa30e246d Add dimension info to media content 2021-10-10 16:06:47 -07:00
c219652dcc Make fetching very aggressive by default 2021-10-10 15:13:32 -07:00
1c2f356ee9 Fix re-downloading making users get marked as content-not-downloaded 2021-10-09 18:58:52 -07:00
100fe6c718 Fix gifs 2021-10-04 21:06:53 -07:00
6aa14c32e2 Add persistence for reply-mentions 2021-09-27 18:29:55 -07:00
0bb9ff6c6b Add handling for URL cards with no link preview thumbnail image 2021-09-17 20:50:28 -07:00
f9ddc0f102 Add downloading for link previews 2021-09-17 20:11:36 -07:00