88 lines
2.9 KiB
Plaintext
88 lines
2.9 KiB
Plaintext
TODO user-images-extract-to-table:
|
|
User profile images and banner images => extract to table. Add `is_downloaded` column. Add test for double-downloading a user, shouldn't change the write-times on these files
|
|
|
|
TODO update-all
|
|
|
|
TODO fetch-older-tweets:
|
|
- get tweets older than 3200 ago
|
|
|
|
TODO: downloading-modes
|
|
- Could compose the following:
|
|
- Primary user's top-level tweets and threads
|
|
- Primary user's replies
|
|
- Primary user's retweets
|
|
- Parents and replies for primary user's tweets/replies
|
|
|
|
|
|
TODO broadcast-cards
|
|
panic: Unknown card type: 3691233323:periscope_broadcast
|
|
panic: Unknown card type: 745291183405076480:broadcast
|
|
|
|
TODO consecutive-tombstones
|
|
Conversation threads with two or more consecutive tombstones don't get scraped correctly
|
|
- e.g.: https://twitter.com/hatecoleslaw/status/1429584549701357568
|
|
- Hard mode: https://twitter.com/masforshort/status/1482000048447705090
|
|
|
|
TODO video-bitrates
|
|
- add "bitrate" and "available bitrates" fields
|
|
- option to upgrade or downgrade video quality
|
|
|
|
TODO: Problem tweets
|
|
- https://twitter.com/CovfefeAnon/status/1427693931047837702: parent thread not loading (possible GUI problem)
|
|
- "account no longer exists" tombstone not being collected => https://twitter.com/michaelmalice/status/1461031030278742020
|
|
- tweet with warning label not getting scraped right: https://twitter.com/michaelmalice/status/1493324611999748098
|
|
- fails to produce any result for the first tweet in the thread => https://twitter.com/CovfefeAnon/status/1498877082838962181
|
|
- twitter fetch_tweet 1517683230421528576 => panic: ScreenName is empty!
|
|
|
|
TODO: videos-view-count
|
|
- videos don't parse properly in APIv2
|
|
|
|
TODO: APIv2-pinned-tweets
|
|
- feed with pinned tweet
|
|
|
|
TODO video-not-downloading: https://twitter.com/JustinPulitzer/status/1496709662602932224
|
|
|
|
TODO sqlx
|
|
- increase coverage (not sure why this is done partially)
|
|
|
|
TODO get-user-tweets-user-doesn't-exist
|
|
|
|
TODO bash-completions
|
|
- Should have bash completions.
|
|
|
|
TODO implement-custom-unmarshal
|
|
- the parsing might be excessively complex, and it doesn't correctly catch failures. Might be good to just implement the Unmarshal interface instead
|
|
|
|
TODO twitter-spaces
|
|
- add `is_details_available` flag
|
|
|
|
TODO authenticated-requests
|
|
- likes tab
|
|
- media tab
|
|
- quote-tweets
|
|
- notifications
|
|
|
|
TODO post-tweets
|
|
- post tweet
|
|
- liking / replying to / quoting / retweeting a tweet
|
|
|
|
TODO fetch-DMs
|
|
|
|
TODO spaces-timestamps-millis
|
|
- spaces timestamps are in milliseconds. Need to be converted to seconds
|
|
|
|
TODO fix-spaces-participants-duplicates
|
|
- It is creating a new copy of the participants every time it gets scraped again
|
|
|
|
TODO scrape-spaces-participants-admin
|
|
- distinguish whether someone was an admin in the space or not
|
|
|
|
TODO authenticated-not-fetching-images
|
|
- Images in tweets don't fetch when you're authenticated
|
|
|
|
TODO tweets-with-italics
|
|
- https://twitter.com/exit_org/status/1656813511572443141
|
|
|
|
TODO authenticated-only-request-warning
|
|
- warn when attempting to run requests which require authentication without a session active
|