TODO user-images-extract-to-table: User profile images and banner images => extract to table. Add `is_downloaded` column. Add test for double-downloading a user, shouldn't change the write-times on these files TODO update-all TODO: downloading-modes - Could compose the following: - Primary user's top-level tweets and threads - Primary user's replies - Primary user's retweets - Parents and replies for primary user's tweets/replies TODO broadcast-cards panic: Unknown card type: 3691233323:periscope_broadcast panic: Unknown card type: 745291183405076480:broadcast TODO video-bitrates - add "bitrate" and "available bitrates" fields - option to upgrade or downgrade video quality TODO: videos-view-count - videos don't parse properly in APIv2 TODO: APIv2-pinned-tweets - feed with pinned tweet TODO video-not-downloading: https://twitter.com/JustinPulitzer/status/1496709662602932224 TODO use-sqlx - Users - Images, Videos, Urls, Polls - Retweets TODO get-user-tweets-user-doesn't-exist TODO bash-completions - Should have bash completions. TODO twitter-spaces - add `is_details_available` flag TODO authenticated-requests - media tab - quote-tweets - notifications TODO post-tweets - post tweets - reply to a tweet - quote-tweet a tweet - retweet a tweet TODO fetch-DMs TODO spaces-timestamps-millis - spaces timestamps are in milliseconds. Need to be converted to seconds TODO fix-spaces-participants-duplicates - It is creating a new copy of the participants every time it gets scraped again TODO spaces-fetch-participants-info - some user information is available in a Spaces result. Right now we just render them as null users (handle = "", id = 0) if they're not fetched already - distinguish whether someone was an admin in the space or not TODO spaces-fetch-sharings - you can fetch the stuff that was posted in the jumbotron TODO authenticated-not-fetching-images - Images in tweets don't fetch when you're authenticated TODO tweets-with-italics - https://twitter.com/exit_org/status/1656813511572443141 TODO authenticated-only-request-warning - warn when attempting to run requests which require authentication without a session active TODO unknown-user-upgrading - if an unknown user (created to satisfy SQLite integrity check on user_id foreign key) has user details fetched, the "Join Date" and other zero-value fields should update with the new value TODO apiv2-tombstones - on Tweet Detail: - tombstones in replies to the focused tweet - on User Feed: - tombstones (e.g, "Age restricted") that show up as top-level entries - keep in mind that these might be retweets; the entry-id is the RetweetID, not the TweetID TODO tidy-up-command-line-flags - make flags that are part of a subcommand go after that subcommand (e.g., `twitter get_user_tweets -n 200 ...`) TODO: fatih-color - replace "terminal-utils" with "fatih/color" TODO: mobile-requests - implement mobile versions of various requests - Check in `pkg/scraper/test_responses/mobile_requests` Webserver --------- TODO: SQL-logger TODO const-select-fields - There's a few queries that use "select ..." and write out all the fields. Put the field names in one spot TODO compound-query-structs - Right now the result structs for the compound queries (e.g., "Feed", "TweetDetail") are in the `persistence` package. They don't go there probably TOOD: login-routes-tests - Make the scraper.API object injectable somehow (get rid of singleton pattern) and add tests for login and change-session sequences - Also test profile.ListSessions() TODO: web-ui-downloading - web UI needs buttons to trigger a scrape / refresh manually TODO: web-ui-search TODO: webserver-session-arg-active-user - make the active user get set on initializing the Application object if a --session flag is given TODO: webserver-tombstones