Changelog ========= v0.0.0 ------ - Initial release. v0.0.1 ------ - Added polls v0.0.2 ------ - Add `is_conversation_scraped` field to Tweets - Add some new tombstone types: "tweet violated twitter rules", "account no longer exists" - No longer re-download media content (tweet content, user profile / banner images) if not needed - Download video thumbnail images as part of video downloading v0.0.3 ------ - Add video `duration` and `view_count` fields - Add `--version` command line option - Improve user image scraping for rando users (download less eagerly) to speed up browsing and save disk space: - Download tiny versions of profile images (48x48 pixels) - Don't download banner images at all v0.0.4 ------ - Escape HTML entities in tweet text - Add support for banned users - Improve tombstone recognition (quote-tweets) - Fix spacing issue around tiny profile images v0.0.5 ------ - BUGFIX: fix parsing of tweet URLs with query params ('?s=20', etc) - BUGFIX: fix bug in scraping users with "600x200" banner images that caused them not to be saved correctly v0.1.0 ------ - Add support for new GraphQL twitter API (dubbed "APIv2" in code). User feed scraping now uses this API - Embedded links (URLs) now have their short versions saved in the DB. v0.1.1 ------ - BUGFIX: fix selection of primary "TimelineAddEntries" field in APIv2 following breaking API change - Add `-h, --help` command line flags to display help message and gracefully exit v0.1.2 ------ - BUGFIX: fix error when trying to match mixed-case user handles in tombstone tweets - BUGFIX: set "Client Language" header to "English" in all cases - Add an option to show debug logging; hide it by default v0.1.3 ------ - Add following users - Add `list_followed` subcommand - Add support for tweets that quote-tweet something from a user who deleted their account v0.1.4 ------ - BUGFIXES: Fixed some annoying bugs caused by Twitter's poor API - ignore empty entries in a user feed instead of crashing - quote tweets no longer include their quoted tweet as an embedded link v0.1.5 ------ - BUGFIX: users changing their profile images should be not broken now - Major refactoring (testing, massive linting improvements, proper Go dependencies like sqlx, etc) - Improve tombstone handling - Update tweet content when a tombstone becomes available - Update tombstone type of a tweet if it changes - If a tweet gets hidden / deleted, it can become a tombstone without losing its content - Enable support for users that change their handles v0.1.6 ------ - Add support for `mobile.twitter.com` tweet URLs - Add basic support for Spaces (i.e., can now basically detect whether a tweet links to a Space or not) v0.1.7 ------ - Add support for multiple videos as well as videos and images in the same tweet - Restructure media file storage into sub-directories to improve performance when retrieving them v0.1.8 ------ - Add proper support for fetching Spaces (with full detail) - Add checking for DMCA takedowns of video content - Add index for fetching tweet threads (index `in_reply_to_id`) - Add indexes to tweet content types to improve fetching performance (index `tweet_id`) - BUGFIX: can now fetch retweets with "visibility results" - BUGFIX: search changed how cursors work at end-of-feed v0.2.0 ------ - Add index on tweets user_id column to accelerate loading of user feeds - Implement login and authenticated requests! v0.2.1 ------ - BUGFIX: fix cursors being broken in authenticated sessions - BUGFIX: fix stats for likes, RTs, etc. getting erased if a tweet gets deleted or hidden - BUGFIX: `fetch_tweet` and `search` subcommands no longer ignore the `-n [number]` flag - ".session" extension for the `--session` flag is now optional, allowing tab-autocompletion v0.2.2 ------ - Add support for Expandable (i.e., "Show more") tweets - Fix an issue with half the tweets on user feeds not getting fetched while logged in