409 lines
12 KiB
Plaintext
409 lines
12 KiB
Plaintext
Changelog
|
|
=========
|
|
|
|
v0.0.0
|
|
------
|
|
|
|
- Initial release.
|
|
|
|
v0.0.1
|
|
------
|
|
|
|
- Added polls
|
|
|
|
v0.0.2
|
|
------
|
|
|
|
- Add `is_conversation_scraped` field to Tweets
|
|
- Add some new tombstone types: "tweet violated twitter rules", "account no longer exists"
|
|
- No longer re-download media content (tweet content, user profile / banner images) if not needed
|
|
- Download video thumbnail images as part of video downloading
|
|
|
|
v0.0.3
|
|
------
|
|
|
|
- Add video `duration` and `view_count` fields
|
|
- Add `--version` command line option
|
|
- Improve user image scraping for rando users (download less eagerly) to speed up browsing and save disk space:
|
|
- Download tiny versions of profile images (48x48 pixels)
|
|
- Don't download banner images at all
|
|
|
|
v0.0.4
|
|
------
|
|
|
|
- Escape HTML entities in tweet text
|
|
- Add support for banned users
|
|
- Improve tombstone recognition (quote-tweets)
|
|
- Fix spacing issue around tiny profile images
|
|
|
|
v0.0.5
|
|
------
|
|
|
|
- BUGFIX: fix parsing of tweet URLs with query params ('?s=20', etc)
|
|
- BUGFIX: fix bug in scraping users with "600x200" banner images that caused them not to be saved correctly
|
|
|
|
v0.1.0
|
|
------
|
|
|
|
- Add support for new GraphQL twitter API (dubbed "APIv2" in code). User feed scraping now uses this API
|
|
- Embedded links (URLs) now have their short versions saved in the DB.
|
|
|
|
v0.1.1
|
|
------
|
|
|
|
- BUGFIX: fix selection of primary "TimelineAddEntries" field in APIv2 following breaking API change
|
|
- Add `-h, --help` command line flags to display help message and gracefully exit
|
|
|
|
v0.1.2
|
|
------
|
|
|
|
- BUGFIX: fix error when trying to match mixed-case user handles in tombstone tweets
|
|
- BUGFIX: set "Client Language" header to "English" in all cases
|
|
- Add an option to show debug logging; hide it by default
|
|
|
|
v0.1.3
|
|
------
|
|
|
|
- Add following users
|
|
- Add `list_followed` subcommand
|
|
- Add support for tweets that quote-tweet something from a user who deleted their account
|
|
|
|
v0.1.4
|
|
------
|
|
|
|
- BUGFIXES: Fixed some annoying bugs caused by Twitter's poor API
|
|
- ignore empty entries in a user feed instead of crashing
|
|
- quote tweets no longer include their quoted tweet as an embedded link
|
|
|
|
v0.1.5
|
|
------
|
|
|
|
- BUGFIX: users changing their profile images should be not broken now
|
|
- Major refactoring (testing, massive linting improvements, proper Go dependencies like sqlx, etc)
|
|
- Improve tombstone handling
|
|
- Update tweet content when a tombstone becomes available
|
|
- Update tombstone type of a tweet if it changes
|
|
- If a tweet gets hidden / deleted, it can become a tombstone without losing its content
|
|
- Enable support for users that change their handles
|
|
|
|
v0.1.6
|
|
------
|
|
|
|
- Add support for `mobile.twitter.com` tweet URLs
|
|
- Add basic support for Spaces (i.e., can now basically detect whether a tweet links to a Space or not)
|
|
|
|
v0.1.7
|
|
------
|
|
|
|
- Add support for multiple videos as well as videos and images in the same tweet
|
|
- Restructure media file storage into sub-directories to improve performance when retrieving them
|
|
|
|
v0.1.8
|
|
------
|
|
|
|
- Add proper support for fetching Spaces (with full detail)
|
|
- Add checking for DMCA takedowns of video content
|
|
- Add index for fetching tweet threads (index `in_reply_to_id`)
|
|
- Add indexes to tweet content types to improve fetching performance (index `tweet_id`)
|
|
- BUGFIX: can now fetch retweets with "visibility results"
|
|
- BUGFIX: search changed how cursors work at end-of-feed
|
|
|
|
v0.2.0
|
|
------
|
|
|
|
- Add index on tweets user_id column to accelerate loading of user feeds
|
|
- Implement login and authenticated requests!
|
|
|
|
v0.2.1
|
|
------
|
|
|
|
- BUGFIX: fix cursors being broken in authenticated sessions
|
|
- BUGFIX: fix stats for likes, RTs, etc. getting erased if a tweet gets deleted or hidden
|
|
- BUGFIX: `fetch_tweet` and `search` subcommands no longer ignore the `-n [number]` flag
|
|
- ".session" extension for the `--session` flag is now optional, allowing tab-autocompletion
|
|
|
|
v0.2.2
|
|
------
|
|
|
|
- Add support for Expandable (i.e., "Show more") tweets
|
|
- Fix an issue with half the tweets on user feeds not getting fetched while logged in
|
|
|
|
v0.3.0
|
|
------
|
|
|
|
- Now use APIv2 for Tweet Detail pages as well as user feeds.
|
|
- BUGFIX: fix a lot of bugs that were unfixable using the old API
|
|
- BUGFIX: fix spaces not updating ended_at time
|
|
|
|
v0.3.1
|
|
------
|
|
|
|
- HOTFIX: it looks like Twitter changed the label for a timeline module from "homeConversation" to "profile-conversation"!
|
|
|
|
v0.3.2
|
|
------
|
|
|
|
- BUGFIX: fix duplicate space participants bug (participants in a space should be unique)
|
|
- Add fetching of a user's Likes tab
|
|
- Implement "liking" and "unliking"
|
|
|
|
v0.4.0
|
|
------
|
|
|
|
- Add Web front-end served by `twitter webserver`
|
|
- Some small improvements to scraping of Spaces
|
|
|
|
v0.4.1
|
|
------
|
|
|
|
- BUGFIX: fix Spaces showing times 50,000 years in the future (timestamps in millis rather than seconds)
|
|
- Few UI improvements
|
|
- fix link preview images overflowing the feed column
|
|
- removed gray thread-connector line showing up in QTs
|
|
- Make the search bar work for searches
|
|
- Add scraper request for HomeTimeline
|
|
|
|
v0.4.2
|
|
------
|
|
|
|
- BUGFIX: fix long tweets displaying with paragraphs in wrong order
|
|
- BUGFIX: fix scrolling issues when using "back" button / history
|
|
- Search bar improvements:
|
|
- can now paste twitter.com / mobile.twitter.com links into the search bar
|
|
- go directly to user feed by entering "@elonmusk" (for example)
|
|
- UI improvements:
|
|
- Make entities (#hashtags, @mentions) clickable in tweets and user bios
|
|
- Make tombstones show
|
|
- Other misc improvements
|
|
- Add manual re-scrape button to user feeds
|
|
- Add automatic timeline scraping in background when logged in
|
|
|
|
v0.5.0
|
|
------
|
|
|
|
- Add support for user "likes"
|
|
- Logged-in user's "liked" tweets show filled-in heart in the UI
|
|
- Can "like" (and "unlike") a tweet by clicking on its heart icon
|
|
- UI improvements:
|
|
- Show entire threads in UI
|
|
- Add tabs to User Feed: "Tweets", "Tweets with replies", "Media", "Likes"
|
|
- Search now supports sort orders: most recent, least recent, most likes, most retweets
|
|
- Add support for a number of upstream stuff: "x.com" links, "Tweets" renamed to "Posts", new API formats, edited tweets
|
|
- Bugfixes
|
|
|
|
v0.5.1
|
|
------
|
|
|
|
- Add a preliminary Lists page which shows all offline-follows
|
|
- Add search page support for searching users
|
|
- Automatic background scraping of logged-in user's Likes
|
|
|
|
v0.6.0
|
|
------
|
|
|
|
- Add DMs
|
|
- Bugfixes:
|
|
- Fix crashing because of the composer button when scraping logged-in user's threads
|
|
- Fix crashing when loading the user feed of a user who isn't scraped yet
|
|
- UI improvements:
|
|
- Make page loads much faster by moving image/video downloading to non-blocking background tasks
|
|
- Get rid of dropdown button on tweets (buttons are just inline now)
|
|
|
|
v0.6.1
|
|
------
|
|
|
|
- UI improvements:
|
|
- add the `--auto-open` flag for `twitter webserver` which opens the homepage in a browser on launch
|
|
- search bar preserves its text between searches
|
|
- spinner now shows when loading / fetching tweets
|
|
- on errors, an error toast notification now shows
|
|
- Add followers/followees pages and queries
|
|
|
|
v0.6.2
|
|
------
|
|
|
|
- make all timestamps across the app use milliseconds (previously used both seconds or milliseconds in different places)
|
|
- add `get_user_likes_all` subcommand
|
|
- UI improvements:
|
|
- make end-of-feed indicator look better
|
|
- add favicon
|
|
- Bugfixes:
|
|
- fix clobbering of interaction stats if a tweet is deleted (previously they'd all get set to 0)
|
|
- fix double HTMX spinner on tweets with quoted tweet
|
|
- fix focused tweets being clickable
|
|
|
|
v0.6.3
|
|
------
|
|
|
|
- Add Lists
|
|
- Add `--default-profile` flag which will use and create a profile in a default location (based on OS)
|
|
- UI improvements:
|
|
- make scrolling in DMs work way better
|
|
- hovering over a tweet now highlights it with a light background color
|
|
- Create a `Twitter.desktop` file in dpkg build
|
|
|
|
v0.6.4
|
|
------
|
|
|
|
- Can now re-scrape search results from the web UI
|
|
- UI improvements:
|
|
- feed "Show More" button looks nicer
|
|
- online-login form now has a HTMX spinner
|
|
- Login flow improved:
|
|
- now auto-scrapes timeline and list of people you follow after logging in
|
|
- command-line supports 2FA challenges
|
|
- web UI notifies login failure if challenged for 2FA instead of failing silently
|
|
- Home page now has a "User Feed" timeline (shows people logged-in user follows) in addition to the "Offline Timeline"
|
|
|
|
v0.6.5
|
|
------
|
|
|
|
- UI improvements:
|
|
- Add image full-size on-click viewer
|
|
- Add images, videos and links to DMs
|
|
|
|
v0.6.6
|
|
------
|
|
|
|
- BUGFIX: fix server crashing if a media download HTTP request times out
|
|
- BUGFIX: scraping user feeds now gets long tweets properly
|
|
- UI improvements:
|
|
- Fix image sizing issues in link preview images and DM images
|
|
- Add support for pinned tweets (both fetching and displaying)
|
|
- New search bar filters: "tombstone:<tombstone_type>" and "followed_by:<userhandle>"
|
|
- Add boilerplate for Progressive Web App
|
|
|
|
v0.6.7
|
|
------
|
|
|
|
- UI improvements:
|
|
- various bugfixes
|
|
- make tweets take up less space
|
|
- Add "list:<id>" search bar filter
|
|
- BUGFIX: Fix pinned tweets with a quoted-tweet not showing the quoted tweet
|
|
|
|
v0.6.8
|
|
------
|
|
|
|
- UI improvements:
|
|
- fix some buggy styling around the pin icon on pinned tweets and error toast messages
|
|
- BUGFIX: fix scraper crashing when a media download times out (e.g., long video)
|
|
|
|
v0.6.9
|
|
------
|
|
|
|
- UI improvements:
|
|
- GIFs now show differently than videos; can be paused and restarted on click
|
|
- Add unread indicators to DMs, and show unread messages count in nav sidebar
|
|
- DM messages now have a message preview if the last message was just an attachment (image, tweet, or video)
|
|
|
|
v0.6.10
|
|
-------
|
|
|
|
- UI improvements:
|
|
- add "Copy Link" button to tweets
|
|
- add "Go Online" button to nav sidebar
|
|
- add header section for DMs with open-in-external button
|
|
- Clicking on the "replying-to" button on a DM scrolls that message into view
|
|
- Add DM pagination / show-more button
|
|
|
|
v0.6.11
|
|
-------
|
|
|
|
- Add 2 buttons in DM chat header: mark as "read", manually re-scrape
|
|
- DM list now updates itself live while on the Messages page
|
|
|
|
v0.6.12
|
|
-------
|
|
|
|
- Add support for scraping bookmarks (not making them, yet)
|
|
- Publish statically linked builds only
|
|
- Various minor bugfixes
|
|
|
|
v0.6.13
|
|
-------
|
|
|
|
- BUGFIX: no longer throws away all scrape results if a pagination request gets rate-limited
|
|
- BUGFIX: can now start the webserver without an internet connection
|
|
- UI improvements:
|
|
- image carousel now closes if you click outside of it
|
|
|
|
v0.6.14
|
|
-------
|
|
|
|
- Add sending replies and emoji reaccs for DMs
|
|
- Bookmarks are now scrapable from the web UI
|
|
- BUGFIX: don't crash when an image or video download 404s
|
|
|
|
v0.6.15
|
|
-------
|
|
|
|
- BUGFIX: `webserver` subcommand now respects the `--session` flag
|
|
- More types of scraping errors now show toasts in the UI: rate limiting, expired session
|
|
- New `--delay` flag added to introduce a delay in paginated scrapes
|
|
|
|
v0.6.16
|
|
-------
|
|
|
|
- Add notifications
|
|
|
|
v0.6.17
|
|
-------
|
|
|
|
- BUGFIX: fix a series of bugs relating to inactive users and duplicate user handles
|
|
|
|
v0.6.18
|
|
-------
|
|
|
|
- BUGFIX: fix scraper crashing on "composer" entry in threads
|
|
- BUGFIX: fix "User followed you" notifications sometimes parsing as "New recommended post"
|
|
- BUGFIX: emojis with skin tones now work in DMs
|
|
- UI improvements:
|
|
- Notifications page is now paginated in web UI
|
|
|
|
v0.6.19
|
|
-------
|
|
|
|
- BUGFIX: fix the scraper `GetUser` function which was broken by an upstream change
|
|
- UI improvements:
|
|
- Profile and banner images in user feed header can now be clicked to open full-size versions
|
|
- Reply previews in DMs now include the username who's being replied to
|
|
- Notifications can now be marked as "read" via command line and web UI
|
|
|
|
v0.6.20
|
|
-------
|
|
|
|
- BUGFIX: fix sending a reacc causing an error if it's not on the latest message in the chat
|
|
- BUGFIX: various fixes around error toasts
|
|
- UI improvements:
|
|
- reaccs: add a little circle around the logged-in user's reaccs, to indicate them better
|
|
- "refresh chat room" button now puts a spinner on the chat while it's refreshing
|
|
|
|
v0.6.21
|
|
-------
|
|
|
|
- UI improvements:
|
|
- on loading a "misshapen" chat image, it will try to reload it a few times
|
|
- Windows:
|
|
- Start menu shortcut should work now
|
|
- "Install for current user only" should now create desktop shortcut properly
|
|
|
|
v0.6.22
|
|
-------
|
|
|
|
- Add background scraping jobs for Bookmarks and logged-in user's User Feed
|
|
- Manually rescraping a User Feed now re-downloads user's profile and banner images as well
|
|
|
|
v0.7.0
|
|
------
|
|
|
|
- PKG: Type definitions are now in `pkg/persistence` instead of `pkg/scraper`
|
|
- BUGFIX: fix notifications page not having user's "likes" filled out on tweets
|
|
- Compound notifications now tell you how many tweets were "liked", how many users followed you, etc
|
|
|
|
v0.7.1
|
|
------
|
|
|
|
- BUGFIX: fix banned users getting their user info (followers count, profile image, display name) getting erased
|
|
- PKG: webserver is now a public package
|