me/offline-twitter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Commit Graph

  • f07da7880c Correct a mistake in the CHANGELOG for v0.2.1 Alessio 2023-06-07 16:06:31 -03:00
  • d93033352e Update CHANGELOG.txt for v0.2.1 Alessio 2023-06-07 15:44:25 -03:00
  • b37977145a Expand debug logging to make debugging scraping errors easier Alessio 2023-06-07 14:46:46 -03:00
  • cba6631a72 Add integration test for no-clobbering of tombstone stats Alessio 2023-06-07 13:23:57 -03:00
  • 2a74941707 Fix lint error Alessio 2023-06-05 18:10:02 -03:00
  • c01c927a92 Use the built image for the main pipeline Alessio 2023-06-05 18:06:50 -03:00
  • 04eb65aef3 Mount the docker socket Alessio 2023-06-03 10:57:09 -03:00
  • 599ba042b8 Create separate build file for the docker image Alessio 2023-06-03 10:53:17 -03:00
  • 6df313558d Fix lint errors Alessio 2023-06-03 09:17:12 -03:00
  • 815a8180da BUGFIX: don't clobber num_likes, num_retweets etc. if tweet is a stub Alessio 2023-06-03 08:30:49 -03:00
  • dea37c7556 Fix cursor bug in authenticated sessions Alessio 2023-06-03 07:51:39 -03:00
  • af93f44ed2 Two bug fixes Alessio 2023-06-03 07:35:09 -03:00
  • a724f32470 Allow specifying session files with .session extension at command line Alessio 2023-03-16 17:50:36 -03:00
  • 6f2ef8953b Update CHANGELOG.txt for v0.2.0 Alessio 2023-03-09 22:55:02 -05:00
  • 1384aa73e9 BUGFIX: fix CSRF tokens getting out of sync on requests that load multiple pages of results Alessio 2023-03-09 22:49:55 -05:00
  • ae9a9c3a47 Remove password from login.py as well Alessio 2023-02-17 13:51:31 -05:00
  • f988d57e1c Remove password from test script Alessio 2023-02-17 13:46:42 -05:00
  • ee4b863d38 Remove integration test masquerading as a unit test Alessio 2023-02-17 13:22:09 -05:00
  • c09b77f497 Add ability to use saved sessions Alessio 2023-02-17 13:07:12 -05:00
  • 933ee9a7f9 Fix lint errors Alessio 2023-02-05 23:40:22 -05:00
  • 3a44f39e9f Add a check alex 2023-02-05 23:33:36 -05:00
  • 28ca0ac4eb added cmdline login functionality Jaeger Aquila 2023-02-05 20:29:57 -05:00
  • 68867718af Fixed linter errors Jaeger Aquila 2023-01-14 17:56:02 -05:00
  • a8b1241898 removing extra TODO Jaeger Aquila 2023-01-14 17:51:54 -05:00
  • 4ee69f1ed5 implemented saving cookies and user session to a file Jaeger Aquila 2023-01-14 17:40:03 -05:00
  • ab9e67bbb1 Add skeleton for saving and loading sessions Alessio 2023-01-08 15:40:17 -05:00
  • 390c831541 Add marshaling and unmarshaling of API objects so state can be persisted between runs Alessio 2023-01-01 21:57:08 -05:00
  • c3494cfe19 Add handle of the logged-in user to API Alessio 2023-01-01 21:45:00 -05:00
  • 6302b3d5ab Fix integration test (pompilivs deleted his account) Alessio 2023-01-01 20:27:35 -05:00
  • 00a73836f5 Fix some lint errors Alessio 2023-01-01 20:12:54 -05:00
  • 8a1b2fc801 Fix lint error Alessio 2022-12-26 15:12:53 -05:00
  • 04842f8c72 REFACTOR: make login flow less verbose Alessio 2022-12-26 14:01:58 -05:00
  • 1087809d63 Add fetching likes API request Alessio 2022-12-22 23:12:11 -05:00
  • c0366a0978 Add cookies and CSRF to authentication flow Alessio 2022-12-22 23:09:46 -05:00
  • 3feaf9caa6 Fix cookies in login python script Alessio 2022-12-22 23:06:31 -05:00
  • 2cf64db93e some refactoring and tests Jaeger Aquila 2022-12-20 19:05:08 -05:00
  • 140024e903 initial login code Jaeger Aquila 2022-12-10 14:33:27 -05:00
  • 58f4026e88 Rename login script Alessio 2022-12-03 17:11:56 -05:00
  • 62c499e95c Tidy up login script, add request to fetch 'likes' (auth-guarded endpoint) Alessio 2022-12-03 17:07:48 -05:00
  • 55b25f4fbf initial login test script in py James Raleigh 2022-12-03 16:01:15 -05:00
  • 1558c321b6 Add index on tweets user_id column to accelerate building user feed Alessio 2022-12-26 13:08:25 -05:00
  • 83a6edb257 Fix migrations to not have nulls Alessio 2022-12-26 12:56:05 -05:00
  • 1570075443 Add new spaces columns to the migrations Alessio 2022-12-25 13:06:17 -05:00
  • 7f9cc62667 Update CHANGELOG.txt for v0.1.8 Alessio 2022-12-24 14:39:04 -05:00
  • 476f00856c Add indexes for in_reply_to_id, and fetching urls/polls/images/videos by tweet_id Alessio 2022-12-24 13:47:43 -05:00
  • dac10fa13c Tighten up no-worsening of spaces, add saving ShortUrl when scraping from feed Alessio 2022-12-23 00:00:13 -05:00
  • 1a9a2cb745 Fix Spaces created-by-id not being scraped Alessio 2022-12-22 23:29:11 -05:00
  • ce11a70bf9 Finished refactoring Jaeger Aquila 2022-12-10 13:30:16 -05:00
  • ec6ecb39cb Remove UpdateQueryCursor helper function as it was inlined Alessio 2022-12-10 12:18:52 -05:00
  • fe57763568 Change entrypoint functions to use NewGuestSession constructor for API Alessio 2022-12-10 10:58:32 -05:00
  • 3c9ed97c64 Fix lint error Alessio 2022-12-10 10:55:55 -05:00
  • 6c5f8b8356 Update API call functions to use do_http Alessio 2022-12-10 10:34:36 -05:00
  • e44a83edae fixed trailing whitespace Jaeger Aquila 2022-12-08 21:52:08 -05:00
  • 6816dcbe2f generic http implem and fixing package errors Jaeger Aquila 2022-12-08 21:44:02 -05:00
  • 77e338b4f7 Add skeleton for API refactoring Alessio 2022-12-05 23:13:34 -05:00
  • f2af4b2403 Fix search Alessio 2022-12-09 23:01:47 -05:00
  • 7bad18e9b1 Tidy up the formatting Alessio 2022-12-03 13:47:34 -05:00
  • 972fd50c5b Fixed tests for DMCAed videos and other fixes James Raleigh 2022-12-02 20:33:54 -05:00
  • a05d97dd71 Updated todo text Jaeger Aquila 2022-11-27 23:20:25 +00:00
  • bacfd519cc fix for dmca video downloads James Raleigh 2022-11-27 18:10:25 -05:00
  • 6830be3c87 Fix error where retweets were not parsing properly sometimes Alessio 2022-11-25 15:51:21 -05:00
  • 37525e58c0 Fix title no-worsening for spaces Alessio 2022-11-25 13:32:22 -05:00
  • 412574e46f Fix parsing of spaces in APIv2; improve no-worsening of Spaces Alessio 2022-11-25 13:22:20 -05:00
  • 433dd5e0bb Enable fetching of Space details Alessio 2022-11-24 19:15:21 -05:00
  • 58c19df11a Add Space persistence queries for Space details Alessio 2022-11-24 19:08:00 -05:00
  • a81d0e80fe Add test file Alessio 2022-11-24 19:02:50 -05:00
  • 55c4db32de Fix lint Alessio 2022-11-24 19:01:30 -05:00
  • 0293f11b9b Fix parsing of empty Space responses Alessio 2022-11-24 18:57:42 -05:00
  • 05cacb1490 Add blank line Alessio 2022-11-13 13:47:56 -05:00
  • 2a45818468 Add parsing of Space object Alessio 2022-11-13 13:17:47 -05:00
  • d54e77b169 Some housekeeping Alessio 2022-11-13 12:04:25 -05:00
  • b78cef34ce Update tests to change account that got banned Alessio 2022-11-13 11:29:47 -05:00
  • 877cdd316b Skip building the binary an extra time when building the dpkg Alessio 2022-10-15 16:57:18 -04:00
  • 0bf73d4bc5 Update CHANGELOG.txt for version 0.1.7 Alessio 2022-10-15 16:52:45 -04:00
  • fe305ec2ab Fix typo Alessio 2022-10-15 16:38:07 -04:00
  • 8ccb55ef79 Add version bump test to woodpecker build Alessio 2022-10-15 16:35:14 -04:00
  • 5877063748 Add some tests for multiple videos / videos+images Alessio 2022-10-15 15:50:25 -04:00
  • 4b5b098b9e Fix error in Curl function where parent directory already exists Alessio 2022-10-15 15:17:15 -04:00
  • 069ddcd976 Put videos, images, and video-thumbnails into subdirectories Alessio 2022-10-15 15:06:06 -04:00
  • 6de2d670e8 Fix lint error Alessio 2022-10-15 12:47:41 -04:00
  • d647307097 Support multiple videos / videos+images Alessio 2022-10-15 12:40:40 -04:00
  • b8b3b56c8e Add woodpecker config, update tests due to an account getting banned Alessio 2022-10-14 22:44:02 -04:00
  • 94b8b551f8 Update CHANGELOG.txt for v0.1.6 Alessio 2022-05-14 18:28:33 -07:00
  • 5d54a46aaf Move cmd help message to a plain text file and embed it using go:embed Alessio 2022-05-14 18:18:05 -07:00
  • e932dd33d5 Add parsing Spaces for API v2 Alessio 2022-05-14 17:36:47 -07:00
  • c97ee8a645 Add integration test for scraping a Space Alessio 2022-05-14 17:00:56 -07:00
  • 99ad0f4a2c Fix lint errors Alessio 2022-05-14 16:43:39 -07:00
  • cf7608eaa9 Save links to Spaces in a Tweet Alessio 2022-05-14 16:36:03 -07:00
  • 3b6c970b74 Make Space test fixture semi-stable based on given ID Alessio 2022-05-14 16:31:53 -07:00
  • 1b675e8200 Add db operations for Spaces Alessio 2022-05-14 16:04:09 -07:00
  • 54857f40cd Add parsing of tweets with Space links in them Alessio 2022-05-14 15:02:15 -07:00
  • 565b7eaf1e Add Spaces type Alessio 2022-05-14 14:34:36 -07:00
  • d98f02954e Add support for 'mobile' twitter links Alessio 2022-05-14 13:51:51 -07:00
  • 8b81dafd28 Update CHANGELOG.txt for v0.1.5 Alessio 2022-05-07 19:26:02 -07:00
  • a39fab14b3 Fix another lint error Alessio 2022-05-07 19:02:02 -07:00
  • 876f617876 Fix lint errors Alessio 2022-05-07 18:50:28 -07:00
  • 3d289ed5f0 Profile images are now downloaded based on whether the local file path exists, not just what the DB says Alessio 2022-05-07 18:46:07 -07:00
  • 8775f5337d Add test to ensure proper user images are being downloaded Alessio 2022-05-07 18:44:03 -07:00
  • 3b4b3fceb9 Improve tombstone handling significantly Alessio 2022-05-07 16:47:46 -07:00
  • 780577d724 Enable updating user handles Alessio 2022-05-07 15:23:37 -07:00