21581b325a
Add support for parsing Tweet Detail in APIv2, including its unusual cursor format and conversation threads
2023-06-08 23:19:50 -03:00
6192c37999
FIXUP: add test file for previous commit
2023-06-08 18:53:50 -03:00
eb566c0612
FIXUP: add test file for previous commit
2023-06-08 18:10:55 -03:00
6830be3c87
Fix error where retweets were not parsing properly sometimes
2022-11-25 15:51:21 -05:00
a81d0e80fe
Add test file
2022-11-24 19:02:50 -05:00
2a45818468
Add parsing of Space object
2022-11-13 13:40:58 -05:00
6de2d670e8
Fix lint error
2022-10-15 12:47:41 -04:00
d647307097
Support multiple videos / videos+images
2022-10-15 12:40:40 -04:00
e932dd33d5
Add parsing Spaces for API v2
2022-05-14 17:36:47 -07:00
54857f40cd
Add parsing of tweets with Space links in them
2022-05-14 15:02:58 -07:00
565b7eaf1e
Add Spaces type
2022-05-14 14:35:49 -07:00
7edc8ad5d3
Some whitespace changes :V
2022-03-06 17:07:05 -08:00
872fc4addf
Add test file for previous commit
2022-03-02 14:37:37 -08:00
aa961b9ff4
Fix handling of empty entries in the feed
...
- e.g., retweets where the author then deleted the original tweet
- I think this is a bug in the Twitter API (it doesn't make sense to return this data) but we have to handle it anyway
2022-03-01 11:51:34 -08:00
3c2aee5016
Add test file for previous commit
2022-02-28 13:50:19 -08:00
52370a4f9d
Recognized deleted users when scraping, create a user with a fake ID
2022-02-27 23:06:06 -08:00
2773214855
Add test file for previous commit
2022-02-04 17:34:15 -08:00
8d406e1b11
Minify that test file
2022-02-02 15:21:42 -08:00
359c166363
Add test file for previous commit
2022-02-02 14:53:10 -08:00
3ca67418e0
Add test file for previous commit
2022-02-02 12:26:02 -08:00
c00ff06b3f
URLs need to have a TweetID lol
2022-02-01 18:21:48 -08:00
9eb3e42539
Add link parsing for APIv2
2022-02-01 16:56:37 -08:00
1bc6aec3b5
Incorporate the new parsing structure into the actual program
2022-02-01 16:04:52 -08:00
12be70f541
Add test files
2022-01-30 17:58:03 -08:00
68a8549f34
Rename api V2 test responses directory
2022-01-30 15:06:07 -08:00
8c79a39c2e
API V2: parse User result
2022-01-30 14:40:17 -08:00
fb421215df
Add parsing of new GraphQL twitter api for user feeds
2022-01-24 20:13:17 -08:00
1a9ba75355
Add checking for banned users
2022-01-06 14:39:31 -05:00
bf2dbede94
Escape HTML entities in tweet text
2022-01-05 21:04:40 -05:00
c15ac9d751
Add persistence for polls
2021-12-12 16:42:32 -08:00
d0cb857acb
Fix embedded link parsing for youtube links with no thumbnails
2021-11-22 14:52:18 -08:00
658f3c338f
Add parsing of twitter polls
2021-11-07 21:10:24 -08:00
c0ae23b4fd
Add 'unavailable' tombstone category
2021-11-01 15:47:52 -07:00
b37bc4bc60
Add test data for tombstones
2021-11-01 14:13:22 -07:00
159084006d
Fix handling of reply-mentions and tweet text normalization
2021-09-27 18:12:28 -07:00
9ae6213025
Clean up the tests some more
2021-09-27 18:08:14 -07:00
89179f2447
Tidy up the scraper tests a bit
2021-09-27 15:30:05 -07:00
0bb9ff6c6b
Add handling for URL cards with no link preview thumbnail image
2021-09-17 20:50:28 -07:00
9d10fd5942
Fix scraper test file to be minified
2021-09-17 20:12:51 -07:00
497a62050b
Add test file for previous commit
2021-09-17 20:02:33 -07:00
690dd99b8f
Clean up the michael malice feed test file
2021-09-17 18:08:16 -07:00
05c3f2289b
Add persistence for new Url type
2021-09-17 18:04:57 -07:00
355ed7ccf5
Add test file for the previous commit
2021-09-17 17:36:35 -07:00
3f9c807efa
Add Url type
2021-09-17 13:41:43 -07:00
5a0b8b5e36
Check for end-of-feed
2021-08-22 15:01:46 -07:00
8462365d70
Add tests for API video parsing
2021-08-04 23:49:45 -07:00
c803794a0f
Change TweetID to be an int64 instead of a string
2021-08-04 14:49:20 -07:00
494ca25dc4
Change method of scraping images
...
- break `Filename` field on Image into `RemoteURL` and `LocalFilename`
- new parsing function for an APIMedia item
2021-08-03 17:34:44 -07:00
647dd8aa6b
Add scraping of videos from tweets
2021-07-25 14:51:17 -07:00
6c111ab976
BUGFIX: fix incorrect text normalization on tweet with multiple images
2021-07-23 19:14:16 -07:00
