Jaeger Aquila
ce11a70bf9
Finished refactoring
2022-12-10 13:30:16 -05:00
ec6ecb39cb
Remove UpdateQueryCursor helper function as it was inlined
2022-12-10 12:18:52 -05:00
fe57763568
Change entrypoint functions to use NewGuestSession constructor for API
2022-12-10 10:58:32 -05:00
3c9ed97c64
Fix lint error
2022-12-10 10:55:55 -05:00
6c5f8b8356
Update API call functions to use do_http
...
- move query-string modifications to individual API call functions:
"referrer=tweet", tweet "include_XYZ" params
- tidy up implementation of retries in `API.GetUser`
- fix out-param of `do_http` being a pointer to an interface when it should just be an interface
2022-12-10 10:49:25 -05:00
Jaeger Aquila
e44a83edae
fixed trailing whitespace
2022-12-09 23:03:22 -05:00
Jaeger Aquila
6816dcbe2f
generic http implem and fixing package errors
2022-12-09 23:03:22 -05:00
77e338b4f7
Add skeleton for API refactoring
2022-12-09 23:03:22 -05:00
f2af4b2403
Fix search
2022-12-09 23:02:12 -05:00
7bad18e9b1
Tidy up the formatting
2022-12-03 13:47:34 -05:00
James Raleigh
972fd50c5b
Fixed tests for DMCAed videos and other fixes
2022-12-02 20:38:57 -05:00
6830be3c87
Fix error where retweets were not parsing properly sometimes
2022-11-25 15:51:21 -05:00
412574e46f
Fix parsing of spaces in APIv2; improve no-worsening of Spaces
2022-11-25 13:22:20 -05:00
433dd5e0bb
Enable fetching of Space details
2022-11-24 19:15:21 -05:00
a81d0e80fe
Add test file
2022-11-24 19:02:50 -05:00
55c4db32de
Fix lint
2022-11-24 19:01:30 -05:00
0293f11b9b
Fix parsing of empty Space responses
2022-11-24 18:57:42 -05:00
05cacb1490
Add blank line
2022-11-13 13:47:56 -05:00
2a45818468
Add parsing of Space object
2022-11-13 13:40:58 -05:00
d54e77b169
Some housekeeping
2022-11-13 12:07:36 -05:00
5877063748
Add some tests for multiple videos / videos+images
2022-10-15 15:50:25 -04:00
069ddcd976
Put videos, images, and video-thumbnails into subdirectories
2022-10-15 15:06:06 -04:00
6de2d670e8
Fix lint error
2022-10-15 12:47:41 -04:00
d647307097
Support multiple videos / videos+images
2022-10-15 12:40:40 -04:00
b8b3b56c8e
Add woodpecker config, update tests due to an account getting banned
2022-10-14 23:32:14 -04:00
e932dd33d5
Add parsing Spaces for API v2
2022-05-14 17:36:47 -07:00
1b675e8200
Add db operations for Spaces
2022-05-14 16:04:09 -07:00
54857f40cd
Add parsing of tweets with Space links in them
2022-05-14 15:02:58 -07:00
565b7eaf1e
Add Spaces type
2022-05-14 14:35:49 -07:00
d98f02954e
Add support for 'mobile' twitter links
2022-05-14 13:51:51 -07:00
d1d80a91cd
Add 'gofmt' linter
2022-03-13 17:13:40 -07:00
223734d001
Remove all references to deprecated 'ioutil' package
2022-03-13 16:33:17 -07:00
f08da27cc2
Enable 'wrapcheck' linter
2022-03-13 16:13:16 -07:00
482f3ae127
Enable the nolintlint linter
2022-03-13 13:05:37 -07:00
f41d072573
Improve error handling some more
2022-03-06 20:31:04 -08:00
15d7cd77a1
Do slightly more proper error handling
2022-03-06 19:27:30 -08:00
2f784c779c
REFACTOR: Add sqlx dependency
2022-03-06 19:17:43 -08:00
40e585e1a6
Fix blunder
2022-03-06 18:17:00 -08:00
26ab9ec937
Create new Timestamp helper type
2022-03-06 18:09:43 -08:00
7edc8ad5d3
Some whitespace changes :V
2022-03-06 17:07:05 -08:00
1582dfe44e
Fix some lint problems
2022-03-06 15:36:07 -08:00
18c9f79429
REFACTOR: LINT: add .golangci.yaml lint config file
...
- Add linting for line lengths
- Fix some linting errors
2022-03-06 15:06:06 -08:00
872fc4addf
Add test file for previous commit
2022-03-02 14:37:37 -08:00
f22f15f3d9
Prevent quoted-tweet URLs from being added as URLs (they should just be quote-tweets)
2022-03-02 14:34:42 -08:00
aa961b9ff4
Fix handling of empty entries in the feed
...
- e.g., retweets where the author then deleted the original tweet
- I think this is a bug in the Twitter API (it doesn't make sense to return this data) but we have to handle it anyway
2022-03-01 11:51:34 -08:00
508670248e
Add list_followed subcommand
2022-02-28 16:06:58 -08:00
3c2aee5016
Add test file for previous commit
2022-02-28 13:50:19 -08:00
0909efb18b
Add support for TweetWithVisibilityResults API type, which has a different JSON structure for some reason
2022-02-28 13:44:42 -08:00
648098352b
Fix scraper treating NotFoundError as a bug requiring retries
2022-02-27 23:35:44 -08:00
52370a4f9d
Recognized deleted users when scraping, create a user with a fake ID
2022-02-27 23:06:06 -08:00
