This website requires JavaScript.
cb3fc469b2
Fix lint error
Alessio
2022-05-07 14:38:21 -07:00
36e537b1cd
Handle IsFollowing calls for users that aren't saved yet
Alessio
2022-05-07 14:35:53 -07:00
6498a97816
Change IsFollowing
method to use id
field instead of handle
Alessio
2022-05-07 14:21:59 -07:00
7e33cc1a9f
Update test script because previously deleted account un-deleted itself somehow
Alessio
2022-03-22 13:28:42 -07:00
da9a5d4de8
Modify some comments, use 'complete-build' command in sourcehut build where appropriate
Alessio
2022-03-22 13:12:21 -07:00
d1d80a91cd
Add 'gofmt' linter
Alessio
2022-03-13 17:09:43 -07:00
223734d001
Remove all references to deprecated 'ioutil' package
Alessio
2022-03-13 16:33:17 -07:00
f08da27cc2
Enable 'wrapcheck' linter
Alessio
2022-03-13 16:13:16 -07:00
8261087103
REFACTOR: make helper function use the scraper tweet-url parsing method
Alessio
2022-03-13 14:50:08 -07:00
8c2fb965aa
Fix integration test (failing because user Denlesks was marked as "sensitive")
Alessio
2022-03-13 14:46:44 -07:00
482f3ae127
Enable the nolintlint linter
Alessio
2022-03-13 13:05:37 -07:00
f41d072573
Improve error handling some more
Alessio
2022-03-06 20:31:04 -08:00
d77c55ec1c
Fix lint errors
Alessio
2022-03-06 19:29:48 -08:00
15d7cd77a1
Do slightly more proper error handling
Alessio
2022-03-06 19:27:30 -08:00
2f784c779c
REFACTOR: Add sqlx
dependency
Alessio
2022-03-06 19:17:43 -08:00
40e585e1a6
Fix blunder
Alessio
2022-03-06 18:17:00 -08:00
78a5ffbf92
Fix another over-eager commit
Alessio
2022-03-06 18:14:06 -08:00
26ab9ec937
Create new Timestamp helper type
Alessio
2022-03-06 18:09:43 -08:00
7edc8ad5d3
Some whitespace changes :V
Alessio
2022-03-06 17:07:05 -08:00
1d990e8a40
Remove users.yaml file
Alessio
2022-03-06 16:11:45 -08:00
1582dfe44e
Fix some lint problems
Alessio
2022-03-06 15:36:07 -08:00
06167eef18
Fix over-enthusiastic commit
Alessio
2022-03-06 15:26:27 -08:00
18c9f79429
REFACTOR: LINT: add .golangci.yaml lint config file
Alessio
2022-03-06 15:06:06 -08:00
524060a79a
Fix a manual git-editing typo
Alessio
2022-03-06 13:45:24 -08:00
5fb6a4aa9a
REFACTOR: update persistence tests to use assert and require
Alessio
2022-03-06 13:30:17 -08:00
3e6b8acdda
Update CHANGELOG.txt for v0.1.4
Alessio
2022-03-02 16:15:12 -08:00
68d4062820
Add a data migration to remove URLs that just link to a quoted tweet
Alessio
2022-03-02 16:09:41 -08:00
3cb3c14e77
Change failing integration test to use different tweet with link preview image (previous one had bad karma anyway)
Alessio
2022-03-02 15:34:38 -08:00
872fc4addf
Add test file for previous commit
Alessio
2022-03-02 14:37:37 -08:00
f22f15f3d9
Prevent quoted-tweet URLs from being added as URLs (they should just be quote-tweets)
Alessio
2022-03-02 14:34:42 -08:00
aa961b9ff4
Fix handling of empty entries in the feed
Alessio
2022-03-01 11:51:34 -08:00
18eafe6e3d
Update CHANGELOG.txt for v0.1.3
Alessio
2022-02-28 16:16:36 -08:00
508670248e
Add list_followed
subcommand
Alessio
2022-02-28 16:06:58 -08:00
3c2aee5016
Add test file for previous commit
Alessio
2022-02-28 13:50:19 -08:00
0909efb18b
Add support for TweetWithVisibilityResults API type, which has a different JSON structure for some reason
Alessio
2022-02-28 13:44:42 -08:00
648098352b
Fix scraper treating NotFoundError as a bug requiring retries
Alessio
2022-02-27 23:35:44 -08:00
5f3013a49b
Update tweet UserIDs in a tweet trove with Unknown Users in it (i.e., deleted users(?))
Alessio
2022-02-27 23:14:30 -08:00
52370a4f9d
Recognized deleted users when scraping, create a user with a fake ID
Alessio
2022-02-27 23:05:37 -08:00
728aaf251f
Fix linting error
Alessio
2022-02-26 22:18:41 -08:00
772fa247f7
Add creation of unknown users with known handles (i.e., deleted accounts)
Alessio
2022-02-26 22:09:27 -08:00
41586bf0f8
Add follow
and unfollow
sub-commands
Alessio
2022-02-26 16:22:00 -08:00
857b9c5255
Fix linting error
Alessio
2022-02-26 16:06:51 -08:00
373b9ce16e
Add IsFollowed field on Users
Alessio
2022-02-26 15:58:30 -08:00
f9eb986aed
Update CHANGELOG.txt for v0.1.2
Alessio
2022-02-16 09:29:11 -08:00
19c4b14704
Use a more lint-approved case-insensitive comparator
Alessio
2022-02-14 18:06:04 -08:00
82f5721ac2
Make user handle matching case-insensitive in tweet trove
Alessio
2022-02-14 17:58:10 -08:00
8a70f55e1b
Update some more stuff to match API changes
Alessio
2022-02-14 17:15:01 -08:00
9cd7f4171a
Add proper logging module, set logging level to 'info' by default (hide very verbose debug results)
Alessio
2022-02-14 13:54:18 -08:00
f9a8841c07
REFACTOR: create SaveTweetTrove
convenience method to reduce code in main.go
Alessio
2022-02-14 13:39:59 -08:00
3d489a8e95
REFACTOR: eliminate 'download-if-needed' functions, add the 'if-needed' check to the regular downloading functions
Alessio
2022-02-14 13:10:12 -08:00
f1a33bab48
Fix cmd help output when entering an invalid command-line flag
Alessio
2022-02-14 13:00:32 -08:00
e0f05cbeea
Refactor all scraper areas to use TweetTroves where applicable
Alessio
2022-02-12 20:39:30 -08:00
c53d8ae24c
Remove old, unneeded debugging files
Alessio
2022-02-12 20:25:18 -08:00
a204d66a44
Update CHANGELOG.txt for v0.1.1
Alessio
2022-02-12 16:04:54 -08:00
4b2df1c628
Fix wrong 'Instruction' item being chosen following new API update :V
Alessio
2022-02-12 15:51:55 -08:00
13170c84b2
Add '-h, --help' flag to print help message then exit
Alessio
2022-02-12 15:41:03 -08:00
2773214855
Add test file for previous commit
Alessio
2022-02-04 17:34:15 -08:00
09b8685f19
Add polls for APIv2
Alessio
2022-02-04 17:05:27 -08:00
5e446f73a6
Update CHANGELOG.txt for v0.1.0
Alessio
2022-02-02 15:30:01 -08:00
8d406e1b11
Minify that test file
Alessio
2022-02-02 15:21:42 -08:00
04a4974ffb
Add '-n' / '--number' flags, and update help message for executable
Alessio
2022-02-02 15:09:50 -08:00
359c166363
Add test file for previous commit
Alessio
2022-02-02 14:53:10 -08:00
8892785aad
Add fetching of tombstoned users for APIv2
Alessio
2022-02-02 14:49:46 -08:00
1e1e97b5ca
Add TweetTrove#FillMissingUserIDs helper method
Alessio
2022-02-02 14:43:17 -08:00
3ca67418e0
Add test file for previous commit
Alessio
2022-02-02 12:26:02 -08:00
df1a7d2388
Add recognizing bottom of feed
Alessio
2022-02-02 12:10:24 -08:00
c00ff06b3f
URLs need to have a TweetID lol
Alessio
2022-02-01 18:21:48 -08:00
32e1558800
Fix link parsing for retweets
Alessio
2022-02-01 17:36:23 -08:00
9eb3e42539
Add link parsing for APIv2
Alessio
2022-02-01 16:56:37 -08:00
1bc6aec3b5
Incorporate the new parsing structure into the actual program
Alessio
2022-02-01 16:04:52 -08:00
b2199e9b59
Add Url#ShortText field
Alessio
2022-02-01 15:48:43 -08:00
069ab83fba
Refactor scraper tests
Alessio
2022-01-31 19:14:14 -08:00
babbdecf6c
Fix linting error
Alessio
2022-01-30 18:03:09 -08:00
12be70f541
Add test files
Alessio
2022-01-30 17:58:03 -08:00
c085a579d9
Add TombstoneUsers to TweetTrove
Alessio
2022-01-30 17:55:18 -08:00
4708ffc3c9
Intermediate results for refactoring :V
Alessio
2022-01-30 17:48:03 -08:00
f0b1acd595
Fix an error
Alessio
2022-01-30 15:44:26 -08:00
3046dd56f4
Add TweetTrove#MergeWith method
Alessio
2022-01-30 15:31:06 -08:00
f264aff265
Update test data since the previous private-user unprivated
Alessio
2022-01-30 15:16:04 -08:00
68a8549f34
Rename api V2 test responses directory
Alessio
2022-01-30 15:06:07 -08:00
4ccd6795d5
Fix import error
Alessio
2022-01-30 14:51:14 -08:00
7532c3ba81
Add testify dependency
Alessio
2022-01-30 14:48:16 -08:00
8c79a39c2e
API V2: parse User result
Alessio
2022-01-30 14:40:17 -08:00
b261f095ad
Change main application to use new API for user feeds
Alessio
2022-01-24 20:19:08 -08:00
fb421215df
Add parsing of new GraphQL twitter api for user feeds
Alessio
2022-01-24 20:13:17 -08:00
8250725d2c
Update CHANGELOG.txt for v0.0.5
Alessio
2022-01-19 17:20:52 -08:00
56870173e9
Fix bug with users with 600x200 banner images
Alessio
2022-01-18 17:48:08 -08:00
dc24ce5ad4
Fix tweet URL parsing for urls with the stupid '?s=20' etc
Alessio
2022-01-10 11:41:03 -05:00
25fddaef9d
Update CHANGELOG.txt for v0.0.4
Alessio
2022-01-08 19:50:14 -05:00
d9f9c9de83
Integrate url expander into Tweet URL parser
Alessio
2022-01-08 18:36:23 -05:00
043a7fd66f
Add link expander
Alessio
2022-01-08 18:25:26 -05:00
e71e32228c
Get rid of tiny default profile images (default image is small enough, doesn't need a tiny version)
Alessio
2022-01-07 17:33:26 -05:00
12f1cc6728
Fix some problems associated with banned users
Alessio
2022-01-07 16:23:02 -05:00
c253486580
Fix lint error
Alessio
2022-01-07 13:52:33 -05:00
d95f183e45
Add parsing of tombstoned quoted-tweets
Alessio
2022-01-07 13:42:00 -05:00
6eac56183d
Allow fetching of banned users
Alessio
2022-01-07 13:40:22 -05:00
fb174df363
Add persistence for is_banned field
Alessio
2022-01-07 12:38:56 -05:00
1ea97e669a
Update tiny profile image logic for users with no profile image (including banned users)
Alessio
2022-01-06 15:21:27 -05:00
1a9ba75355
Add checking for banned users
Alessio
2022-01-06 14:39:31 -05:00
b1c7db6540
Add helper method to parse tweet URLs
Alessio
2022-01-06 13:43:22 -05:00