feat: scrape HN favorites as a Kubernetes CronJob and write to DynamoDB
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
commit
35e9c73cf2
18 changed files with 5353 additions and 0 deletions
18
README.md
Normal file
18
README.md
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
# hackernews-favs-cron
|
||||
|
||||
Runs the Hacker News favorites scrape from `nholtware/hackernews-favs` (the AWS
|
||||
Lambda version) as a Kubernetes CronJob on the homelab k3s cluster. It still
|
||||
writes each batch to the AWS DynamoDB table `hackernews-favs` (same item shape:
|
||||
`pk=date#YYYY-MM-DD`, `sk=batch#<uuid>`), so existing readers are unaffected.
|
||||
|
||||
- Code: `src/favorites.ts` (fetch + parse), `src/main.ts` (one-shot entrypoint).
|
||||
- Deploy: push to `main` -> Forgejo CI builds the image -> `gitops-bump` ->
|
||||
ArgoCD syncs `manifests/hackernews-favs-cron` (CronJob, Sundays 02:00 UTC).
|
||||
- Config env: `TABLE_NAME`, `HN_USER` (default `noaoh`), `AWS_REGION`.
|
||||
- Credentials: Secret `aws-dynamodb` in namespace `hackernews-favs-cron`, keys
|
||||
`AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`. Created by kubectl, never in git.
|
||||
Scope the IAM user to `dynamodb:PutItem` on the table.
|
||||
- Run once now: `kubectl create job -n hackernews-favs-cron --from=cronjob/hackernews-favs-cron manual-$(date +%s)`
|
||||
- Remove the Lambda schedule once happy: `cdk destroy` in `hackernews-favs`
|
||||
would also delete the table (RemovalPolicy.DESTROY), so disable the EventBridge
|
||||
rule instead.
|
||||
Loading…
Add table
Add a link
Reference in a new issue