1.2 KiB
1.2 KiB
BlueSky Bot
A simple project that scrapes websites and publishes tweets on BlueSky.
⚠️ Work In Progress ⚠️
Architecture
The architecture is composed of the following elements:
- The Scrapper
It scrapes data from one or more websites and publishes a JSON on Redis Streams.
It is configured via CLI arguments
Usage: scrapper [OPTIONS] --redis-connection-string <REDIS_CONNECTION_STRING> --redis-stream-name <REDIS_STREAM_NAME>
Options:
-r, --redis-connection-string <REDIS_CONNECTION_STRING>
Redis host
-t, --redis-stream-name <REDIS_STREAM_NAME>
Redis stream name
-s, --scrape-interval-minutes <SCRAPE_INTERVAL_MINUTES>
The scraping interval in minutes [default: 60]
-h, --help
Print help
-V, --version
Print version
- Redis
Redis is a key-value store with lots of features. It has been chosen to keep things simple and due to its powerful features and flexibility[1].
- BlueSky Bot
The BlueSky bot reads data from Redis Streams and publishes it to BlueSky.
[1] - https://redis.io/about/