add readme.md
This commit is contained in:
parent
9a3d65b26d
commit
1eaee1d7df
2 changed files with 46 additions and 0 deletions
BIN
docs/architecture_diagram.drawio.png
Normal file
BIN
docs/architecture_diagram.drawio.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 148 KiB |
46
readme.md
Normal file
46
readme.md
Normal file
|
@ -0,0 +1,46 @@
|
|||
# BlueSky Bot
|
||||
|
||||
A simple project that scrapes websites and publishes tweets on [BlueSky](https://bsky.app/).
|
||||
|
||||
**⚠️ Work In Progress ⚠️**
|
||||
|
||||
|
||||
## Architecture
|
||||
|
||||
![architecture diagram](./docs/architecture_diagram.drawio.png)
|
||||
|
||||
The architecture is composed of the following elements:
|
||||
|
||||
1. The Scrapper
|
||||
|
||||
It scrapes data from one or more websites and publishes a JSON on **Redis Streams**.
|
||||
|
||||
It is configured via CLI arguments
|
||||
|
||||
```bash
|
||||
Usage: scrapper [OPTIONS] --redis-connection-string <REDIS_CONNECTION_STRING> --redis-stream-name <REDIS_STREAM_NAME>
|
||||
|
||||
Options:
|
||||
-r, --redis-connection-string <REDIS_CONNECTION_STRING>
|
||||
Redis host
|
||||
-t, --redis-stream-name <REDIS_STREAM_NAME>
|
||||
Redis stream name
|
||||
-s, --scrape-interval-minutes <SCRAPE_INTERVAL_MINUTES>
|
||||
The scraping interval in minutes [default: 60]
|
||||
-h, --help
|
||||
Print help
|
||||
-V, --version
|
||||
Print version
|
||||
```
|
||||
|
||||
2. Redis
|
||||
|
||||
Redis is a key-value store with lots of features. It has been chosen to keep
|
||||
things simple and due to its powerful features and flexibility[1].
|
||||
|
||||
3. BlueSky Bot
|
||||
|
||||
The BlueSky bot reads data from Redis Streams and publishes it to BlueSky.
|
||||
|
||||
[1] - https://redis.io/about/
|
||||
|
Loading…
Reference in a new issue