dnutiu/social-media-news-bot
dnutiu/social-media-news-bot
1
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
A blue-sky bot that publishes posts from some scrapped websites. https://blog.nuculabs.dev
71 commits 1 branch 0 tags 959 KiB
Rust 95.9%
Dockerfile 3.5%
Makefile 0.6%
Find a file
dnutiu 78bfbf3cd8 Update readme.md
2025-01-07 11:42:07 +00:00
.idea ignore duplicated code in RustRover 2025-01-04 16:25:20 +02:00
bot format code 2025-01-04 18:22:05 +02:00
docs update readme.md 2025-01-04 18:21:56 +02:00
infrastructure refactor read_stream 2024-12-28 10:22:18 +02:00
post implement read from redis streams 2024-12-27 18:36:56 +02:00
scrapper add scrapper TODO: tie stream name to post title 2025-01-04 18:25:40 +02:00
synology add restart 2025-01-06 22:52:21 +02:00
.dockerignore add .idea to docker ignore 2024-12-26 17:14:17 +02:00
.gitignore initial commit 2024-12-17 16:28:57 +02:00
Cargo.toml add bot crate 2024-12-27 11:10:48 +02:00
docker-compose.yaml add application/json Content-Type header to requests 2024-12-30 18:49:21 +02:00
Makefile add bluesky bot deployment file 2025-01-01 16:19:03 +02:00
readme.md Update readme.md 2025-01-07 11:42:07 +00:00

readme.md

Social Media News Bot

A simple Bot that scrapes websites and publishes tweets on BlueSky and Mastodon.

It's built with Rust and Redis and can be extended to include LLM support for content summarization, suggestions and other features.

Demo:

demo bluesky

demo mastodon

Architecture

architecture diagram

The architecture is composed of the following elements:

  1. The Scrapper

It scrapes data from one or more websites and publishes a JSON on Redis Streams.

It is configured via CLI arguments

Usage: scrapper [OPTIONS] --redis-connection-string <REDIS_CONNECTION_STRING> --redis-stream-name <REDIS_STREAM_NAME>

Options:
  -r, --redis-connection-string <REDIS_CONNECTION_STRING>
          Redis host
  -t, --redis-stream-name <REDIS_STREAM_NAME>
          Redis stream name
  -s, --scrape-interval-minutes <SCRAPE_INTERVAL_MINUTES>
          The scraping interval in minutes [default: 60]
  -h, --help
          Print help
  -V, --version
          Print version
  1. Redis

Redis is a key-value store with lots of features. It has been chosen to keep things simple and due to its powerful features and flexibility[1].

  1. BlueSky Bot

The BlueSky bot reads data from Redis Streams and publishes it to BlueSky.

Social media posting bot.

Usage: bot --redis-connection-string <REDIS_CONNECTION_STRING> --redis-stream-name <REDIS_STREAM_NAME> --redis-consumer-group <REDIS_CONSUMER_GROUP> --redis-consumer-name <REDIS_CONSUMER_NAME> <COMMAND>

Commands:
  bluesky   Post on bluesky platform
  mastodon  Post on Mastodon, the FediVerse
  help      Print this message or the help of the given subcommand(s)

Options:
  -r, --redis-connection-string <REDIS_CONNECTION_STRING>  Redis host
  -t, --redis-stream-name <REDIS_STREAM_NAME>              Redis stream name
  -c, --redis-consumer-group <REDIS_CONSUMER_GROUP>        Redis consumer group name
  -n, --redis-consumer-name <REDIS_CONSUMER_NAME>          The current consumer name
  -h, --help                                               Print help
  -V, --version                                            Print version

[1] - https://redis.io/about/