Workers Architecture
Background workers for asynchronous processing in the self-hosted social network stack.
Scheduled (cron) jobs: Scheduled Jobs.
Bus summary (current)
| Bus / mode | Workers |
|---|---|
| Kafka | InteractionWorker, ScoringWorker, TrendingWorker, GraphWorker, HashtagWorker, ElasticsearchSyncWorker |
| RabbitMQ | WebSocketWorker, ChatWorker (also hosts SignalR), WebPushNotificationWorker, ExpoPushNotificationWorker, BlogAutoGenerationWorker, ContentModerationWorker |
| Polling (PostgreSQL) | MediaProcessingWorker (pending MediaFile rows — not Kafka) |
Common mistake
Do not list MediaProcessingWorker or WebSocketWorker as Kafka consumers. Media polls Postgres; WebSocketWorker consumes RabbitMQ and hosts SignalR notification hubs.
Kafka workers
Consume topics such as user_interacted, content_created, content_updated, content_deleted:
| Worker | Purpose |
|---|---|
| InteractionWorker | Interaction summaries + Redis counts |
| ScoringWorker | Personalized content scores (feed ranking) |
| TrendingWorker | Global trends + Redis cache |
| GraphWorker | Neo4j relationship weights / suggestions |
| HashtagWorker | Hashtag extract, counts, trends |
| ElasticsearchSyncWorker | Keep search indexes in sync |
Flow: API publishes event → Kafka topic → worker → PostgreSQL / Neo4j / Elasticsearch / Redis.
RabbitMQ workers
| Worker | Purpose |
|---|---|
| ChatWorker | SignalR /hubs/chat, /hubs/chat-online; chat delivery; publishes call push messages |
| WebSocketWorker | SignalR /hubs/notification, /hubs/onlineusers; friendship/notification consumers |
| WebPushNotificationWorker | Browser Web Push (VAPID) |
| ExpoPushNotificationWorker | Mobile Expo Push |
| BlogAutoGenerationWorker | AI blog generation (+ Quartz schedule) |
| ContentModerationWorker | OpenAI GPT text + vision → hide/reject harmful posts & stories; Content Alerts |
MediaProcessingWorker
- API creates
MediaFilewith statusPendingand stores original in temp object storage - Worker polls pending rows
- Processes with ImageSharp / FFmpeg
- Writes public assets to MinIO/R2, updates DB, caches URLs
See Media.
SignalR hosting
| Hub paths | Host process |
|---|---|
/hubs/chat, /hubs/chat-online | ChatWorker |
/hubs/notification, /hubs/onlineusers | WebSocketWorker |
The main API does not host these hubs. Traefik (or Nginx) must route hub paths to the correct worker. See Deployment and Traefik checklist.
Shared foundation
Workers reuse SharedWorker (config, health, Kafka helpers, JWT for worker→API calls).