LLM Council

Project: LLM Council

Situation

Different language models often produce different answers, and users need a way to compare their responses and identify outliers.

Task

Build a local web application that queries several LLMs, lets them review one another, and produces a final response through a separate chairman model.

Action

Implemented a three-stage pipeline through OpenRouter
Queried multiple models in parallel during the first stage
Displayed individual answers for direct inspection
Anonymized model identities during peer review
Used a chairman model to synthesize the reviewed responses
Added a FastAPI backend, React/Vite frontend, JSON conversation persistence, and Docker Compose deployment

Result

Created a transparent model-comparison workflow where users can inspect raw opinions, peer reviews, and the final synthesis instead of receiving one opaque answer.

Technologies

Python, FastAPI, async HTTP, OpenRouter API, React, Vite, JSON persistence, Docker Compose