Full case study

Ehi Aigiomawu

Backend engineering · Applied AI · Case study brief

ehiagheaigg@gmail.comgithub.com/ehiaig

CS-03In progressSaaS product · LLM classification · Integrations

Snitch: AI social listening that knows when not to act

A product in active development. Its marketing site is live at heysnitch.com. No usage, customer or revenue figures are claimed here.

Context

Teams want to know who is talking about them, about their competitors, and about the problem they solve, and to respond quickly when it matters. Snitch is a multi-tenant SaaS product built for that, with a dashboard, REST API, MCP server and webhooks.

Problem

Keyword monitoring floods teams with noise, especially for brands whose names are ordinary words, while the posts that matter (complaints, bug reports, buying intent) get lost. Acting on them means replying in public, where mistakes are expensive.

Approach

Filter cheapest-first before any model call. Classify with explicit context and a three-way confidence band. Send uncertain matches to human review. Keep approving a reply separate from publishing it. Expose the same capabilities through the dashboard, REST and MCP, with scopes, rate limits, idempotency and an audit trail.

What I did

I’m designing and building the product, including ingestion workers, classification, the review workflow, integrations and the agent and API surface.

Key decisions

Filter cheapest-firstOnly the classifier costs money per mention, so rules and local checks run first.
A three-way confidence bandUncertain matches go to review instead of being guessed either way.
Approval and publishing are separate actionsConsent is recorded; saying it in public is a second, explicit decision.
One capability registry for REST and MCPAgents get the same scopes, limits and audit trail as people.

Risks managed

  • Silently dropping relevant mentions
  • Paying to classify obvious noise
  • Publishing something nobody approved
  • Publishing the same reply twice
  • Over-broad agent access through the API or MCP
  • Losing webhook events on a crash

How success would be measured

  • Share of feed items reviewers confirm as relevant
  • Size and age of the Needs review queue
  • Classifier spend per useful mention
  • Replies published without approval, with a target of zero

Intended outcome

Useful signals surfaced with fewer false alarms, no silent losses, and no unapproved public replies, measurable through review decisions, feed precision and cost per useful mention.