Architecture Overview

This document describes the high-level architecture of the Groundtruth Platform, a multi-tenant SaaS system for AI-powered consulting engagements.

Monorepo Structure

groundtruth-platform/
├── apps/web/                    # Next.js 14 (App Router), TypeScript
│   ├── src/app/                 # Pages + API routes
│   │   ├── (auth)/              # Login, register, forgot-password
│   │   ├── (dashboard)/         # Main app (behind auth)
│   │   │   ├── engagements/     # List, create, detail views
│   │   │   ├── settings/        # Account, billing, team, SSO, branding
│   │   │   └── page.tsx         # Dashboard home
│   │   ├── api/                 # 14 API route groups
│   │   │   ├── account/         # GDPR export, deletion
│   │   │   ├── admin/           # Health dashboard
│   │   │   ├── analytics/       # Tenant analytics, quality, patterns
│   │   │   ├── api-keys/        # API key CRUD
│   │   │   ├── auth/            # Auth callbacks
│   │   │   ├── billing/         # Stripe checkout, portal, webhooks
│   │   │   ├── docs/            # Swagger UI (OpenAPI)
│   │   │   ├── engagements/     # Engagement CRUD + run control
│   │   │   ├── health/          # Health check endpoint
│   │   │   ├── portal/          # Client portal (token-based)
│   │   │   ├── team/            # Team invite, role management
│   │   │   ├── templates/       # Engagement templates
│   │   │   ├── tenant/          # Tenant settings (branding, SSO)
│   │   │   └── webhooks/        # Webhook endpoint CRUD
│   │   └── portal/              # Client portal pages
│   ├── src/components/          # 35+ React components
│   ├── src/lib/                 # 21 shared utility modules
│   ├── src/hooks/               # Custom React hooks (SSE, streaming, mission control)
│   └── prisma/                  # Schema, migrations, seed scripts
│       ├── schema.prisma        # 20 Prisma models
│       ├── migrations/          # Migration history
│       ├── seed.ts              # Agent roster seeder (20 agents)
│       └── seed-templates.ts    # Engagement template seeder
├── packages/engine/             # Python 3.12 engine (FastAPI on Railway)
│   ├── api.py                   # FastAPI server with SSE support
│   ├── runner.py                # Engagement run lifecycle
│   ├── dag.py                   # DAG-based parallel task execution
│   ├── router.py                # Multi-LLM provider dispatch
│   ├── model_selector.py        # Adaptive model selection
│   ├── costs.py                 # Per-run cost tracking + budget breaker
│   ├── observer.py              # Task quality scoring
│   ├── red_team.py              # Adversarial deliverable review
│   ├── context.py               # Context windowing pipeline
│   ├── insights.py              # Shared knowledge bus
│   ├── library.py               # Post-run pattern analysis
│   ├── storage/                 # StorageAdapter implementations
│   │   ├── adapter.py           # Abstract base class
│   │   ├── database.py          # PostgreSQL adapter (production)
│   │   └── filesystem.py        # Local filesystem adapter (dev/test)
│   ├── tests/                   # Engine unit tests
│   └── requirements.txt         # Python dependencies
├── docs/                        # Documentation
│   ├── developer-guide/         # This directory
│   ├── api-reference/           # API documentation
│   ├── operations/              # Operational runbooks
│   └── user-guide/              # End-user documentation
└── infrastructure/
    ├── railway.toml             # Railway deployment config
    └── k6/                      # Load testing scripts

Tech Stack

Layer	Technology	Purpose
Frontend	Next.js 14 (App Router), TypeScript	SSR, API routes, Vercel-native deployment
ORM	Prisma	Type-safe database access, migration management
Database	PostgreSQL (Supabase)	RLS for tenant isolation, JSONB for flexible schemas
Auth	Supabase Auth	Email/password, magic link, JWT, SSO (SAML)
File Storage	Supabase Storage	Engagement attachments (private bucket)
Billing	Stripe	Checkout sessions, subscriptions, usage metering, hosted billing portal
Caching	Upstash Redis	API response caching, rate limiting (sliding window)
Email	Resend	Transactional emails (engagement notifications, invites)
Observability	Sentry	Error tracking, performance monitoring (Next.js + Python)
Hosting (web)	Vercel	Frontend + API routes (serverless)
Hosting (engine)	Railway	Long-running Python crew execution (persistent service)
AI Engine	Python 3.12 + CrewAI	Multi-agent crew orchestration

Data Flow

Standard Web Requests

Browser  -->  Next.js API Route  -->  Prisma  -->  PostgreSQL (Supabase)
                  |                                      ^
                  v                                      |
             Middleware                          RLS policies enforce
         (auth + tenant)                         tenant isolation

All web requests flow through Next.js API routes. Prisma handles query construction with type safety. Supabase RLS policies provide a second layer of tenant isolation at the database level.

Engine Operations (Engagement Runs)

Browser  -->  Next.js API  -->  Railway Engine (HTTP)  -->  PostgreSQL
                                       |
                                       v
                                 CrewAI Agents
                              (LLM API calls via
                                  LLMRouter)

Engagement runs take 10-60 minutes, far exceeding Vercel's serverless timeout (60 seconds). The Python engine runs as a persistent service on Railway. Next.js API routes proxy requests to it.

Real-time Updates (SSE)

Browser  <--  SSE (Next.js proxy)  <--  SSE (Engine)

During active runs, the engine streams real-time updates via Server-Sent Events. The event stream includes log.line, cost.updated, dag.updated (full DAG state with task status), agent.activity (per-agent structured events), run.started, and run.completed. The Next.js API proxies these connections to the browser, where the Mission Control UI renders them as a live dashboard with task pipeline, crew floor, and activity stream.

Multi-Tenancy

Every database query is scoped by tenantId. Tenant isolation is enforced at three layers:

Database layer: Supabase Row-Level Security (RLS) policies on all tables. Even if application code has a bug, cross-tenant data access is blocked at the PostgreSQL level.
Application layer: Middleware extracts the tenant from the JWT on every request. The tenantId is injected into all Prisma queries.
Engine layer: The StorageAdapter receives tenant_id and scopes all storage operations. The engine API requires an X-Tenant-ID header.

Auth Flow

1. User logs in via Supabase Auth (email/password or magic link)
2. Supabase returns JWT stored as HTTP-only cookie
3. Next.js middleware reads cookie on every request
4. getAuthenticatedUser() validates JWT, resolves User + Tenant from DB
5. tenantId is available to all API route handlers

API Key Auth (Programmatic Access)

1. Tenant generates API key via /api/api-keys
2. Key is hashed (SHA-256) and stored; plaintext returned once
3. Client sends: Authorization: Bearer <api-key>
4. Middleware hashes incoming key, looks up ApiKey record
5. Resolves tenant from the key's associated user
6. Supports read-only vs read-write scopes, per-engagement or tenant-wide

Both auth methods produce the same tenant context for downstream code.

Key Abstractions

StorageAdapter (Python Engine)

The central abstraction that decouples the AI engine from its storage backend. All engine modules interact with storage exclusively through this interface -- no direct filesystem calls.

DatabaseStorageAdapter: Production implementation. Reads/writes to PostgreSQL via psycopg2. Handles the CrewAI temp-file problem (CrewAI expects filesystem paths, so the adapter writes to temp files and persists to DB after task completion).
FileSystemStorageAdapter: Local development and testing. Reads/writes to the local filesystem.

Domain	Models
Multi-tenancy	`Tenant`, `User`
Engagements	`Engagement`, `Run`, `Deliverable`
Client Portal	`ClientPortalToken`, `PortalComment`
Templates	`EngagementTemplate`
Audit & Security	`AuditLog`
API Access	`ApiKey`
Webhooks	`WebhookEndpoint`, `WebhookDelivery`
Agent Configuration	`AgentConfig`
Quality & Analytics	`TaskScore`, `LibraryEntry`, `PlatformInsight`, `ImprovementProposal`
Model Routing	`ModelRoutingDecision`
Attachments	`EngagementAttachment`
GDPR	`DataDeletionRequest`

The full schema is at apps/web/prisma/schema.prisma.

RBAC Model

Four roles with hierarchical permissions:

Role	Capabilities
Owner	Full access + billing + team management + delete tenant
Admin	Create/manage engagements + team members + view costs
Member	Create/run engagements + view deliverables
Viewer	Read-only (for client stakeholders with accounts)

Role checks are enforced in middleware on a per-route basis via src/lib/rbac.ts.

Environments & Dev Setup -- dev branches, Supabase branching, environment matrix
Engine Architecture -- detailed breakdown of all 11 engine modules
Local Development -- setup guide for running locally
Deployment -- deploying to Vercel, Railway, and Supabase
Testing -- test framework, patterns, and coverage

Architecture Overview

Architecture Overview

Monorepo Structure

Tech Stack

Data Flow

Standard Web Requests

Engine Operations (Engagement Runs)

Real-time Updates (SSE)

Multi-Tenancy

Auth Flow

API Key Auth (Programmatic Access)

Key Abstractions

StorageAdapter (Python Engine)

LLMRouter (Python Engine)

EngineClient (TypeScript)

Database Schema

RBAC Model

On this page