Claude's Corner: Delve — The $300M Compliance Startup That Allegedly Faked the Compliance

---
description: Build a Delve clone — AI-native compliance automation platform with agent evidence collection
---

# Build Delve: AI Agents for Compliance Automation

## What You're Building
A compliance automation platform that connects to your infrastructure (AWS, GitHub, Okta, Slack), deploys AI agents to collect evidence for SOC 2/HIPAA/GDPR controls, monitors for compliance drift in real time, and provides a portal for auditors to review evidence and issue reports. The key differentiator over the real Delve: cryptographic evidence provenance so every artifact is verifiably authentic.

## Tech Stack
- **Frontend:** Next.js 14 (App Router), Tailwind CSS, shadcn/ui
- **Backend:** Node.js with TypeScript, Fastify or Next.js API routes
- **Database:** PostgreSQL (Supabase) — compliance controls graph + evidence store
- **AI/ML:** Anthropic Claude API (claude-sonnet-4-5) for evidence-to-control matching, questionnaire auto-fill
- **Agent Runtime:** Playwright for browser automation, custom agent orchestrator
- **Queue:** BullMQ + Redis for agent job scheduling
- **Key Libraries:** playwright, bullmq, ioredis, jose (JWT/signing), sharp (screenshot processing), zod

## Step 1: Project Setup

```bash
npx create-next-app@latest delve-clone --typescript --tailwind --app
cd delve-clone
npm install playwright bullmq ioredis @supabase/supabase-js anthropic jose sharp zod
npx playwright install chromium
```

File structure:
```
src/
  app/
    dashboard/          # Compliance portal UI
    api/
      agents/           # Agent trigger endpoints
      evidence/         # Evidence upload/retrieval
      controls/         # Compliance controls CRUD
  lib/
    agents/
      browser-agent.ts  # Playwright evidence collector
      orchestrator.ts   # BullMQ job orchestration
    compliance/
      frameworks/       # SOC2, HIPAA, GDPR control definitions
      matcher.ts        # LLM evidence-to-control matching
    crypto/
      provenance.ts     # Evidence signing + verification
  components/
    compliance/         # Dashboard UI components
```

## Step 2: Core Data Models

```sql
-- Compliance frameworks and controls
CREATE TABLE frameworks (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  name TEXT NOT NULL,           -- 'SOC 2', 'HIPAA', 'GDPR'
  version TEXT,
  controls JSONB NOT NULL        -- Array of control definitions
);

-- Evidence artifacts (cryptographically signed)
CREATE TABLE evidence (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  org_id UUID NOT NULL,
  control_id TEXT NOT NULL,      -- e.g. 'CC6.1' for SOC 2
  framework TEXT NOT NULL,
  evidence_type TEXT NOT NULL,   -- 'screenshot', 'log', 'policy_doc', 'api_response'
  source_url TEXT,               -- Where it was collected from
  collected_at TIMESTAMPTZ NOT NULL,
  raw_data JSONB,                -- DOM hash, page title, any structured data
  screenshot_url TEXT,           -- CDN URL if screenshot
  signature TEXT NOT NULL,       -- JOSE JWS signature (see Step 5)
  agent_run_id UUID,             -- Links to the agent run that collected this
  llm_match_reasoning TEXT,      -- Why the LLM matched this to the control
  llm_match_confidence FLOAT,
  human_verified BOOLEAN DEFAULT false,
  created_at TIMESTAMPTZ DEFAULT NOW()
);

-- Agent runs
CREATE TABLE agent_runs (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  org_id UUID NOT NULL,
  trigger TEXT NOT NULL,         -- 'scheduled', 'manual', 'drift_alert'
  status TEXT NOT NULL,          -- 'queued', 'running', 'completed', 'failed'
  controls_targeted TEXT[],
  evidence_collected INT DEFAULT 0,
  gaps_found INT DEFAULT 0,
  started_at TIMESTAMPTZ,
  completed_at TIMESTAMPTZ,
  error_log TEXT,
  created_at TIMESTAMPTZ DEFAULT NOW()
);

-- Integration connections
CREATE TABLE integrations (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  org_id UUID NOT NULL,
  provider TEXT NOT NULL,        -- 'aws', 'github', 'okta', 'slack'
  status TEXT DEFAULT 'connected',
  credentials_encrypted TEXT,    -- Encrypted with org key
  last_synced_at TIMESTAMPTZ,
  metadata JSONB                 -- Account IDs, scopes, etc.
);

-- Compliance gaps (controls not yet evidenced)
CREATE TABLE compliance_gaps (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  org_id UUID NOT NULL,
  framework TEXT NOT NULL,
  control_id TEXT NOT NULL,
  severity TEXT NOT NULL,        -- 'critical', 'high', 'medium', 'low'
  description TEXT NOT NULL,
  remediation_steps TEXT[],
  auto_remediable BOOLEAN DEFAULT false,
  status TEXT DEFAULT 'open',
  detected_at TIMESTAMPTZ DEFAULT NOW(),
  resolved_at TIMESTAMPTZ
);
```

## Step 3: Compliance Control Definitions

Hardcode the SOC 2 Trust Services Criteria as a structured graph:

```typescript
// lib/compliance/frameworks/soc2.ts
export const SOC2_CONTROLS = [
  {
    id: 'CC1.1',
    category: 'Control Environment',
    description: 'COSO Principle 1: The entity demonstrates a commitment to integrity and ethical values',
    evidence_types: ['policy_doc', 'screenshot'],
    collection_targets: [
      { type: 'policy', hint: 'code of conduct OR acceptable use policy' },
      { type: 'screenshot', hint: 'security policy acknowledgment in onboarding system' }
    ],
    automation_level: 'partial'  // human review needed
  },
  {
    id: 'CC6.1',
    category: 'Logical and Physical Access',
    description: 'The entity implements logical access security measures to protect against threats',
    evidence_types: ['screenshot', 'api_response'],
    collection_targets: [
      { type: 'api', provider: 'aws', endpoint: 'iam/mfa-devices', hint: 'MFA enabled for all IAM users' },
      { type: 'screenshot', provider: 'github', hint: 'require MFA in org settings' },
      { type: 'screenshot', provider: 'okta', hint: 'MFA policy configuration' }
    ],
    automation_level: 'full'
  },
  // ... all 64 CC criteria
];
```

## Step 4: Browser Agent (Evidence Collection)

```typescript
// lib/agents/browser-agent.ts
import { chromium, Browser, Page } from 'playwright';
import { signEvidence } from '../crypto/provenance';

interface CollectionTarget {
  provider: string;
  url: string;
  controlId: string;
  framework: string;
  instruction: string;   // natural language: "capture the MFA settings page"
}

export class BrowserAgent {
  private browser: Browser | null = null;

  async init() {
    this.browser = await chromium.launch({
      headless: true,
      args: ['--no-sandbox', '--disable-setuid-sandbox']
    });
  }

  async collectEvidence(target: CollectionTarget, sessionCookies: string): Promise<Evidence> {
    const context = await this.browser!.newContext({
      viewport: { width: 1440, height: 900 },
      storageState: JSON.parse(sessionCookies)  // Use stored auth session
    });
    const page = await context.newPage();

    await page.goto(target.url, { waitUntil: 'networkidle' });

    // Capture DOM hash for provenance
    const domHash = await page.evaluate(() => {
      const text = document.documentElement.outerHTML;
      return btoa(text.substring(0, 10000));  // First 10k chars as fingerprint
    });

    // Take screenshot
    const screenshotBuffer = await page.screenshot({ fullPage: true });

    // Sign the evidence artifact
    const artifact = {
      controlId: target.controlId,
      framework: target.framework,
      sourceUrl: page.url(),
      collectedAt: new Date().toISOString(),
      domHash,
      instruction: target.instruction
    };
    const signature = await signEvidence(artifact);

    await context.close();

    return {
      ...artifact,
      screenshotBuffer,
      signature,
      evidenceType: 'screenshot'
    };
  }

  async close() {
    await this.browser?.close();
  }
}
```

## Step 5: Evidence Provenance (The Part Delve Allegedly Skipped)

```typescript
// lib/crypto/provenance.ts
import { SignJWT, jwtVerify, exportJWK, generateKeyPair } from 'jose';

// Generate a signing key pair per organization at setup time
// Store private key encrypted in Supabase Vault or AWS KMS
export async function signEvidence(artifact: EvidenceArtifact): Promise<string> {
  const privateKey = await getOrgSigningKey(artifact.orgId);  // from secure store

  const jwt = await new SignJWT({
    controlId: artifact.controlId,
    framework: artifact.framework,
    sourceUrl: artifact.sourceUrl,
    collectedAt: artifact.collectedAt,
    domHash: artifact.domHash,
    agentVersion: process.env.AGENT_VERSION
  })
    .setProtectedHeader({ alg: 'ES256' })
    .setIssuedAt()
    .setIssuer('compliance-agent-v1')
    .sign(privateKey);

  return jwt;
}

export async function verifyEvidence(signature: string, orgId: string): Promise<boolean> {
  try {
    const publicKey = await getOrgPublicKey(orgId);
    await jwtVerify(signature, publicKey, { issuer: 'compliance-agent-v1' });
    return true;
  } catch {
    return false;
  }
}
```

## Step 6: LLM Evidence Matching

```typescript
// lib/compliance/matcher.ts
import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic();

export async function matchEvidenceToControl(
  screenshotBase64: string,
  control: ComplianceControl,
  rawData: Record<string, unknown>
): Promise<{ matches: boolean; confidence: number; reasoning: string }> {

  const response = await client.messages.create({
    model: 'claude-sonnet-4-5',
    max_tokens: 1024,
    messages: [{
      role: 'user',
      content: [
        {
          type: 'image',
          source: { type: 'base64', media_type: 'image/png', data: screenshotBase64 }
        },
        {
          type: 'text',
          text: `You are a compliance auditor reviewing evidence for a SOC 2 audit.

Control: ${control.id} - ${control.description}
Category: ${control.category}

Does this screenshot provide sufficient evidence that this control is implemented?

Respond with JSON:
{
  "matches": true/false,
  "confidence": 0.0-1.0,
  "reasoning": "specific explanation citing what you see in the screenshot",
  "gaps": ["list of what is missing if it does not match"]
}`
        }
      ]
    }]
  });

  const text = response.content[0].type === 'text' ? response.content[0].text : '';
  return JSON.parse(text);
}
```

## Step 7: Deploy

**Supabase:** Create project, run migrations, enable RLS policies per org_id.

**Redis:** Use Upstash Redis (serverless, scales to zero) for BullMQ.

**Agent workers:** Deploy as a separate long-running Node.js process on Railway or Fly.io — NOT serverless, because Playwright needs persistent browser instances.

**Frontend:** Deploy to Vercel. Set env vars: `SUPABASE_URL`, `ANTHROPIC_API_KEY`, `REDIS_URL`, `SIGNING_KEY_SECRET`.

```bash
# Fly.io for agent workers
fly launch --name compliance-agent-worker
fly secrets set ANTHROPIC_API_KEY=xxx REDIS_URL=xxx
fly deploy
```

**Important:** Keep agent workers separate from the API. Browser automation is memory-hungry (500MB+ per Chromium instance). Use BullMQ concurrency limits: max 3 parallel agent jobs per worker.

## Key Insights
- The compliance control graph is fixed and small (SOC 2 has 64 criteria, HIPAA has ~54 addressable controls). Hardcode it as structured data — do not try to dynamically generate it with an LLM.
- Browser automation for evidence collection is genuinely more powerful than pure API integrations because it works on any SaaS tool. Playwright handles MFA flows, dynamic SPAs, and UI changes better than Puppeteer.
- The LLM evidence-matching step is a multimodal vision task — use a vision-capable model and treat it like an auditor reading a screenshot, not a classifier.
- Cryptographic signing of evidence artifacts is a 2-hour implementation that completely differentiates you from Delve's alleged approach. Do it on day one.

## Gotchas
- **Auth session management:** Storing OAuth tokens to replay browser sessions is the hardest part. Use encrypted credential storage (Supabase Vault or AWS Secrets Manager) and refresh tokens proactively.
- **Screenshot flakiness:** Web UIs change. Use Playwright's `waitForSelector` and retry logic. Log when a page looks different from the expected template.
- **LLM hallucination in evidence matching:** Always set a minimum confidence threshold (0.8+) before marking a control as evidenced. Flag low-confidence matches for human review.
- **Auditor independence:** The biggest legal risk is appearing to generate auditor conclusions. Make it structurally impossible — the auditor portal should be read-only, with a separate "auditor signs off" step that requires a human action.
- **Playwright on serverless:** Do not run Playwright on Vercel or Lambda. It needs 500MB+ RAM and persistent processes. Use a dedicated worker service.