Clay Workbook Deep-Dive: HeyDigital ABM Engagers

What This Workbook Does

This is a LinkedIn engagement → enriched lead → personalized email → multi-channel push pipeline. HeyDigital sends LinkedIn post engagement data via webhook. The workbook extracts the engager identity, hunts for their email via three providers, validates the best result, verifies they still work at the company, qualifies their job title authority, enriches their company profile, filters by geography and headcount, runs competitor analysis, cleans the data, generates a cold email opening line, and pushes the lead to four destinations simultaneously: LeadGrow CRM, Bison (email campaigns), HeyReach (LinkedIn outreach), and a social post content bank.

Key Stats

Table t_0tfqyhkneq3VAFhoM6F — "HeyDigital ABM engagers" — is the heart of the system. Each row represents one LinkedIn post engager flowing through a linear enrichment chain with conditional branching (waterfall logic on email find/validate). Here is every phase, in execution order:

How Tables Connect

Primary stitch: The Main table Send table data action (Route Row, Clay Labs package) sends Post Text + Post URL to the Social Post Bank table (t_0tfqyiguHZvcGoypZJV). This is a Clay-native table-to-table routing — no external API needed. The Social Post Bank then enriches with "Hey Digital Relevance" AI scoring.

Secondary stitch (inferred): The enrichment tables (headcount, missing company info) likely feed into the Main table via Clay Lookup or Import from Table actions, pulling enriched company data back into the main pipeline. The "Enrich missing company information" workbook contains multiple tables that appear to be a sub-pipeline for company data.

External stitches: The 4 destination pushes all leave Clay via HTTP API calls — to LeadGrow API, Bison campaigns, and HeyReach. These are the exit points from Clay into the rest of the LeadGrow stack.

Webhook entry: Table 3 ("Pull in data from a Webhook Table") is a separate entry point with its own webhook source. It enriches company data and outputs via HTTP API — possibly used to backfill company info that the main pipeline references.

Clay Credit Model (approximate)

Clay charges credits per enrichment action. Pricing varies by action type and provider. Approximate credit costs based on Clay published pricing and provider tiers:

Architecture Decision: DAG of Tasks, Not One Monolith

Trigger.dev is built on event-driven task execution with retries, idempotency, and observable state. Each Clay "column" becomes a Trigger.dev task step or independent subtask. The 12-phase pipeline becomes a parent task that orchestrates child tasks, with each phase as a discrete retryable unit.

The key insight: Clay columns are just sequential code with field references. In Trigger.dev, this is TypeScript with variables — no graph resolution engine needed. The formula expressions become inline code. Conditional branching becomes if statements. The waterfall becomes explicit try/catch chains.

Trigger.dev Plans (as of mid-2026)

A "run" = one task execution. Our parent task + ~5 child tasks per row = ~6 runs per lead processed. Child tasks count against the run quota. At 10,000 leads/month, that is ~60,000 total runs. On the Team plan, 100K included runs covers this with room. On Pro, 40K overage at $0.002 = $80 extra.

Advantages of Staying on Clay

• Zero engineering time — already built, already running, no migration effort
• Visual debugging — see exactly which column failed, on which row, with what error. Invaluable for operations.
• No infrastructure — no servers, no deploys, no monitoring to set up
• Claygent infrastructure — their AI web browsing handles LinkedIn auth, rate limiting, proxy rotation. Replicating this well is 2-3 weeks of engineering.
• Enrichment library — 100+ providers pre-integrated. Adding a new enrichment is two clicks, not a PR + deploy.
• Non-technical editing — ops team can modify prompts, add columns, change waterfall logic without code.

Advantages of Migrating to Trigger.dev

• 76-81% cost reduction at any scale — $36K-48K/year savings at just 10K leads/month
• Full code control — version in git, test with Jest, review in PRs, rollback with git revert
• No vendor lock-in — Trigger.dev is open source. Can self-host. Can migrate to Inngest, QStash, Temporal.
• TypeScript — type-safe, composable, debuggable. Clay formulas are a proprietary language with no IDE support.
• Deterministic logic is free — country lookups, string transforms, formulas cost nothing vs Clay credits
• Better retry semantics — per-step retries with exponential backoff, idempotency keys, dead letter queues
• Observability — OpenTelemetry traces span the entire pipeline, from webhook to destination push
• Supabase integration — already in the LeadGrow stack, free state persistence for post bank and intermediate results
• Unlimited AI models — not locked to Clay AI model catalog. Can use Claude, Gemini, Grok, or fine-tuned models

Phase 1: Infrastructure Setup (Week 1)

1. Create Trigger.dev project in existing LeadGrow org (use lg-trigger CLI for project scaffolding)
2. Set up provider API accounts — Icypeas direct API key, LeadMagic direct, Serper.dev, Spider Cloud or ScrapingBee, EmailGuard direct
3. Build SDK wrappers — thin TypeScript wrappers for Icypeas, LeadMagic, Kitt, EmailGuard, Serper, Spider Cloud
4. Create Supabase table — social_post_bank to replace the Social Post Bank Clay table
5. Set up environments — dev/staging/production in Trigger.dev with separate API keys

Phase 2: Build Parent Task (Week 1-2)

1. Implement processHeyDigitalEngager — parent task with all 12 phases as inline steps
2. Email waterfall — Icypeas -> Kitt -> LeadMagic with explicit try/catch, stop on first valid result
3. Company enrichment — Serper search LinkedIn -> scrape profile -> gpt-4o-mini extract fields
4. Deterministic helpers — country acronym lookup, author name mapping, pluralization (npm:pluralize)
5. AI tasks — job title qualification, competitor analysis, name cleaning, company cleaning, cold email personalization
6. Output tasks — CUL, Bison, HeyReach pushes + Supabase INSERT
7. Write tests — unit tests for each phase with mock API responses. Integration test for full pipeline.

Phase 3: Parallel Validation (Week 2-3)

1. Duplicate HeyDigital webhook — send to BOTH Clay and Trigger.dev simultaneously (webhook proxy or HeyDigital config)
2. Run 500-1,000 rows through both — collect outputs from both pipelines
3. Diff comparison:
   • Email find rate: what % of rows get a valid email? Is Trigger.dev rate same or better?
   • Email quality: do the same emails get found? If different, which is better?
   • Company enrichment: do LinkedIn company URLs match? Is domain extraction accurate?
   • AI quality: are first lines equally good? Are job title qualifications consistent?
   • Cost tracking: measure actual per-row API costs vs estimates
4. Edge case testing — non-Latin names, non-English company descriptions, missing LinkedIn profiles, personal posts (should output NA)
5. Fix discrepancies — iterate on prompts, scraping approach, fallback logic until Trigger.dev output >= Clay output quality

Phase 4: Gradual Cutover (Week 3-4)

1. Feature-flag destinations — switch CUL/Bison/HeyReach pushes from Clay to Trigger.dev ONE at a time with monitoring
2. Day 1-2: Supabase only — Trigger.dev pushes to Supabase post bank, Clay handles CRM pushes
3. Day 3-4: Add CUL — Trigger.dev handles CUL + Supabase, Clay handles Bison + HeyReach
4. Day 5-6: Add Bison — Trigger.dev handles CUL + Bison + Supabase, Clay handles HeyReach
5. Day 7: Full cutover — Trigger.dev handles all 4 destinations
6. Monitor for 1 week — with both pipelines still receiving data, confirm Trigger.dev outputs are production-quality

Phase 5: Archive Clay (Week 4)

1. Stop Clay webhook — redirect HeyDigital exclusively to Trigger.dev
2. Export Clay data — pull all records from Clay tables as JSON/CSV backup
3. Export Clay schemas — already done (see files below). Archive for reference.
4. Pause (do not delete) Clay workbook — keep for 30 days as fallback. Delete only after a full billing cycle with zero issues.
5. Document the migration — write post-mortem: what was easy, what was hard, what Clay did better, what Trigger.dev did better

Prompt caching strategy

Every prompt follows this structure to maximize caching:

<!-- CACHED (80% of tokens, 50% discount) -->
System prompt + Instructions + Banned words + Scoring rubric + Examples

<!-- UNCACHED (20% of tokens) -->
Variable 1: {{job_title}}
Variable 2: {{company_name}}
Variable 3: {{post_text}}
Variable 4: {{company_description}}
  

Flex API pricing (gpt-4.1-mini): .30/1M input, .15/1M cached input, $1.20/1M output.

Architecture Decisions

• Trigger: HeyDigital webhook → Trigger.dev HTTP endpoint
• Batching: 20 rows per parent task invocation. Trigger.dev batch trigger processes array.
• Parallelism: Employment verification + company enrichment run in parallel (no data dependency). Email waterfall runs after gates clear.
• Gating: Each phase is a step with if (!passes) return early — no wasted API calls.
• Caching: All AI prompts split into fixed header (cached) + variable footer (uncached).
• Idempotency: Keyed on linkedinUrl + postUrl. Duplicate webhooks are no-ops.

// trigger.ts — parent task
import { task, batch } from "@trigger.dev/sdk";

export const processHeyDigitalBatch = task({
  id: "process-heydigital-batch",
  retry: { maxAttempts: 3 },

  run: async (payload: { rows: HeyDigitalWebhookRow[] }, { ctx }) => {
    const results = [];

    for (const row of payload.rows) {
      // GATE 1: Job title qualification (always)
      const { score, qualification } = await qualifyJobTitle(row.jobTitle);
      if (score < 5) {
        await pushToSupabase(row, { status: "disqualified_title" });
        continue; // 35% stop here
      }

      // PARALLEL: employment + company enrichment
      const [empResult, companyResult] = await Promise.all([
        verifyEmployment(row.linkedinUrl),      // Apify harvestapi
        enrichCompany(row.companyName),          // Serper + Firecrawl
      ]);

      // GATE 2: Geo + headcount
      const country = COUNTRY_LOOKUP[companyResult.location]; // free
      if (!TARGET_COUNTRIES.has(country) || companyResult.employees < 10 || companyResult.employees > 300) {
        await pushToSupabase(row, { status: "disqualified_geo" });
        continue; // 25% stop here
      }

      // GATE 3: Competitor analysis
      const { isProspect } = await analyzeCompetitor(companyResult);
      if (!isProspect) {
        await pushToSupabase(row, { status: "disqualified_competitor" });
        continue; // 10% stop here
      }

      // EMAIL WATERFALL
      const email = await findEmail({
        fullName: row.firstName + " " + row.lastName,
        domain: companyResult.domain,
        linkedinUrl: row.linkedinUrl,
      }); // AI Ark -> Blitz -> Kitt -> Icypeas -> LeadMagic

      // CLEANING
      const cleanFirstName = row.firstName.split(" ").length > 1
        ? row.firstName.split(" ")[0].replace(/[^a-zA-Z]/g, "")
        : row.firstName;
      const cleanCompany = await cleanCompanyName(companyResult.companyName);

      // COLD EMAIL (merged call, not two)
      const { firstLine, companyType } = await personalizeEmail({
        authorName: resolveAuthor(row.postAuthorLinkedinUrl),
        postText: row.postText,
        companyName: cleanCompany,
        companyDescription: companyResult.description,
        products: companyResult.products,
      });

      // PUSH TO ALL DESTINATIONS
      await Promise.all([
        pushToCUL({ firstName: cleanFirstName, lastName: row.lastName, email, company: cleanCompany, ... }),
        pushToBison({ email, campaignId: process.env.CAMPAIGN_ID }),
        pushToHeyReach({ firstName: cleanFirstName, lastName: row.lastName, linkedinUrl: row.linkedinUrl, ... }),
        pushToSupabase(row, { status: "qualified", firstLine, companyType, email }),
      ]);
    }
  },
});

// Trigger: HeyDigital webhook -> batch of 20 rows
export const heydigitalWebhook = batch({
  id: "heydigital-webhook",
  trigger: httpEndpoint(),
  batch: { maxItems: 20, maxWaitMs: 5000 },
  run: async (items, { ctx }) => {
    await processHeyDigitalBatch.trigger({ rows: items });
  },
});