Enrichment Pipeline

Goal

Enrich brain pages from external APIs with tiered spend -- full pipeline for key people, light touch for passing mentions, raw data preserved for auditability.

What the User Gets

Without this: brain pages are thin shells with only what the user manually typed, API calls are wasted on nobodies, and enrichment data vanishes after the agent session ends. With this: key people have rich, multi-source portraits; spend scales to importance; raw API responses are preserved for re-processing; and cross-references connect the entire graph.

Implementation

gbrain ships both halves of this: gbrain enrich is the batch enrichment primitive (finds thin pages and enriches at scale), and the enrich skill (skills/enrich/) is the agent-driven page-at-a-time workflow. The pipeline below is the pattern they implement — use it to customize or extend.

on enrich(entity, trigger):
    # trigger: meeting mention, email thread, social interaction, user request

    # Step 1: Identify entities from the incoming signal
    entities = extract_entities(signal)
    #   people names, company names, associations

    # Step 2: Check brain state -- UPDATE or CREATE path?
    for entity in entities:
        existing = gbrain search "{entity.name}"
        if existing:
            page = gbrain get <entity_slug>
            path = "UPDATE"
        else:
            path = "CREATE"

    # Step 3: Determine tier -- scale spend to importance
    tier = classify_tier(entity):
        # Tier 1 (10-15 API calls): key people, inner circle, business partners,
        #         portfolio companies. Full pipeline, ALL data sources.
        # Tier 2 (3-5 API calls): notable people, occasional interactions.
        #         Web search + social + brain cross-reference.
        # Tier 3 (1-2 API calls): minor mentions, everyone else worth tracking.
        #         Brain cross-reference + social lookup if handle known.

    # Step 4: Run external lookups (priority order, stop when enough signal)
    data = {}
    data["brain"] = gbrain search "{entity.name}"          # Always first (free)
    if tier <= 2:
        data["web"] = brave_search("{entity.name}")        # Background, press, talks
    if tier <= 2:
        data["twitter"] = twitter_lookup(entity.handle)    # Beliefs, building, network
    if tier == 1:
        data["linkedin"] = crustdata_enrich(entity.name)   # Career, connections
        data["research"] = happenstance_research(entity)   # Career arcs, web presence
        data["funding"] = captain_api(entity.company)      # Funding, valuation, team
        data["meetings"] = circleback_search(entity.name)  # Transcript search
        data["contacts"] = google_contacts(entity.email)   # Contact data

    # Step 5: Store raw data (auditable, re-processable)
    gbrain call put_raw_data \
        '{"slug": "<entity_slug>", "data": {"sources": {"crustdata": {"fetched_at": "...", "data": {...}}, ...}}}'
    # Overwrite on re-enrichment, don't append

    # Step 6: Write to brain page
    if path == "CREATE":
        gbrain put <entity_slug> --content "<compiled_truth_from_all_sources>"
        gbrain timeline-add <entity_slug> {date} "Page created via enrichment"
    elif path == "UPDATE":
        # Append timeline, update compiled truth ONLY if materially new
        gbrain timeline-add <entity_slug> {date} "Enriched: {new_signal}"
        # Flag contradictions -- don't silently resolve them

    # Step 7: Cross-reference the graph
    gbrain link <person_slug> <company_slug>       # person -> company
    gbrain link <company_slug> <person_slug>       # company -> person
    gbrain link <person_slug> <deal_slug>          # person -> deal
    # Every entity page links to every other entity page that references it

# People page sections: use the person-page structure from compiled-truth.md
# (Executive Summary, State, What They Believe, ... Timeline) -- that doc is
# the single home for the section taxonomy. Enrichment can add texture
# sections on top (What Motivates Them, Hobby Horses, Open Threads).
# Facts are table stakes. TEXTURE is the value.

# Extract texture, not just facts:
#   Opinion expressed?        -> What They Believe
#   Building or shipping?     -> What They're Building
#   Emotion expressed?        -> What Makes Them Tick
#   Who did they engage with? -> Network / Relationship
#   Recurring topic?          -> Hobby Horses
#   Committed to something?   -> Open Threads
#   Energy level?             -> Trajectory

Tricky Spots

Don't overwrite human-written assessments. If the user wrote an Assessment section with their own read on someone, API enrichment NEVER overwrites it. API data goes into State, Contact, Timeline. The user's assessment is sacrosanct.
Don't re-enrich the same page more than once per week. Check put_raw_data timestamps before running the pipeline again. Enrichment is expensive and data doesn't change that fast.
LinkedIn connection count < 20 means wrong person. Crustdata sometimes returns a different person with the same name. If the LinkedIn profile has fewer than 20 connections, it's almost certainly a false match. Discard it.
X/Twitter is the most underrated data source. When you have someone's handle, their tweets reveal beliefs, what they're building, hobby horses, network (reply patterns), and trajectory (posting frequency, tone shifts). This is richer than LinkedIn for "What They Believe" and "What Makes Them Tick."
Cross-references are not optional. After enriching a person, update their company page. After enriching a company, update founder pages. An enriched page without cross-links is a dead end in the graph.

How to Verify

Enrich a Tier 1 person. Run gbrain get <slug> and confirm the page has Executive Summary, State, What They Believe, Contact, and Timeline sections populated from multiple sources.
Run gbrain call get_raw_data '{"slug": "<slug>"}'. Confirm raw API responses are stored with sources.{provider}.fetched_at timestamps.
Run gbrain call get_links '{"slug": "<slug>"}'. Confirm cross-reference links exist to the person's company page, deal pages, and related entities.
Check a page that was enriched AND has a user-written Assessment. Confirm the Assessment section was preserved, not overwritten by API data.
Try to re-enrich the same person. Confirm the system checks the fetched_at timestamp and skips if less than a week old.

Part of the GBrain Skillpack. See also: Compiled Truth for the person-page section taxonomy, Spend Controls for gbrain's own embedding/LLM spend gates.

enrichment pipeline