Skip to content

Commit ba998b6

Browse files
authored
fix(enrichments): reduce cost, increase accuracy (#8936)
* fix(enrichments): verified-first waterfalls, billing fixes, and an error state for outages - Work email: hold unverified hits (Hunter accept_all, Wiza risky, Dropcontact non-nominative) as a fallback while later providers look for a verified one; the held hit still fills the cell if none verifies. Ask Prospeo for verified emails only, keep personal addresses out of Wiza results, and move People Data Labs (unverified, $0.28 per hit) to the end, skipping it once a fallback is held. Order is otherwise unchanged. - Phone: skip PDL on a bare name (not a valid match), pass the LinkedIn profile to PDL, prefer its mobile, and ask Prospeo for verified mobiles only (no billed person match without one). - Email verification: unverified/error verdicts fall through; bill nothing for ZeroBounce unknown or MillionVerifier catch-all/unknown, which those vendors do not charge for. - Table cells: when every provider errored, mark the cell errored instead of completing it blank, so an outage no longer silently finalizes empty rows. * fix(enrichments): address review — blank outputs on an all-error run, LeadMagic verified check, toArray * fix(enrichments): accept both Dropcontact verified-address labels
1 parent 5313021 commit ba998b6

11 files changed

Lines changed: 304 additions & 105 deletions

File tree

‎.agents/skills/add-enrichment/SKILL.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ Because enrichments run on Sim's hosted keys by default, **every provider tool y
2222

2323
## Architecture (what you're plugging into)
2424

25-
- **`enrichments/types.ts`** — `EnrichmentConfig { id, name, description, icon, inputs, outputs, providers }` and `EnrichmentProvider { id, label, toolId, buildParams, projectFailure, mapOutput }` — `toolProvider` fills `projectFailure` with the standard HTTP projection; override it only for a provider whose failure shape is nonstandard (see `enrichments/provider-failures/`). Providers are **plain data** (no `@/tools` import) so the catalog stays client-safe.
25+
- **`enrichments/types.ts`** — `EnrichmentConfig { id, name, description, icon, inputs, outputs, providers }` and `EnrichmentProvider { id, label, toolId, buildParams, projectFailure, mapOutput, isVerified? }` — `toolProvider` fills `projectFailure` with the standard HTTP projection; override it only for a provider whose failure shape is nonstandard (see `enrichments/provider-failures/`). Providers are **plain data** (no `@/tools` import) so the catalog stays client-safe.
2626
- **`enrichments/providers.ts`** — `toolProvider(...)` (typed passthrough) plus shared input helpers: `str(v)`, `normalizeDomain(v)`, `firstNonEmpty(arr)`, `splitName(fullName)`, and `projectEnrichmentProviderFailure`.
2727
- **`enrichments/run.ts`** — the server-only cascade runner. Calls `executeTool(provider.toolId, { ...params, _context: { workspaceId, userId } })`, accumulates hosted-key cost, returns the first non-empty mapped result. **You do not edit this** — it works for any registry entry.
2828
- **`enrichments/registry.ts`** — `ENRICHMENT_REGISTRY` / `ALL_ENRICHMENTS` / `getEnrichment`. Register new entries here.
@@ -37,7 +37,9 @@ For each output the enrichment produces, decide which existing tool provides it.
3737
- Its `params` accept what you can derive from table columns (read the tool's `params`).
3838
- Its `outputs` / `transformResponse` actually expose the field you need (read the real output shape — don't assume).
3939

40-
Order providers **cheapest / most-likely-to-hit first**; the cascade stops at the first non-empty result. Apollo and LinkedIn APIs are not hosted-safe (ToS) — never call them as providers.
40+
Order providers by **accuracy first, then price per hit**. Order decides which answer wins when two providers would disagree, so a cheaper but less accurate provider ahead of a better one trades quality for cost; reorder for price only with benchmark evidence. Among similarly accurate providers, put the ones that bill on a miss or charge a high price per hit after the cheaper, free-on-miss ones (check each vendor's billing docs and the tool's `hosting.pricing`); never move a more accurate provider behind a less accurate one to save cost. The cascade stops at the first non-empty, verified result. Apollo and LinkedIn APIs are not hosted-safe (ToS) — never call them as providers.
41+
42+
Prefer a vendor's own verified-only option (e.g. Prospeo `only_verified_email`) when it turns an unverified result into a free miss. Otherwise, when a provider can return an unconfirmed hit (a catch-all guess, a `risky` status), declare `isVerified(output)`: an unconfirmed hit is held while later providers look for a confirmed one, and fills the cell only if none does. `isVerified: false` marks a provider that never confirms its hits (a database record); it runs only while nothing is held.
4143

4244
## Step 2: Verify hosted-key support — chain to `/add-hosted-key` if missing
4345

@@ -136,7 +138,8 @@ export const ENRICHMENT_REGISTRY: EnrichmentRegistry = {
136138

137139
- [ ] Each output mapped to a real tool field (verified against the tool's `params`/`outputs`)
138140
- [ ] **Every provider tool has a `hosting` block — ran `/add-hosted-key` for any that didn't**
139-
- [ ] Providers ordered cheapest / most-likely-first; Apollo/LinkedIn not used
141+
- [ ] Providers ordered accuracy-first, then price per hit (billing on a miss checked against vendor docs); Apollo/LinkedIn not used
142+
- [ ] Providers that can return an unconfirmed hit declare `isVerified`
140143
- [ ] Enrichment file is client-safe (no `@/tools` import); uses `toolProvider` + shared helpers
141144
- [ ] `buildParams` returns `null` on insufficient inputs; `mapOutput` returns `null` on a miss
142145
- [ ] Registered in `enrichments/registry.ts`

‎apps/sim/background/enrichment-capability-subject.test.ts‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ import {
77
billingUsageGateCacheMock,
88
billingUsageGateCacheMockFns,
99
} from '@sim/testing/mocks/billing-usage-gate-cache.mock'
10+
import {
11+
billingUsageLogMock,
12+
billingUsageLogMockFns,
13+
} from '@sim/testing/mocks/billing-usage-log.mock'
1014
import { executeWorkflowMock } from '@sim/testing/mocks/execute-workflow.mock'
1115
import { networkConfigMock, networkConfigMockFns } from '@sim/testing/mocks/network-config.mock'
1216
import { tableEventsMock } from '@sim/testing/mocks/table-events.mock'
@@ -60,6 +64,7 @@ vi.mock('@/enrichments/run', () => ({
6064
}))
6165
vi.mock('@/lib/billing/core/billing-attribution', () => billingAttributionMock)
6266
vi.mock('@/lib/billing/core/usage-gate-cache', () => billingUsageGateCacheMock)
67+
vi.mock('@/lib/billing/core/usage-log', () => billingUsageLogMock)
6368
vi.mock('@/lib/table/rows/secret-provenance', () => tableRowsSecretProvenanceMock)
6469
vi.mock('@/executor/utils/resolved-secret-trace-registry', () => ({
6570
ResolvedSecretTraceRegistry: class {
@@ -112,7 +117,7 @@ const GROUP = {
112117
type: 'enrichment' as const,
113118
enrichmentId: 'company-lookup',
114119
workflowId: '',
115-
outputs: [{ columnName: 'col-out', blockId: '', path: '' }],
120+
outputs: [{ columnName: 'col-out', blockId: '', path: '', outputId: 'email' }],
116121
inputMappings: [{ columnName: 'col-in', inputName: 'domain' }],
117122
}
118123

@@ -288,4 +293,24 @@ describe('enrichment cell capability subject', () => {
288293
await runRowCascadeLoop(payload(null, 'legacy-trigger-user') as never)
289294
expect(gatedUserId()).toBeNull()
290295
})
296+
297+
/** A completed cell is final on the auto path, so an outage must not complete blank. */
298+
it('marks the cell errored when every provider errored', async () => {
299+
mocks.runEnrichment.mockResolvedValue({
300+
result: {},
301+
cost: 0.02,
302+
error: 'Rate limit exceeded',
303+
detail: { providers: [] },
304+
})
305+
306+
await runRowCascadeLoop(payload('acting-user', 'acting-user') as never)
307+
308+
const terminal = mocks.writeWorkflowGroupState.mock.calls.at(-1) as unknown[]
309+
const write = terminal[1] as { executionState: Record<string, unknown>; dataPatch?: unknown }
310+
const state = write.executionState
311+
expect(state.status).toBe('error')
312+
expect(state.error).toBe('Rate limit exceeded')
313+
expect(write.dataPatch).toEqual({ 'col-out': '' })
314+
expect(billingUsageLogMockFns.mockRecordUsage).toHaveBeenCalledTimes(1)
315+
})
291316
})

‎apps/sim/background/workflow-column-execution.ts‎

Lines changed: 31 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -681,27 +681,29 @@ async function runWorkflowAndWriteTerminal(
681681
await enrichmentRegistry.importCrossingProvenance(inputProvenance, enrichInputs, {
682682
trusted: true,
683683
})
684-
const { result, cost, detail } = await withResourceOutboundScope({ workspaceId }, () =>
685-
runEnrichment(enrichment, enrichInputs, {
686-
tableId,
687-
rowId,
688-
workspaceId,
689-
/**
690-
* The person who asked, not who pays. `triggeredByUserId` is an
691-
* attribution: for a workspace-API-key run it names the workspace's
692-
* billing owner, and running that bystander's tool denylist against
693-
* an actorless request is wrong in both directions — it fails cells
694-
* nobody meant to govern, and it skips the denylist for the person
695-
* who actually triggered one. The governed subject is carried
696-
* separately from the dispatch. `null` means no per-tool gate
697-
* applies, which is the documented behavior for an actorless run —
698-
* stated, because the field is required precisely so it cannot be
699-
* skipped by omission.
700-
*/
701-
userId: payload.capabilityGovernedUserId ?? null,
702-
signal: attemptSignal,
703-
resolvedSecretTraceRegistry: enrichmentRegistry,
704-
})
684+
const { result, cost, error, detail } = await withResourceOutboundScope(
685+
{ workspaceId },
686+
() =>
687+
runEnrichment(enrichment, enrichInputs, {
688+
tableId,
689+
rowId,
690+
workspaceId,
691+
/**
692+
* The person who asked, not who pays. `triggeredByUserId` is an
693+
* attribution: for a workspace-API-key run it names the workspace's
694+
* billing owner, and running that bystander's tool denylist against
695+
* an actorless request is wrong in both directions — it fails cells
696+
* nobody meant to govern, and it skips the denylist for the person
697+
* who actually triggered one. The governed subject is carried
698+
* separately from the dispatch. `null` means no per-tool gate
699+
* applies, which is the documented behavior for an actorless run —
700+
* stated, because the field is required precisely so it cannot be
701+
* skipped by omission.
702+
*/
703+
userId: payload.capabilityGovernedUserId ?? null,
704+
signal: attemptSignal,
705+
resolvedSecretTraceRegistry: enrichmentRegistry,
706+
})
705707
)
706708

707709
// An abort during the cascade must not be recorded as a completed cell.
@@ -720,7 +722,7 @@ async function runWorkflowAndWriteTerminal(
720722

721723
/**
722724
* Record the triggerer or system fallback as actor while charging the
723-
* exact workspace payer. Billing failures do not fail a successful cell.
725+
* exact workspace payer. Billing failures do not fail the cell.
724726
*/
725727
if (cost > 0) {
726728
try {
@@ -753,28 +755,31 @@ async function runWorkflowAndWriteTerminal(
753755

754756
// Write every output column: the result value when present, else clear
755757
// it. A partial/empty result must blank the columns it didn't fill so a
756-
// re-run that finds less than before doesn't leave stale values.
758+
// re-run that finds less than before doesn't leave stale values, and a
759+
// failed run must not leave old values for dependent groups to consume.
757760
const dataPatch: RowData = {}
758761
for (const out of group.outputs) {
759762
if (!out.outputId) continue
760763
const value = result[out.outputId]
761764
dataPatch[out.columnName] =
762765
value === undefined || value === null ? '' : (value as RowData[string])
763766
}
767+
// Every provider that ran errored (rate limit, missing hosted key, auth):
768+
// `error` rather than `completed`, so a manual re-run picks the cell up.
764769
await writeState(
765770
{
766-
status: 'completed',
771+
status: error ? 'error' : 'completed',
767772
executionId,
768773
jobId: null,
769774
workflowId: statusId,
770-
error: null,
775+
error,
771776
enrichmentDetails: detail,
772777
},
773778
dataPatch,
774779
undefined,
775780
createTableRowSecretProvenanceFromRegistry(dataPatch, enrichmentRegistry)
776781
)
777-
return 'completed'
782+
return error ? 'error' : 'completed'
778783
} catch (err) {
779784
await writeState({
780785
status: 'error',

‎apps/sim/enrichments/email-verification/email-verification.ts‎

Lines changed: 20 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,26 @@ import { ShieldCheck } from '@sim/emcn/icons'
22
import { str, toolProvider } from '@/enrichments/providers'
33
import type { EnrichmentConfig } from '@/enrichments/types'
44

5+
/**
6+
* Verdicts that say nothing about the mailbox; the cascade falls through to the
7+
* next verifier on these. MillionVerifier reports `unverified` (and may report
8+
* `error`) when it could not complete a check.
9+
*/
10+
const INCONCLUSIVE_STATUSES = new Set(['unknown', 'unverified', 'error'])
11+
12+
function mapVerdict(output: Record<string, unknown>) {
13+
const status = str(output.status)
14+
if (!status || INCONCLUSIVE_STATUSES.has(status)) return null
15+
return { status, deliverable: output.deliverable === true }
16+
}
17+
518
/**
619
* Email Verification enrichment. Checks an email address's deliverability via a
720
* verifier waterfall — ZeroBounce first (highest coverage), then NeverBounce,
8-
* then MillionVerifier, then Enrow. A provider that returns a
9-
* definitive verdict (valid / invalid / catch_all / disposable / etc.) fills the
10-
* cell; a provider that can only return `unknown` falls through to the next so
11-
* the row gets the most confident answer available. All providers support hosted
12-
* keys.
21+
* then MillionVerifier, then Enrow. A provider that returns a definitive verdict
22+
* (valid / invalid / catch_all / disposable / etc.) fills the cell; an
23+
* inconclusive one falls through to the next so the row gets the most confident
24+
* answer available. All providers support hosted keys.
1325
*/
1426
export const emailVerificationEnrichment: EnrichmentConfig = {
1527
id: 'email-verification',
@@ -31,12 +43,7 @@ export const emailVerificationEnrichment: EnrichmentConfig = {
3143
if (!email) return null
3244
return { email }
3345
},
34-
mapOutput: (output) => {
35-
const status = str(output.status)
36-
// Fall through to the next verifier when the verdict is missing or inconclusive.
37-
if (!status || status === 'unknown') return null
38-
return { status, deliverable: output.deliverable === true }
39-
},
46+
mapOutput: mapVerdict,
4047
}),
4148
toolProvider({
4249
id: 'neverbounce',
@@ -47,11 +54,7 @@ export const emailVerificationEnrichment: EnrichmentConfig = {
4754
if (!email) return null
4855
return { email }
4956
},
50-
mapOutput: (output) => {
51-
const status = str(output.status)
52-
if (!status || status === 'unknown') return null
53-
return { status, deliverable: output.deliverable === true }
54-
},
57+
mapOutput: mapVerdict,
5558
}),
5659
toolProvider({
5760
id: 'millionverifier',
@@ -62,11 +65,7 @@ export const emailVerificationEnrichment: EnrichmentConfig = {
6265
if (!email) return null
6366
return { email }
6467
},
65-
mapOutput: (output) => {
66-
const status = str(output.status)
67-
if (!status || status === 'unknown') return null
68-
return { status, deliverable: output.deliverable === true }
69-
},
68+
mapOutput: mapVerdict,
7069
}),
7170
toolProvider({
7271
id: 'enrow',

‎apps/sim/enrichments/phone-number/phone-number.ts‎

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,11 @@ import type { EnrichmentConfig } from '@/enrichments/types'
77
/**
88
* Phone Number enrichment. Finds a contact's phone number from a full name plus
99
* any available identifiers (company domain, LinkedIn URL) via a waterfall:
10-
* People Data Labs (name match) → Wiza reveal → Findymail (LinkedIn) → Prospeo
11-
* mobile → LeadMagic (LinkedIn) → Datagma (LinkedIn). Each provider
12-
* opportunistically uses whatever identifiers the row provides and self-skips
13-
* when it has none usable, so adding more inputs widens coverage without
14-
* reordering. First phone wins; all providers support hosted keys.
10+
* People Data Labs (name + company or LinkedIn) → Wiza reveal → Findymail
11+
* (LinkedIn) → Prospeo mobile → LeadMagic (LinkedIn) → Datagma (LinkedIn). Each
12+
* provider opportunistically uses whatever identifiers the row provides and
13+
* self-skips when it has none usable, so adding more inputs widens coverage
14+
* without reordering. First phone wins; all providers support hosted keys.
1515
*/
1616
export const phoneNumberEnrichment: EnrichmentConfig = {
1717
id: 'phone-number',
@@ -31,19 +31,23 @@ export const phoneNumberEnrichment: EnrichmentConfig = {
3131
toolId: 'pdl_person_enrich',
3232
buildParams: (inputs) => {
3333
const name = str(inputs.fullName)
34-
if (!name) return null
34+
const company = normalizeDomain(inputs.companyDomain)
35+
const linkedin = str(inputs.linkedinUrl)
36+
// A bare name is not a valid PDL match and scores below min_likelihood 6.
37+
if (!name || !(company || linkedin)) return null
3538
// `required` makes PDL 404 (free) when the profile has no phone,
3639
// instead of charging a credit for a match we'd discard as a no-match.
3740
return filterUndefined({
3841
name,
39-
company: normalizeDomain(inputs.companyDomain) || undefined,
42+
company: company || undefined,
43+
profile: linkedin || undefined,
4044
min_likelihood: 6,
4145
required: 'phone_numbers OR mobile_phone',
4246
})
4347
},
4448
mapOutput: (output) => {
4549
const person = output.person as Record<string, unknown> | undefined
46-
const phone = firstNonEmpty(person?.phone_numbers) ?? str(person?.mobile_phone)
50+
const phone = str(person?.mobile_phone) || firstNonEmpty(person?.phone_numbers)
4751
return phone ? { phone } : null
4852
},
4953
}),
@@ -102,6 +106,8 @@ export const phoneNumberEnrichment: EnrichmentConfig = {
102106
full_name: fullName || undefined,
103107
company_website: companyWebsite || undefined,
104108
enrich_mobile: true,
109+
// An unverified mobile becomes a free NO_MATCH instead of a billed hit.
110+
only_verified_mobile: true,
105111
})
106112
},
107113
projectFailure: projectProspeoEnrichmentFailure,

0 commit comments

Comments
 (0)