fix(credentials): Splunk password sign-in, Snowflake key pair in the CLI, Apollo OAuth people search, and one short SailPoint throttle wait

Author: waleedlatif1

apps/docs/content/docs/integrations/splunk.mdx

@@ -10,11 +10,11 @@ import { ReferenceDetails } from "@/components/ui/reference-details"
 
 
 {/* MANUAL-CONTENT-START:intro */}
-Use [Splunk](https://www.splunk.com/) with a bearer token to search logs, metrics, and events in Splunk Enterprise or Splunk Cloud. **Run Search** returns a synchronous result; for longer searches, create a job, inspect its status, page through its results, and cancel it when needed.
+Use [Splunk](https://www.splunk.com/) with an authentication token or a username and password to search logs, metrics, and events in Splunk Enterprise or Splunk Cloud. **Run Search** returns a synchronous result; for longer searches, create a job, inspect its status, page through its results, and cancel it when needed.
 
 Saved-search actions list, read, and dispatch existing searches. Alert actions inspect fired alerts and their instances. Index and app lookups help identify the data and configuration available to the credential.
 
-Connect Splunk once as a Splunk credential: create an authentication token in Splunk Web under **Settings > Tokens** (token authentication must be turned on), and enter the management URL with its port, such as `https://splunk.example.com:8089`. Sim checks the token by reading the user it authenticates as. The management URL must be reachable from Sim over https with a certificate from a public certificate authority; Sim never skips certificate verification, so a management port serving Splunk's default self-signed certificate cannot be saved. Workflows built with the earlier Splunk block keep the URL, token, or username and password they entered in the block.
+Connect Splunk once as a Splunk credential, and enter the management URL with its port, such as `https://splunk.example.com:8089`. To sign in with a token, create an authentication token in Splunk Web under **Settings > Tokens** (token authentication must be turned on) and leave the username blank. Token authentication is off by default on Splunk Enterprise, so you can instead enter a username and its password, which Sim sends with HTTP Basic authentication. Sim checks either sign-in by reading the user it authenticates as. The management URL must be reachable from Sim over https with a certificate from a public certificate authority; Sim never skips certificate verification, so a management port serving Splunk's default self-signed certificate cannot be saved. Workflows built with the earlier Splunk block keep the URL, token, or username and password they entered in the block.
 {/* MANUAL-CONTENT-END */}
 
 ## Use the block [#usage-instructions]

apps/sim/lib/credentials/client-credential-accounts/minters/sailpoint.test.ts

@@ -113,7 +113,50 @@ describe('SailPoint API client token exchange', () => {
     })
   })
 
-  it('returns the first throttled answer with its Retry-After instead of retrying it', async () => {
+  it('waits out one short throttle and mints on the retry', async () => {
+    vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout'] })
+    try {
+      mockFetch
+        .mockResolvedValueOnce(
+          jsonResponse({ error: 'rate limited' }, { status: 429, headers: { 'Retry-After': '2' } })
+        )
+        .mockResolvedValueOnce(jsonResponse({ access_token: 'sp-access', expires_in: 749 }))
+
+      const minted = mintSailPointServiceAccountToken(FIELDS)
+      await vi.advanceTimersByTimeAsync(1_999)
+      expect(mockFetch).toHaveBeenCalledOnce()
+      await vi.advanceTimersByTimeAsync(1)
+
+      await expect(minted).resolves.toMatchObject({ accessToken: 'sp-access' })
+      expect(mockFetch).toHaveBeenCalledTimes(2)
+    } finally {
+      vi.useRealTimers()
+    }
+  })
+
+  it('retries a short throttle only once and returns the second answer with its Retry-After', async () => {
+    vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout'] })
+    try {
+      mockFetch.mockImplementation(async () =>
+        jsonResponse({ error: 'rate limited' }, { status: 429, headers: { 'Retry-After': '1' } })
+      )
+
+      const minted = mintSailPointServiceAccountToken(FIELDS)
+      const settled = expect(minted).rejects.toMatchObject({
+        code: 'provider_unavailable',
+        status: 429,
+        retryAfterMs: 1_000,
+      })
+      await vi.advanceTimersByTimeAsync(10_000)
+
+      await settled
+      expect(mockFetch).toHaveBeenCalledTimes(2)
+    } finally {
+      vi.useRealTimers()
+    }
+  })
+
+  it('returns a long throttle with its Retry-After instead of retrying it', async () => {
     mockFetch.mockImplementation(async () =>
       jsonResponse({ error: 'rate limited' }, { status: 429, headers: { 'Retry-After': '120' } })
     )

apps/sim/lib/credentials/client-credential-accounts/minters/sailpoint.ts

@@ -20,6 +20,8 @@ import {
 const STEP = 'sailpoint_token_mint'
 /** Bounds the whole exchange. */
 const TOKEN_EXCHANGE_TIMEOUT_MS = 30_000
+/** The longest throttle the mint waits out in place, leaving both requests well inside the exchange budget. */
+const MAX_IN_PLACE_THROTTLE_WAIT_MS = 5_000
 
 /**
  * Mints a SailPoint Identity Security Cloud access token for an API client or
@@ -32,8 +34,10 @@ const TOKEN_EXCHANGE_TIMEOUT_MS = 30_000
  * from `expires_in`; the credential cache re-mints before it lapses.
  *
  * The token response names no identity, so the connect-time check is the mint.
- * A 429 is returned at once with its `Retry-After` rather than retried here, so
- * the credential resolver's throttling backoff waits as long as SailPoint asks.
+ * A 429 asking for at most {@link MAX_IN_PLACE_THROTTLE_WAIT_MS} is retried once
+ * here, so a brief throttle does not fail a cold mint or a connect-time check.
+ * A longer one, or a second 429, is returned at once with its `Retry-After`, and
+ * the credential resolver replays it for that long (within its own bounds).
  *
  * @see https://developer.sailpoint.com/docs/api/authentication
  */
@@ -60,7 +64,7 @@ export async function mintSailPointServiceAccountToken(
       fields.clientId,
       clientSecret,
       options?.signal ? AbortSignal.any([options.signal, timeout]) : timeout,
-      { maxThrottleRetries: 0 }
+      { maxThrottleRetries: 1, maxThrottleWaitMs: MAX_IN_PLACE_THROTTLE_WAIT_MS }
     )
   } catch (error) {
     options?.signal?.throwIfAborted()

apps/sim/lib/credentials/token-service-accounts/descriptors.ts

@@ -113,6 +113,13 @@ export interface TokenServiceAccountDescriptor {
    * token route surfaces this so tool header builders can switch schemes.
    */
   authStyle?: 'bearer' | 'x-api-token'
+  /**
+   * The pasted secret is a password whenever the connection also stores a username in
+   * `providerWorkspaceId` (Splunk, whose token sign-in is off by default): such a credential
+   * resolves with `authStyle: 'basic'` so its tools send HTTP Basic. One saved without a username
+   * resolves exactly as a pasted token.
+   */
+  passwordWithUsername?: true
   /**
    * A client-credentials grant the provider offers beside the pasted token. A credential saved
    * with `authMethod: 'client_credentials'` stores the client ID and secret (and `domain`, when
@@ -2732,10 +2739,17 @@ export const TOKEN_SERVICE_ACCOUNT_DESCRIPTORS: Record<
     fields: [
       {
         id: 'apiToken',
-        label: 'Authentication token',
-        placeholder: 'Paste a Splunk authentication token',
+        label: 'Authentication token or password',
+        placeholder: 'Paste a Splunk authentication token, or the password',
         secret: true,
       },
+      {
+        id: 'providerWorkspaceId',
+        label: 'Username',
+        placeholder: 'Only to sign in with a password',
+        secret: false,
+        optional: true,
+      },
       {
         id: 'domain',
         label: 'Management URL',
@@ -2744,11 +2758,12 @@ export const TOKEN_SERVICE_ACCOUNT_DESCRIPTORS: Record<
         url: true,
       },
     ],
+    passwordWithUsername: true,
     docsUrl: 'https://docs.sim.ai/integrations/splunk',
     helpText:
-      'Create a token in Splunk Web under Settings > Tokens, and enter the management URL with its port (8089 by default). The URL must be reachable from Sim over https with a certificate from a public certificate authority; Sim does not skip certificate checks.',
+      'Create a token in Splunk Web under Settings > Tokens and leave the username blank, or enter a username and its password when token authentication is turned off. Enter the management URL with its port (8089 by default). The URL must be reachable from Sim over https with a certificate from a public certificate authority; Sim does not skip certificate checks.',
     invalidCredentialsHelp:
-      'Splunk rejected this token. Check that token authentication is turned on, that the token is enabled and not expired, and that the management URL points at the same deployment.',
+      'Splunk rejected these credentials. For a token, check that token authentication is turned on and that the token is enabled and not expired; for a password, check the username and password. Also check that the management URL points at the same deployment.',
   },
   [CBINSIGHTS_SERVICE_ACCOUNT_PROVIDER_ID]: {
     providerId: CBINSIGHTS_SERVICE_ACCOUNT_PROVIDER_ID,

apps/sim/lib/credentials/token-service-accounts/validators/token-checks.test.ts

@@ -879,6 +879,43 @@ describe('Splunk saved authentication token verification', () => {
       validateSplunkServiceAccount({ apiToken: 'any', domain: 'https://splunk.example.com:8089' })
     ).rejects.toMatchObject({ code: 'provider_unavailable' })
   })
+
+  it('signs a username and password in with HTTP Basic on the same check, and stores the username', async () => {
+    reply(jsonResponse({ entry: [{ content: { username: 'svc-sim' } }] }))
+
+    await expect(
+      validateSplunkServiceAccount({
+        apiToken: 'p@ss word',
+        providerWorkspaceId: ' svc-sim ',
+        domain: 'https://splunk.example.com:8089',
+      })
+    ).resolves.toMatchObject({
+      normalizedDomain: 'https://splunk.example.com:8089',
+      normalizedWorkspaceId: 'svc-sim',
+    })
+    expect(secureFetch).toHaveBeenCalledWith(
+      'https://splunk.example.com:8089/services/authentication/current-context?output_mode=json',
+      '203.0.113.7',
+      expect.objectContaining({
+        headers: expect.objectContaining({
+          Authorization: `Basic ${Buffer.from('svc-sim:p@ss word').toString('base64')}`,
+        }),
+      })
+    )
+  })
+
+  it('refuses a username HTTP Basic cannot carry without sending it', async () => {
+    reply(jsonResponse({ entry: [{ content: { username: 'svc-sim' } }] }))
+
+    await expect(
+      validateSplunkServiceAccount({
+        apiToken: 'password',
+        providerWorkspaceId: 'svc:sim',
+        domain: 'https://splunk.example.com:8089',
+      })
+    ).rejects.toMatchObject({ code: 'invalid_credentials' })
+    expect(secureFetch).not.toHaveBeenCalled()
+  })
 })
 
 describe('CB Insights saved API client verification', () => {

apps/sim/lib/credentials/token-service-accounts/validators/token-checks.ts

@@ -1270,18 +1270,30 @@ export const validateKalshiServiceAccount = tokenCheckValidator({
 
 /**
  * Splunk's `authentication/current-context` returns the user a request authenticates as and
- * needs no capability, so any valid authentication token reads it; a refused token answers 401.
- * The management URL is one the user chose, so the check runs through the guarded egress, and
- * an answer without a user context (a proxy or login page) is not taken as proof.
+ * needs no capability, so any valid authentication token, or username and password sent as HTTP
+ * Basic the way the tools send them, reads it; refused credentials answer 401. A username is the
+ * Basic username, so one with a colon is refused unsent. The management URL is one the user
+ * chose, so the check runs through the guarded egress, and an answer without a user context (a
+ * proxy or login page) is not taken as proof.
  * https://help.splunk.com/en/splunk-enterprise/leverage-rest-apis/rest-api-reference/10.6/access-endpoints/access-endpoint-descriptions
  */
 export const validateSplunkServiceAccount = tokenCheckValidator({
   serviceLabel: 'Splunk',
   domain: ({ domain }) => normalizeHttpBaseUrl(domain),
+  workspaceId: ({ providerWorkspaceId }) => {
+    const username = (providerWorkspaceId ?? '').trim()
+    if (!username) return undefined
+    if (!/^[^:\x00-\x1f\x7f]+$/.test(username)) throw new Error('invalid username')
+    return username
+  },
   egress: 'configuredEndpoint',
-  request: ({ apiToken, domain = '' }) => ({
+  request: ({ apiToken, domain = '', providerWorkspaceId }) => ({
     url: buildSplunkUrl({ baseUrl: domain }, '/authentication/current-context'),
-    headers: buildSplunkHeaders({ baseUrl: domain, authToken: apiToken }),
+    headers: buildSplunkHeaders(
+      providerWorkspaceId
+        ? { baseUrl: domain, username: providerWorkspaceId, password: apiToken }
+        : { baseUrl: domain, authToken: apiToken }
+    ),
   }),
   acceptsBody: (body) => {
     const [entry] = toArray(toRecord(body).entry)

apps/sim/lib/internal/sailpoint/client.ts

@@ -184,8 +184,10 @@ async function readBoundedBody(
  * Exchanges API client credentials for an access token at a tenant's token
  * endpoint (`client_credentials` grant, credentials in the form body), with
  * redirects refused and up to four backoff retries on 429 (`maxThrottleRetries`).
- * A refusal throws {@link SailPointAuthError} with the provider's status,
- * message, and `Retry-After`, even when its error body cannot be read.
+ * A 429 whose wait (its `Retry-After`, or the backoff when it names none)
+ * exceeds `maxThrottleWaitMs` is not retried. A refusal throws
+ * {@link SailPointAuthError} with the provider's status, message, and
+ * `Retry-After`, even when its error body cannot be read.
  *
  * @see https://developer.sailpoint.com/docs/api/authentication
  */
@@ -194,7 +196,10 @@ export async function requestSailPointToken(
   clientId: string,
   clientSecret: string,
   signal: AbortSignal,
-  { maxThrottleRetries = MAX_FETCH_RETRIES }: { maxThrottleRetries?: number } = {}
+  {
+    maxThrottleRetries = MAX_FETCH_RETRIES,
+    maxThrottleWaitMs = Number.POSITIVE_INFINITY,
+  }: { maxThrottleRetries?: number; maxThrottleWaitMs?: number } = {}
 ): Promise<{ token: string; expiresInSeconds: number }> {
   let attempt = 0
 
@@ -216,11 +221,14 @@ export async function requestSailPointToken(
       signal,
     })
 
-    if (response.status === 429 && attempt < maxThrottleRetries) {
-      const retryAfterMs = parseRetryAfter(response.headers.get('retry-after'))
+    const throttleWaitMs =
+      response.status === 429 && attempt < maxThrottleRetries
+        ? backoffWithJitter(attempt + 1, parseRetryAfter(response.headers.get('retry-after')))
+        : undefined
+    if (throttleWaitMs !== undefined && throttleWaitMs <= maxThrottleWaitMs) {
       await consumeOrCancelBody(response, DEFAULT_MAX_ERROR_BODY_BYTES)
       attempt += 1
-      await interruptibleSleep(backoffWithJitter(attempt, retryAfterMs), signal)
+      await interruptibleSleep(throttleWaitMs, signal)
       signal?.throwIfAborted()
       continue
     }

apps/sim/lib/oauth/credential-service.test.ts

@@ -1480,6 +1480,38 @@ describe('token provider credentials saved in client-credentials mode', () => {
     })
   })
 
+  it('marks a saved Splunk username and password as basic, and a saved token as before', async () => {
+    storeBlob('splunk-login', {
+      type: 'token_service_account',
+      providerId: 'splunk-service-account',
+      apiToken: 'saved-password',
+      providerWorkspaceId: 'svc-sim',
+      domain: 'https://splunk.example.com:8089',
+    })
+    await expect(
+      resolveServiceAccountToken('splunk-login', 'splunk-service-account')
+    ).resolves.toEqual({
+      accessToken: 'saved-password',
+      domain: 'https://splunk.example.com:8089',
+      providerWorkspaceId: 'svc-sim',
+      authStyle: 'basic',
+    })
+
+    storeBlob('splunk-token', {
+      type: 'token_service_account',
+      providerId: 'splunk-service-account',
+      apiToken: 'splunk-token',
+      domain: 'https://splunk.example.com:8089',
+    })
+    await expect(
+      resolveServiceAccountToken('splunk-token', 'splunk-service-account')
+    ).resolves.toEqual({
+      accessToken: 'splunk-token',
+      domain: 'https://splunk.example.com:8089',
+      providerWorkspaceId: undefined,
+    })
+  })
+
   it('still returns a pasted Shopify token as is', async () => {
     storeBlob(
       'shopify-pasted',

apps/sim/lib/oauth/credential-service.ts

@@ -612,7 +612,7 @@ export interface ServiceAccountTokenResult {
    * Sim-signed OIDC token a keyless Snowflake credential presents through
    * workload identity federation, or a JWT a key-pair Snowflake credential
    * signs with its private key, or `basic` for an HTTP Basic credential (a
-   * Zendesk API token sign-in), or `bearer` for a token minted for a provider
+   * Zendesk API token sign-in, or a Splunk username and password), or `bearer` for a token minted for a provider
    * whose pasted mode is not a bearer token (ServiceNow client credentials).
    * Absent means Bearer, except to tools whose pasted mode says otherwise;
    * OAuth credentials never carry it.
@@ -1117,12 +1117,17 @@ export async function resolveServiceAccountToken(
     if ('authMethod' in secret) {
       return resolveTokenClientCredentialToken(credentialId, secret, stored, options)
     }
-    const descriptorAuthStyle = getTokenServiceAccountDescriptor(providerId)?.authStyle
+    const descriptor = getTokenServiceAccountDescriptor(providerId)
+    const descriptorAuthStyle = descriptor?.authStyle
     return {
       accessToken: secret.apiToken,
       domain: secret.domain,
       providerWorkspaceId: secret.providerWorkspaceId,
-      ...(descriptorAuthStyle === 'x-api-token' ? { authStyle: descriptorAuthStyle } : {}),
+      ...(descriptor?.passwordWithUsername && secret.providerWorkspaceId
+        ? { authStyle: 'basic' as const }
+        : descriptorAuthStyle === 'x-api-token'
+          ? { authStyle: descriptorAuthStyle }
+          : {}),
     }
   }
   if (providerId && isClientCredentialAccountProviderId(providerId)) {