Non-streaming /v1/completions reports finish_reason: "stop" when max_tokens was hit

Non-streaming /v1/completions reports finish_reason: "stop" even when generation was cut off by max_tokens. The chat endpoint and the streaming text path both report "length" in that case.

Repro (main @ 7e219b4, mlx-community/Qwen3-0.6B-4bit, M1 Max):

curl -s localhost:5413/v1/completions -H 'Content-Type: application/json' \
  -d '{"prompt":"Write a long story about a dragon.","max_tokens":5,"temperature":0}'
# → "finish_reason":"stop", "usage":{"completion_tokens":5,…}

Same request with "stream": true → the final chunk says "finish_reason":"length". Same prompt via /v1/chat/completions → "length".

Cause: handleTextNonStreaming takes generationTokenCount from the final .info event but ignores info.stopReason, and finish_reason is hardcoded:

SwiftLM/Sources/SwiftLM/Server.swift

Lines 3868 to 3878 in 7e219b4

    
               case .info(let info): 
        
                   completionTokenCount = info.generationTokenCount  // real count; see handleChatStreaming 
        
               case .toolCall, .rejectedToolCall: 
        
                   break 
        
               } 
        
           } 
        
           let duration = Date().timeIntervalSince(genStart) 
        
           await stats.requestFinished(tokens: completionTokenCount, duration: duration) 
        
           slot.release() 
        
           var finishReason = "stop"

handleChatNonStreaming already does the mapping a few hundred lines up (L3495-L3499), and handleTextStreaming does it too (L3813-L3817), so this is just the one path that was missed.

Why it matters: finish_reason is the only signal a client has to tell a truncated completion from a finished one. With this path always saying "stop", anything that ran out of budget looks complete.

I have a small fix with a model-free unit test and a contract-script assertion; will send it as a PR.

@HamedShams

Thanks @HamedShams — good catch, and a clear report. I checked main: handleTextNonStreaming hardcodes finishReason = "stop" and ignores info.stopReason, while handleChatNonStreaming and handleTextStreaming both map .length to "length", exactly as you describe.

Please go ahead with the PR. One thing I'll look at in review: when a stop sequence matches, the response should still say "stop", even if the token limit was reached on the same request.

PR is up: #229.
It’s stacked on #228, since that rewrote the same loop, so the diff collapses to one commit once #228 lands.

--

One thing I'll look at in review: when a stop sequence matches, the response should still say "stop", even if the token limit was reached on the same request.

On your point: a stop-sequence match still reports stop whatever ended generation, and TextCompletionFinishReasonTests covers that case (stop sequence plus .length from the generator → stop, text trimmed).

Navigation Menu

Repository navigation

Non-streaming `/v1/completions` reports `finish_reason: "stop"` when `max_tokens` was hit #227

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	case .info(let info):
	completionTokenCount = info.generationTokenCount // real count; see handleChatStreaming
	case .toolCall, .rejectedToolCall:
	break
	}
	}
	let duration = Date().timeIntervalSince(genStart)
	await stats.requestFinished(tokens: completionTokenCount, duration: duration)
	slot.release()

	var finishReason = "stop"

Non-streaming /v1/completions reports finish_reason: "stop" when max_tokens was hit #227

Description

Activity

solderzzc commented on Oct 11, 2026

HamedShams commented on Oct 11, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Non-streaming `/v1/completions` reports `finish_reason: "stop"` when `max_tokens` was hit #227