Repository navigation
fix(server): treat developer messages as system prompts - #354
Closed
WhitePepperLambSoup wants to merge 1 commit into
Closed
WhitePepperLambSoup wants to merge 1 commit into
WhitePepperLambSoup wants to merge 1 commit into
Conversation
OpenAI clients send `developer` messages instead of `system` messages for o-series and newer models. parse_conversation only handled system, user and assistant, so a developer message was dropped before any approach ran and the approach got an empty system prompt. Handle `developer` the same way as `system`, including optillm_approach tag extraction and list-style content. Add tests/test_developer_role.py and run it in CI.
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
OpenAI clients send
developermessages instead ofsystemmessages for o-series and newer models.parse_conversationonly handledsystem,userandassistant, so adevelopermessage was dropped before any approach ran. With a technique selected (moa-gpt-4o-mini,optillm_approach, a prompt tag), the approach got an empty system prompt and never saw the instructions. Thenoneapproach was not affected because it forwards the original messages.parse_conversationnow handlesdeveloperthe same way assystem, including<optillm_approach>tag extraction and list-style content. If a request contains several system/developer messages, the last one still wins, as it already did for several system messages.There is no existing issue for this.
Tests: added
tests/test_developer_role.py(also added to the unit test list in.github/workflows/test.yml). It checksparse_conversationdirectly and sends a request through/v1/chat/completionswith a mocked MCTS approach to confirm the developer text reaches the approach as its system prompt. The four developer-role tests fail onmainand pass with the change.Ran locally on Windows (Python 3.12):
tests/test_developer_role.py,tests/test_completions_endpoint.py,tests/test_proxy_plugin.py,tests/test_max_completion_tokens.pyandtests/test_plugin_gating.py. I did not run the integration tests that need a running server and a local model.