Back to AI Q&A
Does Hermes Agent not work after receiving vLLM?

Does Hermes Agent not work after receiving vLLM?

AI Q&A Admin 211 views

If the model only outputs the tool call as text after receiving the vLLM, it is usually not that Hermes is not connected, but that the tool call parameters are missing when the vLLM service is started. Hermes usestool_choice: autoby default, and the vLLM side also needs to be explicitly turned on.

Check the launch command first

_CODE_BLOCK_0_______

Different models need to select the corresponding parser.

Hermes/Qwen class models are commonly used hermes , Llama 3.x commonly used llama3_json , DeepSeek, Mistral, etc. must follow the parser supported by vLLM.

depends on whether these items
  • _CODE_INLINE_4__can be accessed by Hermes.

  • _CODE_INLINE_5__Whether the memory is exceeded, whether the service has failed to start.
  • Does the model itself support tool calls? Unsupported models will be unstable.

    One sentence: vLLM tool does not work, first add--enable-auto-tool-choice and correct--tool-call-parser.

    How to determine whether it is a vLLM or a model problem

  • Use the same model to directly request/v1/chat/completions, bring the simplest tool schema, and see if there are structured tool calls in the return. If the API layer has returned only ordinary text, the problem is most likely vLLM parser or model capabilities; if the API layer is normal but Hermes is abnormal, check Hermes 'custom endpoint, model name, context length, and log.
  • Official open source address: https://github.com/NousResearch/hermes-agent; Official document entry: https://hermes-agent.nousresearch.com/_.

  • Recommended Tools

    More