This page permanently redirects to gemini://agnos.is/projects/open-webui-filters/thinking-filters/.

🧠 Thinking Filters

Simple filters for supporting models that use chain-of-thought reasoning capabilities. These filters rewrite output to separate reasoning/thought process from the final output.

What is Chain-of-Thought?

Chain-of-Thought prompting is an experimental way for large language models to reason about problems better. Because LLMs are fundamentally text prediction machines (that is, they predict what text will follow the text given to them), they often generate all kinds of strange yet confident answers. Chain-of-Thought reasoning aims to reduce these issues by training the model to output text for reasoning about user queries.

So while the LLM is still fundamentally generating predicted text based on input, that generated text is now designed to follow a reasoning path. This leads to notable reduction in hallucinations and calculation errors. It's not perfect, but it's better than nothing.

The biggest downside of Chain-of-Thought prompting is the significant increase in token output, resulting in slower response times and lots of extra text. It becomes difficult to focus on the essential information: the LLM's final answer. These filters mitigate this by providing a clear distinction between the model's thought process and its conclusion.

Filters and Supported Models

These are the current filters and what models they support:

=> ⌛ Filter: Artificium Thinking Filter | 🤖 Supported Model: Artificium Llama 3.1 8b

=> ⌛ Filter: Collapsible Thought Filter | 🤖 Supported Model: Reflection 70b

Configuration

Documentation is separated by filter.

Collapsible Thought Filter

Has 4 settings besides filter priority:

• Thought Title: What to use for the header that holds the thoughts.
• Thought Tag: XML tag to use for the thoughts section.
• Output Tag: XML tag to use for the final output section.
• Use Thoughts as Context: Whether or not to submit thought process text to the AI.

Artificium Thinking Filter

Has 3 settings besides filter priority:

• Task Title: What to use for the header that holds the initial task breakdown.
• Breakdown Title: What to use for the header that holds the reasoning/thought breakdown.
• Use Thoughts as Context: Whether or not to submit thought process text to the AI.

Usage

These filters are meant to make output from Chain-of-Thought LLMs more readable. They are also designed specifically for certain models and their derivatives. Everything should work out of the box.

• It is possible to use the filters with other models, as long as their output matches the format expected by the filter.

The "Use Thoughts as Context" setting is disabled by default because it will send a lot more tokens to the LLM, resulting in longer processing and fuller contexts. Enabling it might get more accurate answers over longer conversations.

‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗

=> ⤴️ [/projects/open-webui-filters] | 🏠 Home