Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models

we present Chameleon, an AI system that mitigates these limitations by augmenting LLMs with plug-and-play modules for compositional reasoning.

Chameleon synthesizes programs by composing various tools (e.g., LLMs, off-the-shelf vision models, web search engines, Python functions, and heuristic-based modules) for accomplishing complex reasoning tasks.

At the heart of Chameleon is an LLM-based planner that assembles a sequence of tools to execute to generate the final response.

Figure 1

画像と質問文（3択）

text detector（画像の中の文章）

knowledge retriever（単語を元に知識取得）

Chain of thought

https://github.com/lupantech/chameleon-llm/blob/main/assets/showcase_scienceqa.png?raw=true

使ったデータセット（どちらも同じlupantechから）

Table 1 (Chameleonと他のtool-augmented LLMとの比較)

Tool-Augmented Language Models (2 Related Work)

例

Module inventory

プロンプトで性能も上がる