Module LLMをTCP/IPから使う
Module LLMとTCP/IP
Module LLMでは、llm_sys(/opt/m5stack/bin/llm_sys: API)経由でTCP/IP 10001からシステムにアクセスできるようになっています。 TCP/IP 10001にアクセスするには、telnetコマンドなどを利用することができます(apt install telnet)。
$ telnet localhost 10001
入出力は、JSON形式のデータです
jqコマンドがあると、整形できて便利です。
$ apt install jq
システム情報を取得する
以下のように、システムの情報を取得することができます。
この他にも、リセットをかけたり("action": "reset")、リブートしたり("action": "reboot")することが可能です。
llm_sysへの接続
$ telnet localhost 10001
code:システム情報:入力.json
{
"request_id": "001",
"work_id": "sys",
"action": "hwinfo"
}
code:システム情報:出力.json
## 出力
{"created":1737943019,"data":{"cpu_loadavg":0,"eth_info":{"ip":"192.168.11.130","name":"wlan0","speed":""},"mem":23,"temperature":39350},"error":{"code":0,"message":""},"object":"sys.hwinfo","request_id":"001","work_id":"sys"} ### 出力を整形したもの(以下の例では、整形版のみを記載)
# $ cat | jq
{
"created": 1737943019,
"data": {
"cpu_loadavg": 0,
"eth_info": [
{
"ip": "192.168.11.130",
"name": "wlan0",
"speed": ""
}
],
"mem": 23,
"temperature": 39350
},
"error": {
"code": 0,
"message": ""
},
"object": "sys.hwinfo",
"request_id": "001",
"work_id": "sys"
}
LLM(llm_llm)を使う
初めに、llmを起動します。
code:起動:入力.json
{
"request_id": "2",
"work_id": "llm",
"action": "setup",
"object": "llm.setup",
"data": {
"model": "qwen2.5-0.5B-prefill-20e",
"response_format": "llm.utf-8.stream",
"input": "llm.utf-8",
"enoutput": true,
"max_token_len": 256,
"prompt": "You are a knowledgeable assistant capable of answering various questions and providing information."
}
}
code:起動:出力.json
## 出力
{
"created": 1737946219,
"data": "None",
"error": {
"code": 0,
"message": ""
},
"object": "None",
"request_id": "2",
"work_id": "llm.1000"
}
起動時に返ってきた"work_id"の"llm.1000"に対して、"action": "inference"を実行します。
code:LLMを動かす:英語入力.json
{
"request_id": "4",
"work_id": "llm.1000",
"action": "inference",
"object": "llm.utf-8.stream",
"data": {
"delta": "What's ur name?",
"index": 0,
"finish": true
}
}
code:LLMを動かす:日本語入力.json
{
"request_id": "4",
"work_id": "llm.1000",
"action": "inference",
"object": "llm.utf-8.stream",
"data": {
"delta": "あなたの名前は?",
"index": 0,
"finish": true
}
}
code:LLMを動かす:日本語出力.json
{"created":1737951273,"data":{"delta":"私はAIア","finish":false,"index":0},"error":{"code":0,"message":""},"object":"llm.utf-8.stream","request_id":"4","work_id":"llm.1000"}
{"created":1737951273,"data":{"delta":"シスタント","finish":false,"index":1},"error":{"code":0,"message":""},"object":"llm.utf-8.stream","request_id":"4","work_id":"llm.1000"}
{"created":1737951273,"data":{"delta":"で、名","finish":false,"index":2},"error":{"code":0,"message":""},"object":"llm.utf-8.stream","request_id":"4","work_id":"llm.1000"}
{"created":1737951273,"data":{"delta":"前はありません。","finish":false,"index":3},"error":{"code":0,"message":""},"object":"llm.utf-8.stream","request_id":"4","work_id":"llm.1000"}
{"created":1737951274,"data":{"delta":"しかし、私は","finish":false,"index":4},"error":{"code":0,"message":""},"object":"llm.utf-8.stream","request_id":"4","work_id":"llm.1000"}
{"created":1737951274,"data":{"delta":"あなたの質問","finish":false,"index":5},"error":{"code":0,"message":""},"object":"llm.utf-8.stream","request_id":"4","work_id":"llm.1000"}
{"created":1737951274,"data":{"delta":"に答えたり","finish":false,"index":6},"error":{"code":0,"message":""},"object":"llm.utf-8.stream","request_id":"4","work_id":"llm.1000"}
{"created":1737951275,"data":{"delta":"、情報を提供","finish":false,"index":7},"error":{"code":0,"message":""},"object":"llm.utf-8.stream","request_id":"4","work_id":"llm.1000"}
{"created":1737951275,"data":{"delta":"したりすることができます。","finish":false,"index":8},"error":{"code":0,"message":""},"object":"llm.utf-8.stream","request_id":"4","work_id":"llm.1000"}
{"created":1737951275,"data":{"delta":"何かお手","finish":false,"index":9},"error":{"code":0,"message":""},"object":"llm.utf-8.stream","request_id":"4","work_id":"llm.1000"}
{"created":1737951275,"data":{"delta":"伝いできる","finish":false,"index":10},"error":{"code":0,"message":""},"object":"llm.utf-8.stream","request_id":"4","work_id":"llm.1000"}
{"created":1737951276,"data":{"delta":"ことがありましたら","finish":false,"index":11},"error":{"code":0,"message":""},"object":"llm.utf-8.stream","request_id":"4","work_id":"llm.1000"}
{"created":1737951276,"data":{"delta":"、お知","finish":false,"index":12},"error":{"code":0,"message":""},"object":"llm.utf-8.stream","request_id":"4","work_id":"llm.1000"}
{"created":1737951276,"data":{"delta":"らせください。","finish":false,"index":13},"error":{"code":0,"message":""},"object":"llm.utf-8.stream","request_id":"4","work_id":"llm.1000"}
{"created":1737951277,"data":{"delta":"\n","finish":false,"index":14},"error":{"code":0,"message":""},"object":"llm.utf-8.stream","request_id":"4","work_id":"llm.1000"}
{"created":1737951277,"data":{"delta":"","finish":true,"index":15},"error":{"code":0,"message":""},"object":"llm.utf-8.stream","request_id":"4","work_id":"llm.1000"}
workerの情報などを取得することも可能です。
code:worker一覧:入力.json
{
"request_id": "2",
"work_id": "llm",
"action": "taskinfo"
}
code:worker一覧:出力.json
## 出力:"data"部分にworkerの一覧が入る
{
"created": 1737948979,
"data": [
"llm.1000"
],
"error": {
"code": 0,
"message": ""
},
"object": "llm.tasklist",
"request_id": "2",
"work_id": "llm"
}
code:worker情報:入力.json
{
"request_id": "2",
"work_id": "llm.1000",
"action": "taskinfo"
}
code:worker情報:出力.json
## 出力:どのモデルで初期化されているのかなどがわかる
{
"created": 1737948810,
"data": {
"enoutput": true,
"inputs": [
"llm.utf-8"
],
"model": "qwen2.5-0.5B-prefill-20e",
"response_format": "llm.utf-8.stream"
},
"error": {
"code": 0,
"message": ""
},
"object": "llm.taskinfo",
"request_id": "2",
"work_id": "llm.1000"
}
bashexecを使った任意コマンドの実行
bashexecコマンド(action)を使って、任意のコマンドを実行することが可能です。
対応コード(int sys_bashexec(int com_id, const nlohmann::json &json_obj)) code:任意のコマンド実行:入力.json
{
"request_id": "1",
"work_id": "sys",
"action": "bashexec",
"object": "stream",
"data": {
"index": 0,
"delta": "ls",
"finish": true
}
}
code:任意のコマンド実行:出力.json
{
"created": 1738031137,
"data": "None",
"error": {
"code": 0,
"message": ""
},
"object": "None",
"request_id": "1",
"work_id": "sys"
}
{
"created": 1738031137,
"data": {
"delta": "bin data lib\tlibhv.20230822.log libhv.20250127.log\tscripts share\r\n",
"finish": false,
"index": 0
},
"error": {
"code": 0,
"message": ""
},
"object": "sys.utf-8.stream",
"request_id": "1",
"work_id": "sys"
}
{
"created": 1738031137,
"data": {
"delta": "exit\r\n",
"finish": false,
"index": 1
},
"error": {
"code": 0,
"message": ""
},
"object": "sys.utf-8.stream",
"request_id": "1",
"work_id": "sys"
}
参考文献
公式ドキュメント