「Intel MacでBonsai-8Bを動かす」の版間の差分
Administrator (トーク | 投稿記録) |
Administrator (トーク | 投稿記録) |
||
| (同じ利用者による、間の1版が非表示) | |||
| 28行目: | 28行目: | ||
=== bonsai のモデルをダウンロード === | === bonsai のモデルをダウンロード === | ||
brewで入れたuvを使ってモデルをダウンロードする | |||
<source lang=bash> | <source lang=bash> | ||
% uvx hf download prism-ml/Bonsai-8B-gguf --local-dir ./models | % uvx hf download prism-ml/Bonsai-8B-gguf --local-dir ./models | ||
</source> | </source> | ||
=== cli で動かす === | |||
<source lang=bash> | <source lang=bash> | ||
% ./llama.cpp/build/bin/llama-cli -m models/Bonsai-8B.gguf | % ./llama.cpp/build/bin/llama-cli -m models/Bonsai-8B.gguf | ||
</source> | </source> | ||
チャットできるようになった。めっちゃ遅い。 | |||
=== Web UI / Web API を動かす === | |||
<source lang=bash> | <source lang=bash> | ||
% ./llama.cpp/build/bin/llama-server \ | % ./llama.cpp/build/bin/llama-server \ | ||
| 56行目: | 57行目: | ||
{"models":[{"name":"bonsai-8b","model":"bonsai-8b","modified_at":"","size":"","digest":"","type":"model","description":"","tags":[""],"capabilities":["completion"],"parameters":"","details":{"parent_model":"","format":"gguf","family":"","families":[""],"parameter_size":"","quantization_level":""}}],"object":"list","data":[{"id":"bonsai-8b","aliases":["bonsai-8b"],"tags":[],"object":"model","created":1775728104,"owned_by":"llamacpp","meta":{"vocab_type":2,"n_vocab":151669,"n_ctx_train":65536,"n_embd":4096,"n_params":8188548096,"size":1152704128}}]} | {"models":[{"name":"bonsai-8b","model":"bonsai-8b","modified_at":"","size":"","digest":"","type":"model","description":"","tags":[""],"capabilities":["completion"],"parameters":"","details":{"parent_model":"","format":"gguf","family":"","families":[""],"parameter_size":"","quantization_level":""}}],"object":"list","data":[{"id":"bonsai-8b","aliases":["bonsai-8b"],"tags":[],"object":"model","created":1775728104,"owned_by":"llamacpp","meta":{"vocab_type":2,"n_vocab":151669,"n_ctx_train":65536,"n_embd":4096,"n_params":8188548096,"size":1152704128}}]} | ||
</source> | </source> | ||
OpenClawも試してみたが、思考数が多いと全く反応がなくなる。使い物にならない。 | |||
== 結論 == | == 結論 == | ||