ローカルLLMをApple silicon上のGPUコンテナで動かす（Ollama）
#Ollama #GPU #macOS
やりたいこと
OllamaをmacOSに直にインストールせずにコンテナ上でローカルLLMを使いたい。
コンテナでもGPUをちゃんと使って。
直にインストールする方法で良いのであれば：https://formulae.brew.sh/formula/ollama
MLXが使える利点はある。
はまだGPU対応していないようで、podmanが知見が多いのでpodmanを使う。
https://podman.io/
やりかた
podmanデフォルトでGPUを使うための設定が有効になっているはずだが、以下に説明がある。
https://podman-desktop.io/docs/podman/gpu
以下を実行するとDockerfileなしでlocalhost/ollama-vulkanイメージをビルドできる。
（podman build -f . <<EOF...EOFをしている）
code:bash
podman build \
--arch arm64 \
--layers=true \
--build-arg OLLAMA_VERSION=0.30.11 \
--build-arg FEDORA_VERSION=44 \
--build-arg MESA_VERSION=25.3.6-102.fc44 \
--build-arg VULKAN_LOADER_VERSION=1.4.341.0-1.fc44 \
-t localhost/ollama-vulkan \
-f - . <<'EOF'
ARG OLLAMA_VERSION
ARG FEDORA_VERSION
ARG MESA_VERSION
ARG VULKAN_LOADER_VERSION
# ============================================================
# ダウンロード環境
# ============================================================
FROM registry.fedoraproject.org/fedora:${FEDORA_VERSION} AS download-base
RUN --mount=type=cache,target=/var/cache/libdnf5 \
dnf -y install \
curl \
gzip \
tar \
zstd \
--setopt=install_weak_deps=False
# ============================================================
# 公式Linux ARM64配布物
# ============================================================
FROM download-base AS release
ARG OLLAMA_VERSION
RUN test -n "${OLLAMA_VERSION}"
RUN --mount=type=cache,id=ollama-downloads,target=/var/cache/ollama \
set -eux; \
archive="/var/cache/ollama/ollama-linux-arm64-${OLLAMA_VERSION}.tar.zst"; \
partial="${archive}.part"; \
if ! -s "${archive}" ; then \
curl \
--fail \
--location \
--retry 5 \
--retry-delay 2 \
--retry-all-errors \
--continue-at - \
--output "${partial}" \
"https://github.com/ollama/ollama/releases/download/v${OLLAMA_VERSION}/ollama-linux-arm64.tar.zst"; \
mv "${partial}" "${archive}"; \
fi; \
rm -rf /opt/ollama-release; \
mkdir -p /opt/ollama-release; \
tar --zstd -xf "${archive}" -C /opt/ollama-release
RUN set -eux; \
test -x /opt/ollama-release/bin/ollama; \
test -x /opt/ollama-release/lib/ollama/llama-server; \
/opt/ollama-release/bin/ollama --version \
/tmp/ollama-version.txt 2>&1 || true; \
cat /tmp/ollama-version.txt; \
grep -F -- "${OLLAMA_VERSION}" /tmp/ollama-version.txt; \
rm -f /tmp/ollama-version.txt
RUN set -eux; \
find /opt/ollama-release/lib/ollama \
-mindepth 1 \
-maxdepth 1 \
-type d \
\( \
-name 'cuda*' \
-o -name 'jetpack*' \
-o -name 'rocm*' \
-o -name 'mlx*' \
\) \
-print \
-exec rm -rf -- {} +
# ============================================================
# 同じバージョンの公式ソースからVulkanバックエンドをビルド
# ============================================================
FROM download-base AS vulkan-builder
ARG OLLAMA_VERSION
RUN test -n "${OLLAMA_VERSION}"
RUN --mount=type=cache,target=/var/cache/libdnf5 \
dnf -y install \
ccache \
cmake \
gcc \
gcc-c++ \
git \
glslc \
ninja-build \
pkgconf-pkg-config \
spirv-headers-devel \
vulkan-headers \
vulkan-loader-devel \
--setopt=install_weak_deps=False
RUN command -v ccache \
&& command -v cmake \
&& command -v glslc \
&& command -v ninja
RUN --mount=type=cache,id=ollama-downloads,target=/var/cache/ollama \
set -eux; \
archive="/var/cache/ollama/ollama-source-${OLLAMA_VERSION}.tar.gz"; \
partial="${archive}.part"; \
if ! -s "${archive}" ; then \
curl \
--fail \
--location \
--retry 5 \
--retry-delay 2 \
--retry-all-errors \
--continue-at - \
--output "${partial}" \
"https://github.com/ollama/ollama/archive/refs/tags/v${OLLAMA_VERSION}.tar.gz"; \
mv "${partial}" "${archive}"; \
fi; \
rm -rf /src/ollama; \
mkdir -p /src/ollama; \
tar -xzf "${archive}" \
--strip-components=1 \
-C /src/ollama
WORKDIR /src/ollama
RUN test -f llama/server/CMakePresets.json \
&& grep -q '"name": "vulkan"' llama/server/CMakePresets.json
ENV CMAKE_GENERATOR=Ninja
ENV CCACHE_DIR=/root/.cache/ccache
RUN --mount=type=cache,id=ollama-ccache,target=/root/.cache/ccache \
cmake \
-S llama/server \
--preset vulkan \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
RUN --mount=type=cache,id=ollama-ccache,target=/root/.cache/ccache \
cmake \
--build build/llama-server-vulkan \
--parallel "$(nproc)"
RUN set -eux; \
cmake \
--install build/llama-server-vulkan \
--component llama-server \
--strip; \
test -f dist/lib/ollama/vulkan/libggml-vulkan.so
# ============================================================
# 最終イメージ用root filesystem
# ============================================================
FROM registry.fedoraproject.org/fedora:${FEDORA_VERSION} AS runtime-root
ARG OLLAMA_VERSION
ARG FEDORA_VERSION
ARG MESA_VERSION
ARG VULKAN_LOADER_VERSION
RUN set -eux; \
printf '%s\n' \
'mesa-libkrun-vulkan' \
'name=Patched Mesa for libkrun' \
"baseurl=https://download.copr.fedorainfracloud.org/results/slp/mesa-libkrun-vulkan/fedora-${FEDORA_VERSION}-aarch64/" \
'enabled=1' \
'gpgcheck=1' \
'repo_gpgcheck=0' \
'gpgkey=https://download.copr.fedorainfracloud.org/results/slp/mesa-libkrun-vulkan/pubkey.gpg' \
/etc/yum.repos.d/mesa-libkrun-vulkan.repo
RUN --mount=type=cache,target=/var/cache/libdnf5 \
set -eux; \
mkdir -p /runtime-root; \
dnf -y \
--installroot=/runtime-root \
--use-host-config \
--releasever="${FEDORA_VERSION}" \
--setopt=cachedir=/var/cache/libdnf5 \
--setopt=system_cachedir=/var/cache/libdnf5 \
--setopt=install_weak_deps=False \
--setopt=tsflags=nodocs \
install \
ca-certificates \
libstdc++ \
openblas \
"vulkan-loader-${VULKAN_LOADER_VERSION}.aarch64" \
"mesa-vulkan-drivers-${MESA_VERSION}.aarch64"; \
rpm \
--root=/runtime-root \
-q mesa-vulkan-drivers \
| grep -F -- "mesa-vulkan-drivers-${MESA_VERSION}.aarch64"; \
rpm \
--root=/runtime-root \
-q vulkan-loader \
| grep -F -- "vulkan-loader-${VULKAN_LOADER_VERSION}.aarch64"; \
rm -rf \
/runtime-root/var/cache/* \
/runtime-root/var/log/* \
/runtime-root/var/lib/dnf \
/runtime-root/var/lib/rpm \
/runtime-root/etc/yum.repos.d
COPY --from=release \
/opt/ollama-release/bin/ollama \
/runtime-root/usr/bin/ollama
COPY --from=release \
/opt/ollama-release/lib/ollama/ \
/runtime-root/usr/lib/ollama/
COPY --from=vulkan-builder \
/src/ollama/dist/lib/ollama/vulkan/ \
/runtime-root/usr/lib/ollama/vulkan/
RUN set -eux; \
mkdir -p \
/runtime-root/etc \
/runtime-root/tmp \
/runtime-root/var/tmp \
/runtime-root/var/lib/ollama/models; \
printf '%s\n' \
'root:x:0:0:root:/root:/sbin/nologin' \
'ollama:x:10001:10001:Ollama:/var/lib/ollama:/sbin/nologin' \
/runtime-root/etc/passwd; \
printf '%s\n' \
'root:x:0:' \
'ollama:x:10001:' \
/runtime-root/etc/group; \
chmod 1777 \
/runtime-root/tmp \
/runtime-root/var/tmp; \
chown -R 10001:10001 \
/runtime-root/var/lib/ollama; \
test -x /runtime-root/usr/bin/ollama; \
test -x /runtime-root/usr/lib/ollama/llama-server; \
test -f /runtime-root/usr/lib/ollama/libggml-base.so.0; \
test -f /runtime-root/usr/lib/ollama/vulkan/libggml-vulkan.so; \
test -f /runtime-root/usr/share/vulkan/icd.d/virtio_icd.aarch64.json; \
chroot /runtime-root \
/lib/ld-linux-aarch64.so.1 \
--library-path /usr/lib/ollama:/usr/lib/ollama/vulkan:/usr/lib64 \
--list /usr/lib/ollama/vulkan/libggml-vulkan.so \
/tmp/vulkan-libraries.txt; \
cat /tmp/vulkan-libraries.txt; \
! grep -q 'not found' /tmp/vulkan-libraries.txt; \
chroot /runtime-root \
/lib/ld-linux-aarch64.so.1 \
--library-path /usr/lib/ollama:/usr/lib/ollama/vulkan:/usr/lib64 \
/usr/bin/ollama --version \
/tmp/ollama-version.txt 2>&1 || true; \
cat /tmp/ollama-version.txt; \
grep -F -- "${OLLAMA_VERSION}" /tmp/ollama-version.txt
# ============================================================
# パッケージマネージャーを含まない最終イメージ
# ============================================================
FROM scratch
ARG OLLAMA_VERSION
ARG FEDORA_VERSION
ARG MESA_VERSION
ARG VULKAN_LOADER_VERSION
COPY --from=runtime-root /runtime-root/ /
LABEL org.opencontainers.image.title="Ollama Vulkan for Apple Silicon"
LABEL org.opencontainers.image.source="https://github.com/ollama/ollama"
LABEL org.opencontainers.image.version="${OLLAMA_VERSION}"
LABEL io.ollama-vulkan.fedora-version="${FEDORA_VERSION}"
LABEL io.ollama-vulkan.mesa-version="${MESA_VERSION}"
LABEL io.ollama-vulkan.vulkan-loader-version="${VULKAN_LOADER_VERSION}"
ENV HOME=/var/lib/ollama
ENV OLLAMA_MODELS=/var/lib/ollama/models
ENV LD_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/vulkan:/usr/lib64
ENV VK_DRIVER_FILES=/usr/share/vulkan/icd.d/virtio_icd.aarch64.json
ENV OLLAMA_VULKAN=1
ENV OLLAMA_IGPU_ENABLE=1
ENV OLLAMA_LLM_LIBRARY=vulkan
ENV OLLAMA_HOST=0.0.0.0:11434
WORKDIR /var/lib/ollama
USER 10001:10001
VOLUME "/var/lib/ollama"
EXPOSE 11434
ENTRYPOINT "/usr/bin/ollama"
CMD "serve"
EOF
（GPT-5.5 High on ChatGPTで試行錯誤して作成）
以下でで起動させる。
code:bash
podman run --rm \
--name ollama-vulkan \
--device /dev/dri \
-p 127.0.0.1:21434:11434 \
-v ollama-vulkan-models:/var/lib/ollama:U \
-e 'OLLAMA_ORIGINS=*' \
localhost/ollama-vulkan
動作確認
code:bash
podman exec -it ollama-vulkan ollama run qwen3:0.6b 'こんにちは！'
https://gyazo.com/d955498a4dcc7306c015f0d88b88317a
以下のようにPROCESSORが100% GPUになっている。
code:console
$ podman exec ollama-vulkan ollama ps
NAME ID SIZE PROCESSOR CONTEXT UNTIL
qwen3:0.6b 7df6b6e09427 5.4 GB 100% GPU 40960 4 minutes from now
以下はOpenAI互換APIでの例。
code:bash
curl -N http://localhost:21434/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "qwen3:0.6b",
"messages": [
{
"role": "user",
"content": "こんにちは！"
}
],
"stream": true
}'
以下はpullする方法で、がモデル名一覧。
code:bash
podman exec ollama-vulkan ollama pull <モデル名>