WindowsでMagicAnimateを動かす
Colabだとすぐタイムアウトして悲しいことになるので。
毎回ダウンロードする間抜けなことを避けたい。
Docker コンテナを作って、単純に実行するだけにしたい。
実際、どういう Dockerfile を作ったらいいのかを試すために、生でコマンド叩きながら不足している要素などを確認していく。
ベースは ubuntu:22.04 にするが、スリムにするなら純粋に実行に必要なだけのシステムに変えた方がいいかもしれない。
$ docker run --gpus all -it --rm ubuntu:22.04 /bin/bash
とりあえず root ですべて実行してしまうが、一般ユーザーにした方がいいか?
以下を参考にした
ディレクトリが Google Colab 基準 /content/ 配下になってるようなので調整。
https://gyazo.com/cf4c83b88b05332a39cc4eb5a1f2b842
code:console
$ apt-get update
.,.
$ apt-get install git
...
$ python3
bash: python3: command not found
$ apt-get -y install python3 python3-pip
...
$ mkdir /content
$ cd /content
$ cd magic-animate
$ pip install -r requirements.txt
git: 'lfs' is not a git command. See 'git --help'.
The most similar command is
log
$ apt-get -y install git-lfs
WARNING: 'git lfs clone' is deprecated and will not be updated
with new flags from 'git clone'
'git clone' has been updated in upstream Git to have comparable
speeds to 'git lfs clone'.
ここまでで環境はできているはず
Interface の指示通り試す
$ bash scripts/animate.sh
code:console
The cache for model files in Transformers v4.22.0 has been updated. Migrating your old cache. This is a one-time only operation. You can interrupt this and resume the migration later on by calling transformers.utils.move_cache().
/content/magic-animate/magicanimate/pipelines/pipeline_animation.py:43: FutureWarning: Importing DiffusionPipeline or ImagePipelineOutput from diffusers.pipeline_utils is deprecated. Please import from diffusers.pipelines.pipeline_utils instead.
from diffusers.pipeline_utils import DiffusionPipeline
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 261, in hf_raise_for_status
response.raise_for_status()
File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 428, in cached_file
resolved_file = hf_hub_download(
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1195, in hf_hub_download
metadata = get_hf_file_metadata(
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1541, in get_hf_file_metadata
hf_raise_for_status(r)
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 293, in hf_raise_for_status
raise RepositoryNotFoundError(message, response) from e
huggingface_hub.utils._errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-65752dbf-261bffd9594e0e3b3bd8448e;f7737634-270f-4ac0-840f-6373507ea3ec)
Please make sure you specified the correct repo_id and repo_type.
If you are trying to access a private or gated repo, make sure you are authenticated.
Invalid username or password.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/content/magic-animate/magicanimate/pipelines/animation.py", line 282, in <module>
run(args)
File "/content/magic-animate/magicanimate/pipelines/animation.py", line 271, in run
main(args)
File "/content/magic-animate/magicanimate/pipelines/animation.py", line 75, in main
tokenizer = CLIPTokenizer.from_pretrained(config.pretrained_model_path, subfolder="tokenizer")
File "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py", line 1813, in from_pretrained
resolved_vocab_filesfile_id = cached_file( File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 449, in cached_file
raise EnvironmentError(
If this is a private repository, make sure to pass a token having permission to this repo either by logging in with huggingface-cli login or by passing token=<your_token>
stable-diffusion のディレクトリ位置がずれていることに気が付いて修正
$ mv stable-diffusion-v1-5 pretrained_models
code:console
# bash scripts/animate.sh
/content/magic-animate/magicanimate/pipelines/pipeline_animation.py:43: FutureWarning: Importing DiffusionPipeline or ImagePipelineOutput from diffusers.pipeline_utils is deprecated. Please import from diffusers.pipelines.pipeline_utils instead.
from diffusers.pipeline_utils import DiffusionPipeline
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 479, in load_state_dict
return torch.load(checkpoint_file, map_location=map_location)
File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 815, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1033, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/content/magic-animate/magicanimate/pipelines/animation.py", line 282, in <module>
run(args)
File "/content/magic-animate/magicanimate/pipelines/animation.py", line 271, in run
main(args)
File "/content/magic-animate/magicanimate/pipelines/animation.py", line 76, in main
text_encoder = CLIPTextModel.from_pretrained(config.pretrained_model_path, subfolder="text_encoder")
File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 2805, in from_pretrained
state_dict = load_state_dict(resolved_archive_file)
File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 484, in load_state_dict
raise OSError(
OSError: You seem to have cloned a repository without having git-lfs installed. Please install git-lfs and run git lfs install followed by git lfs pull in the folder you cloned.
git lfs がちゃんとインストールされていない。
code:console
# git lfs install
Updated git hooks.
Git LFS initialized.
$ cd pretrained_models
$ rm -rf *
$ cd ..
$ bash scripts/animate.sh
色々警告が出まくる。
code:console
/content/magic-animate/magicanimate/pipelines/pipeline_animation.py:43: FutureWarning: Importing DiffusionPipeline or ImagePipelineOutput from diffusers.pipeline_utils is deprecated. Please import from diffusers.pipelines.pipeline_utils instead.
from diffusers.pipeline_utils import DiffusionPipeline
loaded temporal unet's pretrained weights from pretrained_models/stable-diffusion-v1-5/unet ...
### missing keys: 560;
### unexpected keys: 0;
### Temporal Module Parameters: 417.1376 M
The config attributes {'addition_embed_type': None, 'addition_embed_type_num_heads': 64, 'addition_time_embed_dim': None, 'conditioning_channels': 3, 'encoder_hid_dim': None, 'encoder_hid_dim_type': None, 'global_pool_conditions': False, 'num_attention_heads': None, 'transformer_layers_per_block': 1} were passed to ControlNetModel, but are not expected and will be ignored. Please verify your config.json configuration file.
It is recommended to provide attention_head_dim when calling get_down_block. Defaulting attention_head_dim to 8.
It is recommended to provide attention_head_dim when calling get_down_block. Defaulting attention_head_dim to 8.
It is recommended to provide attention_head_dim when calling get_down_block. Defaulting attention_head_dim to 8.
It is recommended to provide attention_head_dim when calling get_down_block. Defaulting attention_head_dim to 8.
/content/magic-animate/magicanimate/pipelines/pipeline_animation.py:103: FutureWarning: The configuration file of this scheduler: DDIMScheduler {
"_class_name": "DDIMScheduler",
"_diffusers_version": "0.21.4",
"beta_end": 0.012,
"beta_schedule": "linear",
"beta_start": 0.00085,
"clip_sample": true,
"clip_sample_range": 1.0,
"dynamic_thresholding_ratio": 0.995,
"num_train_timesteps": 1000,
"prediction_type": "epsilon",
"rescale_betas_zero_snr": false,
"sample_max_value": 1.0,
"set_alpha_to_one": true,
"steps_offset": 0,
"thresholding": false,
"timestep_spacing": "leading",
"trained_betas": null
}
is outdated. steps_offset should be set to 1 instead of 0. Please make sure to update the config accordingly as leaving steps_offset might led to incorrect results in future versions. If you have downloaded this checkpoint from the Hugging Face Hub, it would be very nice if you could open a Pull request for the scheduler/scheduler_config.json file
deprecate("steps_offset!=1", "1.0.0", deprecation_message, standard_warn=False)
/content/magic-animate/magicanimate/pipelines/pipeline_animation.py:116: FutureWarning: The configuration file of this scheduler: DDIMScheduler {
"_class_name": "DDIMScheduler",
"_diffusers_version": "0.21.4",
"beta_end": 0.012,
"beta_schedule": "linear",
"beta_start": 0.00085,
"clip_sample": true,
"clip_sample_range": 1.0,
"dynamic_thresholding_ratio": 0.995,
"num_train_timesteps": 1000,
"prediction_type": "epsilon",
"rescale_betas_zero_snr": false,
"sample_max_value": 1.0,
"set_alpha_to_one": true,
"steps_offset": 1,
"thresholding": false,
"timestep_spacing": "leading",
"trained_betas": null
}
has not set the configuration clip_sample. clip_sample should be set to False in the configuration file. Please make sure to update the config accordingly as not setting clip_sample in the config might lead to incorrect results in future versions. If you have downloaded this checkpoint from the Hugging Face Hub, it would be very nice if you could open a Pull request for the scheduler/scheduler_config.json file
deprecate("clip_sample not set", "1.0.0", deprecation_message, standard_warn=False)
/content/magic-animate/magicanimate/pipelines/pipeline_animation.py:624: FutureWarning: Accessing config attribute in_channels directly via 'UNet3DConditionModel' object attribute is deprecated. Please access 'in_channels' over 'UNet3DConditionModel's config object instead, e.g. 'unet.config.in_channels'.
num_channels_latents = self.unet.in_channels
100%|███████████████████████████████████████████████████████████████████████████████████| 25/25 30:12<00:00, 72.50s/it 100%|███████████████████████████████████████████████████████████████████████████████████| 48/48 00:19<00:00, 2.49it/s IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (1544, 516) to (1552, 528) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/content/magic-animate/magicanimate/pipelines/animation.py", line 282, in <module>
run(args)
File "/content/magic-animate/magicanimate/pipelines/animation.py", line 271, in run
main(args)
File "/content/magic-animate/magicanimate/pipelines/animation.py", line 185, in main
source_image = np.array(Image.open(source_image).resize((size, size)))
File "/usr/local/lib/python3.10/dist-packages/PIL/Image.py", line 3236, in open
fp = builtins.open(filename, "rb")
FileNotFoundError: Errno 2 No such file or directory: 'inputs/applications/source_image/0002.png' root@2a6b34856cc2:/content/magic-animate# ls inputs/applications/source_image/
dalle2.jpeg dalle8.jpeg demo4.png monalisa.png multi1_source.png
存在しないファイルを参照して落ちた
0002.png がないので、configs/prompts/animation.yaml から 0002.png の行を削除。
並列にある、demo4.mp4 も削除。
https://gyazo.com/427d431798d2ed234feeaa04a1cb9790
ffmpeg が不足しているみたいなので追加。
$ apt-get install ffmpeg
code:console
# bash scripts/animate.sh
/content/magic-animate/magicanimate/pipelines/pipeline_animation.py:43: FutureWarning: Importing DiffusionPipeline or ImagePipelineOutput from diffusers.pipeline_utils is deprecated. Please import from diffusers.pipelines.pipeline_utils instead.
from diffusers.pipeline_utils import DiffusionPipeline
loaded temporal unet's pretrained weights from pretrained_models/stable-diffusion-v1-5/unet ...
### missing keys: 560;
### unexpected keys: 0;
### Temporal Module Parameters: 417.1376 M
The config attributes {'addition_embed_type': None, 'addition_embed_type_num_heads': 64, 'addition_time_embed_dim': None, 'conditioning_channels': 3, 'encoder_hid_dim': None, 'encoder_hid_dim_type': None, 'global_pool_conditions': False, 'num_attention_heads': None, 'transformer_layers_per_block': 1} were passed to ControlNetModel, but are not expected and will be ignored. Please verify your config.json configuration file.
It is recommended to provide attention_head_dim when calling get_down_block. Defaulting attention_head_dim to 8.
It is recommended to provide attention_head_dim when calling get_down_block. Defaulting attention_head_dim to 8.
It is recommended to provide attention_head_dim when calling get_down_block. Defaulting attention_head_dim to 8.
It is recommended to provide attention_head_dim when calling get_down_block. Defaulting attention_head_dim to 8.
/content/magic-animate/magicanimate/pipelines/pipeline_animation.py:103: FutureWarning: The configuration file of this scheduler: DDIMScheduler {
"_class_name": "DDIMScheduler",
"_diffusers_version": "0.21.4",
"beta_end": 0.012,
"beta_schedule": "linear",
"beta_start": 0.00085,
"clip_sample": true,
"clip_sample_range": 1.0,
"dynamic_thresholding_ratio": 0.995,
"num_train_timesteps": 1000,
"prediction_type": "epsilon",
"rescale_betas_zero_snr": false,
"sample_max_value": 1.0,
"set_alpha_to_one": true,
"steps_offset": 0,
"thresholding": false,
"timestep_spacing": "leading",
"trained_betas": null
}
is outdated. steps_offset should be set to 1 instead of 0. Please make sure to update the config accordingly as leaving steps_offset might led to incorrect results in future versions. If you have downloaded this checkpoint from the Hugging Face Hub, it would be very nice if you could open a Pull request for the scheduler/scheduler_config.json file
deprecate("steps_offset!=1", "1.0.0", deprecation_message, standard_warn=False)
/content/magic-animate/magicanimate/pipelines/pipeline_animation.py:116: FutureWarning: The configuration file of this scheduler: DDIMScheduler {
"_class_name": "DDIMScheduler",
"_diffusers_version": "0.21.4",
"beta_end": 0.012,
"beta_schedule": "linear",
"beta_start": 0.00085,
"clip_sample": true,
"clip_sample_range": 1.0,
"dynamic_thresholding_ratio": 0.995,
"num_train_timesteps": 1000,
"prediction_type": "epsilon",
"rescale_betas_zero_snr": false,
"sample_max_value": 1.0,
"set_alpha_to_one": true,
"steps_offset": 1,
"thresholding": false,
"timestep_spacing": "leading",
"trained_betas": null
}
has not set the configuration clip_sample. clip_sample should be set to False in the configuration file. Please make sure to update the config accordingly as not setting clip_sample in the config might lead to incorrect results in future versions. If you have downloaded this checkpoint from the Hugging Face Hub, it would be very nice if you could open a Pull request for the scheduler/scheduler_config.json file
deprecate("clip_sample not set", "1.0.0", deprecation_message, standard_warn=False)
/content/magic-animate/magicanimate/pipelines/pipeline_animation.py:624: FutureWarning: Accessing config attribute in_channels directly via 'UNet3DConditionModel' object attribute is deprecated. Please access 'in_channels' over 'UNet3DConditionModel's config object instead, e.g. 'unet.config.in_channels'.
num_channels_latents = self.unet.in_channels
100%|███████████████████████████████████████████████████████████████████████████████████| 25/25 21:45<00:00, 52.21s/it 100%|███████████████████████████████████████████████████████████████████████████████████| 48/48 00:31<00:00, 1.51it/s IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (1544, 516) to (1552, 528) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
100%|█████████████████████████████████████████████████████████████████████████████████| 112/112 01:42<00:00, 1.09it/s IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (1544, 516) to (1552, 528) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
100%|█████████████████████████████████████████████████████████████████████████████████| 128/128 02:09<00:00, 1.01s/it IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (1544, 516) to (1552, 528) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
100%|█████████████████████████████████████████████████████████████████████████████████| 112/112 01:42<00:00, 1.09it/s IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (1544, 516) to (1552, 528) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
100%|███████████████████████████████████████████████████████████████████████████████████| 96/96 01:34<00:00, 1.02it/s IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (1544, 516) to (1552, 528) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
5時間47分で5つの動画ができた。
(RTX3060Ti)