Custom nodes, imports fails. Moving mamba env to workspace? #68
Comments
|
Which nodes? Can you provide a log? Persisting the environment is only useful if you want the node dependencies to remain installed after the container has been removed. It's not usually necessary as nodes will have their dependencies installed on ComfyUI start. I need to update the documentation because the default is now If you set that variable to |
|
Yes that is exactly what I want. For my environment to be persistent after the container has been removed. I imagine installing many custom nodes and dependencies at ComfyUI start would take some time. By the way, Im using the template from Runpod.io and the image pull everytime I want to use it is very slow. About 40 mins for the 4.8GB. Any solution for this? I will try and find a log and come back. |
|
Here's my log: The module not found errors aren't related to their dependencies? ** ComfyUI startup time: 2024-04-18 02:49:12.047262
[2024-04-18 02:49] ** Platform: Linux
[2024-04-18 02:49] ** Python version: 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
[2024-04-18 02:49] ** Python executable: /opt/micromamba/envs/comfyui/bin/python
[2024-04-18 02:49] ** Log path: /workspace/ComfyUI/comfyui.log
[2024-04-18 02:49]
Prestartup times for custom nodes:
[2024-04-18 02:49] 0.0 seconds: /workspace/ComfyUI/custom_nodes/rgthree-comfy
[2024-04-18 02:49] 0.3 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-Manager
[2024-04-18 02:49]
Total VRAM 24217 MB, total RAM 257590 MB
[2024-04-18 02:49] xformers version: 0.0.25.post1
[2024-04-18 02:49] Set vram state to: NORMAL_VRAM
[2024-04-18 02:49] Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync
[2024-04-18 02:49] VAE dtype: torch.bfloat16
[2024-04-18 02:49] Using xformers cross attention
[2024-04-18 02:49] Traceback (most recent call last):
File "/workspace/ComfyUI/nodes.py", line 1864, in load_custom_node
module_spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/workspace/ComfyUI/custom_nodes/ComfyUI-KJNodes/__init__.py", line 1, in <module>
from .nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
File "/workspace/ComfyUI/custom_nodes/ComfyUI-KJNodes/nodes.py", line 7, in <module>
import matplotlib.pyplot as plt
ModuleNotFoundError: No module named 'matplotlib'
[2024-04-18 02:49] Cannot import /workspace/ComfyUI/custom_nodes/ComfyUI-KJNodes module for custom nodes: No module named 'matplotlib'
[2024-04-18 02:49]
[2024-04-18 02:49]� [92m[rgthree] Loaded 37 exciting nodes.�[0m
[2024-04-18 02:49] �[92m[rgthree] Will use rgthree's optimized recursive execution.�[0m
[2024-04-18 02:49]
[2024-04-18 02:49] [VideoHelperSuite] - �[0;33mWARNING�[0m - Failed to import imageio_ffmpeg
[2024-04-18 02:49] [AnimateDiffEvo] - �[0;31mERROR�[0m - No motion models found. Please download one and place in: ['/workspace/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/models', '/workspace/ComfyUI/models/animatediff_models']
[2024-04-18 02:49] Traceback (most recent call last):
File "/workspace/ComfyUI/nodes.py", line 1864, in load_custom_node
module_spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/workspace/ComfyUI/custom_nodes/ComfyUI_FizzNodes/__init__.py", line 57, in <module>
from .ScheduledNodes import (
File "/workspace/ComfyUI/custom_nodes/ComfyUI_FizzNodes/ScheduledNodes.py", line 4, in <module>
import numexpr
ModuleNotFoundError: No module named 'numexpr'
[2024-04-18 02:49] Cannot import /workspace/ComfyUI/custom_nodes/ComfyUI_FizzNodes module for custom nodes: No module named 'numexpr'
[2024-04-18 02:49] Traceback (most recent call last):
File "/workspace/ComfyUI/nodes.py", line 1864, in load_custom_node
module_spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/workspace/ComfyUI/custom_nodes/was-node-suite-comfyui/__init__.py", line 1, in <module>
from .WAS_Node_Suite import NODE_CLASS_MAPPINGS
File "/workspace/ComfyUI/custom_nodes/was-node-suite-comfyui/WAS_Node_Suite.py", line 37, in <module>
from numba import jit
ModuleNotFoundError: No module named 'numba'
[2024-04-18 02:49] Cannot import /workspace/ComfyUI/custom_nodes/was-node-suite-comfyui module for custom nodes: No module named 'numba'
[2024-04-18 02:49]
�[36mEfficiency Nodes:�[0m Attempting to add Control Net options to the 'HiRes-Fix Script' Node (comfyui_controlnet_aux add-on)...�[92mSuccess!�[0m
[2024-04-18 02:49] �[93mEfficiency Nodes Warning:�[0m Failed to import python package 'simpleeval'; related nodes disabled.
[2024-04-18 02:49]
[2024-04-18 02:49]
Full error log from comfyui_controlnet_aux:
Traceback (most recent call last):
File "/workspace/ComfyUI/custom_nodes/comfyui_controlnet_aux/__init__.py", line 32, in load_nodes
module = importlib.import_module(
File "/opt/micromamba/envs/comfyui/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/workspace/ComfyUI/custom_nodes/comfyui_controlnet_aux/node_wrappers/dwpose.py", line 5, in <module>
from controlnet_aux.dwpose import DwposeDetector, AnimalposeDetector
File "/workspace/ComfyUI/custom_nodes/comfyui_controlnet_aux/src/controlnet_aux/dwpose/__init__.py", line 15, in <module>
from . import util
File "/workspace/ComfyUI/custom_nodes/comfyui_controlnet_aux/src/controlnet_aux/dwpose/util.py", line 3, in <module>
import matplotlib
ModuleNotFoundError: No module named 'matplotlib'
[2024-04-18 02:49]
[2024-04-18 02:49] Traceback (most recent call last):
File "/workspace/ComfyUI/nodes.py", line 1864, in load_custom_node
module_spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/workspace/ComfyUI/custom_nodes/ComfyUI-Crystools/__init__.py", line 10, in <module>
from .core import version, logger
File "/workspace/ComfyUI/custom_nodes/ComfyUI-Crystools/core/__init__.py", line 5, in <module>
from .common import *
File "/workspace/ComfyUI/custom_nodes/ComfyUI-Crystools/core/common.py", line 4, in <module>
from deepdiff import DeepDiff
ModuleNotFoundError: No module named 'deepdiff'
[2024-04-18 02:49] Cannot import /workspace/ComfyUI/custom_nodes/ComfyUI-Crystools module for custom nodes: No module named 'deepdiff'
[2024-04-18 02:49] ### Loading: ComfyUI-Manager (V2.19)
[2024-04-18 02:49] ### ComfyUI Revision: 2129 [719fb2c8] *DETACHED | Released on '2024-04-14'
[2024-04-18 02:49]
Import times for custom nodes:
[2024-04-18 02:49] 0.0 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-Image-Selector
[2024-04-18 02:49] 0.0 seconds: /workspace/ComfyUI/custom_nodes/websocket_image_save.py
[2024-04-18 02:49] 0.0 seconds: /workspace/ComfyUI/custom_nodes/mikey_nodes
[2024-04-18 02:49] 0.0 seconds: /workspace/ComfyUI/custom_nodes/comfyui_controlnet_aux
[2024-04-18 02:49] 0.0 seconds (IMPORT FAILED): /workspace/ComfyUI/custom_nodes/ComfyUI-KJNodes
[2024-04-18 02:49] 0.0 seconds (IMPORT FAILED): /workspace/ComfyUI/custom_nodes/was-node-suite-comfyui
[2024-04-18 02:49] 0.0 seconds (IMPORT FAILED): /workspace/ComfyUI/custom_nodes/ComfyUI_FizzNodes
[2024-04-18 02:49] 0.1 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus
[2024-04-18 02:49] 0.1 seconds (IMPORT FAILED): /workspace/ComfyUI/custom_nodes/ComfyUI-Crystools
[2024-04-18 02:49] 0.1 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-Custom-Scripts
[2024-04-18 02:49] 0.1 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-Advanced-ControlNet
[2024-04-18 02:49] 0.1 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite
[2024-04-18 02:49] 0.2 seconds: /workspace/ComfyUI/custom_nodes/Derfuu_ComfyUI_ModdedNodes
[2024-04-18 02:49] 0.2 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved
[2024-04-18 02:49] 0.2 seconds: /workspace/ComfyUI/custom_nodes/rgthree-comfy
[2024-04-18 02:49] 0.2 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-Manager
[2024-04-18 02:49] 0.5 seconds: /workspace/ComfyUI/custom_nodes/efficiency-nodes-comfyui
[2024-04-18 02:49]
[2024-04-18 02:49] Starting server
[2024-04-18 02:49] To see the GUI go to: http://127.0.0.1:18188
[2024-04-18 02:49] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json
[2024-04-18 02:49] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
[2024-04-18 02:49] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
[2024-04-18 02:49] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
|
|
@dkampien did you ever find a solution for this custom node installation failure? |
|
Running into the same issue... for example: I can work around it temporarily by using the "install pip packages" from the manager UI node, but... currently looking into doing something to work around this in the meanwhile to make it persistent |
|
Theres a bug with the default provisioning script. It should run an install for all declared nodes, but it won't unless AUTO_UPDATE is enabled. I'll fix this asap - I can also extend the behaviour to install requirements for all nodes which seems sensible. Should be able to do this within 24 hours |
|
I tried my hand at it in #73 -- feedback welcome, or, if you were thinking of something else feel free to close/replace it. (I'm still in the process of testing it...) |
|
@dkampien , re:
I think 4.8 gig is actually a "not too bad" of a image size for this kind of app (and in this case, might only be one layer? I just pulled the whole image locally and I got 13 gig for In some cases, it might be runpod (and/or the image registry?), I also have times where the image pull takes forever randomly. Also -- potentially you could log into the ghcr registry using runpod.io? e.g. with a github personal access token? I'm wondering if it's getting ratelimited by not logging in. |
I am curious; is it fixed? Seem that it is still not reinstalling the packages . I need to write a provisioning script to make it works. (running in runpod) |
|
Unfortunately not. There have been very few updates pushed actually but there will be a big update in the next few days hopefully that will alleviate this problem along with other small issues. I'm removing micromamba and moving everything to venv. Remember, the provisioning script is just a bash script and it can be written in any style. You can just use a list of commands as you'd write them into a terminal if you'd like. The default script was only ever meant as an example, but it will definitely be fixed when my update is ready |
|
got it. thx By the way, out of curiosity, what is the reason to drop micromamba? |
|
It causes more issues than it solves:
I like micromamba and outside of containers it's still my go-to, but my new solution is cleaner, leaner and less likely to give me a future headache - I hope! |
|
@benlau New method to sync environment: Run it manually whenever it's convenient to backup your container environment to a persistent disk |
|
Thanks for providing means to sync the env! It looks like it can't be re-run, however, as |
No you can't re-run it, but I am not sure why you would want to. If you're using the same docker image for each run then you'd only be moving the same files and potentially overwriting any modifications you wanted to keep. If you're using a more recent image then you probably break version control - My view is that once you take the environment out of the container then you need to manage updates manually, although You can delete the workspace env and then re-run for a clean start though |
I installed some custom nodes but they keep getting import failed. I think this is due to the mambe env not being persistent in the workspace folder. You mentioned in the base image docks something about moving the env to workspace and that it's the default to true. But I can't find the env in workspace. Do I need to do something specific?
The text was updated successfully, but these errors were encountered: