Running multiple workflows just after each other

Hi,

If I try to run a second comfyscript workflow before the first one is done, I receive the following error. Otherwise, workflow works perfectly fine.

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

Any ideas to solve this issue?

Thank you!

Bests,
Cihan

Are you using real mode? What do you exactly mean by "run a second comfyscript workflow before the first one is done"? Multiple threads? Or one with Workflow() inside another with Workflow()?

Are you using real mode? What do you exactly mean by "run a second comfyscript workflow before the first one is done"? Multiple threads? Or one with Workflow() inside another with Workflow()?

I am indeed using real mode and I mean multiple threads. I am calling a function with 'Workflow()' twice. Whenever i call the same function 2nd time, before the first call ends, I receive this error.

I can't reproduce this, both with sleep and without. Maybe it's related to specific nodes. Which node do you received this error? What are its parent nodes (inputs)?

I can't reproduce this, both with sleep and without. Maybe it's related to specific nodes. Which node do you received this error? What are its parent nodes (inputs)?

I will share a simple demonstrative script, tomorrow

Not exactly the same, but I've got the following error when I shared the model between the two threads:

import threading
import time

model, clip, vae = CheckpointLoaderSimple(Checkpoints.v1_5_pruned_emaonly)

def gen():
    with Workflow():
        conditioning = CLIPTextEncode('beautiful scenery nature glass bottle landscape, , purple galaxy bottle,', clip)
        conditioning2 = CLIPTextEncode('text, watermark', clip)
        latent = EmptyLatentImage(64, 64, 1)
        latent = KSampler(model, 42, 10, 1, 'euler', 'normal', conditioning, conditioning2, latent, 1)
        image = VAEDecode(latent, vae)
        print(util.get_images(image))
threading.Thread(target=gen).start()
time.sleep(1)
threading.Thread(target=gen).start()

Exception in thread Thread-50 (gen):
Traceback (most recent call last):
  File "C:\ProgramData\Miniconda3\envs\py310\lib\threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "d:\venv\lib\site-packages\ipykernel\ipkernel.py", line 761, in run_closure
    _threading_Thread_run(self)
  File "C:\ProgramData\Miniconda3\envs\py310\lib\threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\Chaoses\AppData\Local\Temp\ipykernel_93352\2960785196.py", line 12, in gen
  File "D:\ComfyUI\custom_nodes\ComfyScript\src\comfy_script\runtime\real\nodes.py", line 183, in new
    outputs = getattr(obj, obj.FUNCTION)(*args, **kwds)
  File "D:\ComfyUI\nodes.py", line 1533, in sample
    return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
  File "D:\ComfyUI\nodes.py", line 1500, in common_ksampler
    samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
  File "D:\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\sample_error_enhancer.py", line 9, in informative_sample
    return original_sample(*args, **kwargs)  # This code helps interpret error messages that occur within exceptions but does not have any impact on other operations.
  File "D:\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\control_reference.py", line 47, in refcn_sample
    return orig_comfy_sample(model, *args, **kwargs)
  File "D:\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\utils.py", line 111, in uncond_multiplier_check_cn_sample
    return orig_comfy_sample(model, *args, **kwargs)
  File "D:\ComfyUI\comfy\sample.py", line 45, in sample
    samples = sampler.sample(noise, positive, negative, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
  File "D:\ComfyUI\comfy\samplers.py", line 1031, in sample
    return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
  File "D:\ComfyUI\comfy\samplers.py", line 921, in sample
    return cfg_guider.sample(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
  File "D:\ComfyUI\comfy\samplers.py", line 907, in sample
    output = executor.execute(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
  File "D:\ComfyUI\comfy\patcher_extension.py", line 110, in execute
    return self.original(*args, **kwargs)
  File "D:\ComfyUI\comfy\samplers.py", line 876, in outer_sample
    output = self.inner_sample(noise, latent_image, device, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
  File "D:\ComfyUI\comfy\samplers.py", line 860, in inner_sample
    samples = executor.execute(self, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar)
  File "D:\ComfyUI\comfy\patcher_extension.py", line 110, in execute
    return self.original(*args, **kwargs)
  File "D:\ComfyUI\comfy\samplers.py", line 715, in sample
    samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
  File "d:\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "D:\ComfyUI\comfy\k_diffusion\sampling.py", line 161, in sample_euler
    denoised = model(x, sigma_hat * s_in, **extra_args)
  File "D:\ComfyUI\comfy\samplers.py", line 380, in __call__
    out = self.inner_model(x, sigma, model_options=model_options, seed=seed)
  File "D:\ComfyUI\comfy\samplers.py", line 840, in __call__
    return self.predict_noise(*args, **kwargs)
  File "D:\ComfyUI\comfy\samplers.py", line 843, in predict_noise
    return sampling_function(self.inner_model, x, timestep, self.conds.get("negative", None), self.conds.get("positive", None), self.cfg, model_options=model_options, seed=seed)
  File "D:\ComfyUI\comfy\samplers.py", line 360, in sampling_function
    out = calc_cond_batch(model, conds, x, timestep, model_options)
  File "D:\ComfyUI\comfy\samplers.py", line 196, in calc_cond_batch
    return executor.execute(model, conds, x_in, timestep, model_options)
  File "D:\ComfyUI\comfy\patcher_extension.py", line 110, in execute
    return self.original(*args, **kwargs)
  File "D:\ComfyUI\comfy\samplers.py", line 230, in _calc_cond_batch
    model.current_patcher.prepare_state(timestep)
AttributeError: 'NoneType' object has no attribute 'prepare_state'
[<PIL.Image.Image image mode=RGB size=64x64 at 0x22E80524A60>]

So looks like at least models can't be shared freely. Maybe your error is also caused by some shared states, like top-level global varaibles or some global varaibles used inside some nodes.

Adding some locks may fix this type of errors. For example, to use the same model:

import threading
import time

lock = threading.Lock()
model, clip, vae = CheckpointLoaderSimple(Checkpoints.v1_5_pruned_emaonly)

def gen():
    with Workflow():
        conditioning = CLIPTextEncode('beautiful scenery nature glass bottle landscape, , purple galaxy bottle,', clip)
        conditioning2 = CLIPTextEncode('text, watermark', clip)
        latent = EmptyLatentImage(64, 64, 1)
        with lock:
            latent = KSampler(model, 42, 10, 1, 'euler', 'normal', conditioning, conditioning2, latent, 1)
        image = VAEDecode(latent, vae)
        print(util.get_images(image))
threading.Thread(target=gen).start()
time.sleep(1)
threading.Thread(target=gen).start()

Though locking sampling is not ideal. Cloning the model patcher may be better, if workable.

Ok, here is a demonstration. When I run test2.py (importing the workflow from test1.py) with threading as you suggested, I receive that error. test1.py, test2.py and test_log.txt are attached.

test1.txt
test2.txt
test_log.txt

Can you tell me which node has the error and upload the full error log (full stack trace) here? I just realized that I don't have a NVIDIA GPU at the moment so my tensors will never be on two different devices to get the same error.

The problematic node is FooocusKSampler node. How can I upload the full error log (full stack trace), can you help me please? Do I need to run a logger?

Navigation Menu

Repository navigation

Running multiple workflows just after each other #94

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Running multiple workflows just after each other #94

Description

Activity

Chaoses-Ib commented on Feb 6, 2025

cihangoksu commented on Feb 6, 2025

Chaoses-Ib commented on Feb 6, 2025

cihangoksu commented on Feb 6, 2025

Chaoses-Ib commented on Feb 6, 2025

cihangoksu commented on Feb 7, 2025

Chaoses-Ib commented on Feb 7, 2025

cihangoksu commented on Feb 7, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions