Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: ComfyUI Custom Workflow Support #1336

Open
Michelklingler opened this issue Mar 28, 2024 · 8 comments
Open

feat: ComfyUI Custom Workflow Support #1336

Michelklingler opened this issue Mar 28, 2024 · 8 comments

Comments

@Michelklingler
Copy link
Contributor

Is your feature request related to a problem? Please describe.

The feature will enhance the text to image capability of OpenWebUI.
Especially for company using Multimedia and Image/video content creation pipeline.
By enabling this feature the user could use multiple workflow not only to generate image from text, but would open multiple other image generation custom features. Compatible ComfyUI Workflow could be shared within the OpenWebUI Prompt/ModelFile webpage.

Describe the solution you'd like

The option to load an Exported workflow into the setting windows of OpenWebUI.
With a few basic setting exposed: Dimension, steps, models, etc... (like it is already the case with the current integration)

Describe alternatives you've considered

Manually update the comfyui.py file in the repo for a hardcoded alternative

@Michelklingler
Copy link
Contributor Author

I did try to manually update the comfyui.py file with a basic SDXL refiner workflow that works well, but it does not seems to work. I get the following error when I click the generate image button:

Screenshot 2024-03-28 at 8 57 33 AM

comfyui_sdxl.py.zip

@Michelklingler
Copy link
Contributor Author

Michelklingler commented Mar 28, 2024

FYI - I manage to make a custom ComfyUI work flow work!
I was missing a "," and also as ComfyUI nodes ID are generated randomly depending at which moment you create the node, it needs to be updated accordingly in the code to point the variable for the prompt, model, steps and dimension.

Below is the updated comfyui.py file that can be replaced in /app/backend/apps/images/utils/comfyui.py.
comfyui.py.zip

If you have enough ressource to load SDXL the REFINER on a machine it provide pretty high quality image as output.
Just make sure you download the 2 models below in your ComfyUI checkpoint server before inferring.
Setup the settings like the screenshot attached for best results.

Screenshot 2024-03-28 at 1 05 29 PM
Screenshot 2024-03-28 at 1 07 42 PM

@tjbck tjbck changed the title ComfyUI Custom Workflow Support feat: ComfyUI Custom Workflow Support Mar 28, 2024
@Michael-YongWang
Copy link

@cmalpass
Copy link

cmalpass commented Apr 6, 2024

@Michelklingler I second this request.

I have different workflows for different kinds of output types and subjects. Would be great to switch between them as needed so that I can get high quality results for each rather than subpar one-size-fits-all as it is right now. Love having comfyui as a part of the application, so this would just make it that much better.

EDIT: I looked into this some more and found that this is offered as part of an add-on for ComfyUI that provides API endpoints that list workflows: https://github.com/pythongosssss/ComfyUI-Custom-Scripts

It does not appear to yet be something that ComfyUI can provide out of the box.

This ComfyUI add-on uses a couple of api endpoints that could facilitate:
Provide a list of workflows by name: /pysssss/workflows
Get the defintion of a workflow: pysssss/workflows/workflownameFromPreviousEndpoint

The output from the second endpoint could be manipulated and inserted into the request script in place of the default workflow.

The relevant add-on files are here: https://github.com/pythongosssss/ComfyUI-Custom-Scripts/blob/main/web/js/workflows.js#L50

I don't know if I'll have an opportunity to write anything out to support this but it's definitely of interest to me.

@schuster-rainer
Copy link

it doesn't work this way. The change to support this is more complex. Each workflow is unique. So either the open-webui team sets up some standards what need to be in a workflow or we put a mapping mechanism in place that let's a user use any workflow he likes put it in some folder in open-webui and a configuration file "xpath" or some other mechanism to tell open-webui where to put the negative prompt ... and so on. So many options on how to advance this. If you even be possible to give the LLM tools/functions it can call, to customize the negative prompt, model, seed and so on, from within the chat

@alex13by
Copy link

alex13by commented May 3, 2024

It may be possible to support some special workflows in the form of lists, such as SDXL, LCM, cascade, etc.

@ER-EPR
Copy link

ER-EPR commented Jul 10, 2024

maybe just a COMFYUI_DEFAULT_PROMPT replacement UI should be enough

@silentoplayz
Copy link
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants