Install OmniGen AI Locally – Fast 8-Bit Version 6GB+ GPU – Amazing Text to Image Editing AI

Introduction

Hello there and welcome!

Before we begin, you should note, that THIS video, covers the complete and clean local install, of the fast 8-bit quantization OmniGen version, that currently runs on CUDA capable NVIDIA GPUs, with only 6GB or more VRAM.

If you have an NVIDIA GPU with at least 12GB VRAM, then you’re able of run the original FP32 OmniGen version, and in that case, you probably want to take a look at my other OmniGen local install video, that covers that original FP32 version.

Of course, even if you do have a 12GB or more GPU, you can also run the fast 8-bit quantization OmniGen version, shown in this video, if you’re looking for much faster generation times, with minimal image generation quality loss. The choice is yours.

It’s worth pointing out, that this “fast 8-bit quantization OmniGen version” doesn’t use a different FP8 model, it uses the same original FP32 15 and a half gig one. However, it converts the model’s weights to 8-bit integers instead of 32-bit floating point, which vastly reduces memory requirements and generation times, and in reality, this has minimal impact on the quality of the final image.

OmniGen is completely free, open-source, and can be installed locally, with unlimited private use, which makes it a no-brainer to install and use.

In this tutorial, I’ll walk you through the complete, clean and fresh, step-by-step process, to fully install the fast 8-bit quantization version of OmniGen, aimed at only 6GB plus GPUs, on your local machine, using the user friendly Gradio graphical interface, so you can get straight on with using it, without running into any install errors.

I’ll be installing everything on a Windows 11 PC, running a CUDA capable NVIDIA GPU.

I’ll leave all the links and references that I cover in this tutorial, in the Resources section at the end.

So, let’s crack on.

OmniGen Overview

Before we begin the install, let’s take a quick look at OmniGen’s main functionality, so we know what we can, and can’t do with it, after installation.

Sure it’s just been released, and there’s room for improvement, but it really is a solid start. I honestly believe, text prompt based image and photo editing and manipulation, is the way forward.

OmniGen has common image generation capability. It can process various image generation tasks, including image generation, image editing, controlnet based image-conditional generation, and a lot more.

OmniGen can tackle classical vision tasks. It’s able to process some classical computer vision tasks, such as deblur, derain, inpainting, human pose estimation, and depth estimation.

OmniGen has flexible and controllable image generation built in. You can generate images from text, edit parts of the generated images, generate redraws based on the human poses in the generated images, and extract objects from another image, to integrate into the new image.

OmniGen has referring expression generation capability. You can input multiple images, and use simple, general language to refer to the objects within those images. OmniGen can automatically recognise the necessary objects in each image, and generate new images based on them. No additional operations, such as image cropping or face detection, are required.

OmniGen is also working on inference capabilities, and in-context learning ability. You’ll be able to edit images, without even needing to be precisely specific in your text prompt.

Anyway, let’s get into the installation.

Install Process Overview

The good news is that this install process, is easier and quicker than the one for the original FP32 OmniGen version. Each step is actually pretty straight forward, and quick and easy, to follow along to and complete. Many of you, will already have a lot of the required pieces of software installed on your machine, so if you do, you can jump straight past those sections of the tutorial.

So, let’s get stuck into it.

Install Latest NVIDIA GPU Drivers

Step 1, is to make sure we have the latest drivers installed for our CUDA capable NVIDIA GPU, to utilise CUDA 12.1, and give us the fastest image generation times. If you already know that you have a CUDA capable NVIDIA GPU, with an updated driver, of at least version 527.41, which is needed for CUDA version 12.1, then you can skip to the next step.

CUDA, is basically an NVIDIA technology, that gives a significant processing speed boost to CUDA capable NVIDIA GPUs. Most newer NVIDIA GPUs are CUDA capable.

If you don’t know whether your NVIDIA GPU is CUDA capable, then take a look at the official NVIDIA GPU CUDA list.

To download the correct driver from NVIDIA, we need to know our GPU details. To find our GPU model and VRAM amount, we’ll right-click on our taskbar, and select “Task Manager”. From the left menu, we’ll select “Performance”, and then “GPU”. In the top right hand corner, we can see that I have an NVIDIA GeForce RTX 3070, and if we look down at the bottom of the window, where it says “Dedicated GPU Memory”, we can see that I have 8GB of GPU VRAM. Ignore the 16GB “GPU Memory”, that’s a shared memory number. You should make a mental note of your GPU model, as you’ll need it later.

Let’s find, download and install the latest drivers for our GPU. We’ll open the NVIDIA drivers download page in our browser. This is where we’ll need to select our GPU model, that we made a mental note of previously. Let me select mine. “GeForce”, “RTX 30 Series”, “GeForce RTX 3070”. For “Operating System”, I’m running 64-bit Windows 11, so I’ll select Windows 11. I’ll leave the default “US English” language, and click “Find”.

There’s two types of NVIDIA drivers available. A “Game Ready Driver”, which is best for gaming, offering priority gaming support and speedy fixes. Then there’s the “Studio Driver”, aimed at Content Creators, offering a focus on stability and quality. I’ll select the Game Ready Driver, by clicking on its “View” button, because I like to have all the latest bug fixes and speed improvements.

The download page for my specific driver will open, showing the driver details, including the driver version. We just click the “Download” button. On the “Save As” window, I’ll save the file in the root of my E drive, then click Save. The file will take a little while to download.

Once the file has finished downloading, we’ll navigate to my E drive. In here, we can see the downloaded “driver.exe” file. We’ll double-click this file to begin the installation process.

The installation process first needs to extract the required files. We’ll click “OK” to confirm the default extraction location, and extract the required files. All these files will be auto deleted after installation.

Once all the files have been extracted, the first installation window will pop up. Here, we have the choice to either install the NVIDIA driver AND the “GeForce Experience”, or just the NVIDIA driver. The GeForce Experience adds additional functionality, and auto checks for driver updates in the future. I don’t need the additional bloat, so I’ll select to install only the driver. Then we’ll click the “Agree and Continue” button.

On the next install window, we have the choice between an “Express” or “Custom” install. The main difference between the two options, is that the Express option retains our current NVIDIA settings, and the Custom option allows us to reset all settings during install. We’ll select the Express option, and then click “Next” to start the installation, which will take a bit of time. Eventually, a message will be shown, saying that “NVIDIA Installer has finished”. We’ll click the “Close” button.

Let’s have a quick check to make sure the correct NVIDIA drivers, have been installed correctly. We’ll show our Desktop. We’ll right-click on our Desktop, and select “Show more options”, then select “NVIDIA Control Panel”. If a license agreement confirmation window appears, we’ll just click the “Agree and Continue” button, to accept the terms. On the opening NVIDIA Control Panel window, our driver version and GPU will be shown. Everything looks good. We can close the window.

OK. That’s the latest NVIDIA drivers installed. Onto the next step.

Install Git

Step 2 is to install Git, so we can copy the OmniGen files to our computer in the next step, and use Git to run future updates automatically, if needed. If you already have Git installed, then you can skip to the next step.

We’ll open the Git download page in our browser. I’m running a 64-bit version of Windows, so we’ll click on the Windows link. The “Download for Windows” page will open. Then we want the “Standalone Installer”, so we’ll click on the “64-bit Git for Windows Setup” file link. On the “Save As” window, I’ll save the file in the root of my E drive, then click Save.

Once the file has finished downloading, we’ll navigate to my E drive. In here, we can see the downloaded “Git.exe” file. We’ll double-click this file to begin the installation process. If a “Do you want this app to make changes to your device” warning window pops up, we’ll click the “Yes” button. Then we’ll just click the “Next” button, on the next setup windows, to accept the default installation settings. On the last setup window, we’ll click the “Install” button, to start the installation process. The installation won’t take very long.

Once the installation has completed, the completion window will be shown. We’ll untick the “View Release Notes” checkbox, and click the “Finish” button.

OK. That’s Git installed. Onto the next step.

Clone OmniGen GitHub Repository

Step 3, is to download all the OmniGen files to our computer, by cloning the NewGenAI OmniGen GitHub repository using Git.

I want to install OmniGen in the root of my E drive, so we’ll navigate there. Then, we’ll click into the address bar, and type in “cmd”, and press “Enter”. This will open the Command Prompt in this location. At the prompt, we’ll copy and paste in Syntax 01, to clone the NewGenAI OmniGen GitHub Repository. Then we press “Enter”. It’ll take a little while to download all the files.

Once cloning has completed, the prompt will return to the starting prompt state. We can now close the Command Prompt window.

If we go into my E drive, we can see a new “omnigen” parent folder, and if we go into this folder, we can see all the folders and files that were downloaded to our computer.

OK. That’s OmniGen cloned. Onto the next step.

Install Miniconda

Step 4, is to install Miniconda, which we’ll use to create an OmniGen virtual environment in the next step.

You could just download and install Python, and use that for creating a virtual environment, but Conda is more robust, and offers other capabilities that we’ll utilise.

Although, this step, and the creation and use of a virtual environment, is technically optional, it’s certainly good practice and highly recommended, to keep everything for OmniGen segregated, and prevent any issues with any other software, we have installed on our computer.

If you already have ANY version of Miniconda or Conda installed, then you can skip this step.

We’ll open the Miniconda download page in our browser. We’ll scroll down a bit, to the “Latest Miniconda installer links” section. This is the latest version of Miniconda, that comes bundled with the latest version of Python. We won’t actually be using this base Python version 3.12 at all, instead, when we create virtual environments, we’ll follow best practices, and create them with specific versions of Python that we choose, which Miniconda allows us to do seamlessly. The advantage of using this LATEST Miniconda version, is that it has all the latest bug fixes and improvements contained, so the Conda functionality is better for our purposes.

Anyway, I’m running Windows, so we’ll click on the “Miniconda3 Windows 64-bit” file link. On the “Save As” window, I’ll save the file in the root of my E drive, then click Save.

Once the file has finished downloading, we’ll navigate to my E drive. In here, we can see the downloaded “miniconda3.exe” file. We’ll double-click this file to begin the installation process. We’ll just click “Next” on the first setup window. “I Agree” on the next window. On the next “Install Type” window, we’ll leave it at the recommended default, to install for “Just Me”, and click “Next”. On the next “Install Location” window, we’ll leave it at the default location, under our Username. Then we just click “Next”.

On the last “Advanced Options” window, we’ll leave the first “create shortcuts” checkbox ticked. We’ll also tick the second “add to PATH” checkbox. We won’t worry about the warning message that appears. The warning is based on a legacy issue for mainly non-Windows systems, and for us Windows users, ticking this option, saves us the hassle of fully adding Miniconda to PATH manually, after installation. Then we’ll make sure that the third checkbox is UNTICKED, since we don’t want to add the bundled Python 3.12 to our system registry, as the default version, in case it overwrites any current Python version already set, that our system may use. We’ll also tick the last “clear cache” checkbox, to delete the temporary install files after installation. Then we’ll click “Install”, to start the install process. The installation won’t take very long.

When it’s completed, on the “Installation Complete” window, we’ll click “Next”. On the final window, we’ll untick the two checkboxes, and click the “Finish” button.

OK. That’s Miniconda installed. Onto the next step.

Set Up OmniGen Conda Virtual Environment with Python

Step 5, is to create and activate a virtual environment for OmniGen, named “omnigen”, with Python 3.10, using Conda.

We’ll navigate to, and open the “omnigen” folder that we cloned into my E drive. Then we’ll click into the address bar, and type in “cmd”, and press “Enter”, to launch the Command Prompt window in this location.

At the prompt, we’ll copy and paste in Syntax 02, to create a new Conda environment, named “omnigen”, with the latest version of Python 3.10. Then we press “Enter”. It’ll ask us to confirm the default environment location. We’ll type “y”, and press “Enter”. It won’t take very long to create the virtual environment. Once the environment has been successfully created, the prompt will return to the starting prompt state.

You can always find all the virtual environments that you create, in the miniconda3 environments folder, under your username. I leave the full PATH in the description below for you.

Now we need to activate our new “omnigen” virtual environment. At the prompt, we’ll copy and paste in Syntax 03, to activate the environment. Then we press “Enter”. It won’t take long to activate.

Once it’s activated, we’ll see the virtual environment name “omnigen”, in brackets, preceding the prompt. Don’t close the Command Prompt window, we’ll come back to it shortly.

OK. That’s the OmniGen virtual environment created and activated with Conda. Onto the next step.

Install OmniGen Requirements and Dependencies

Step 6, is to install all the OmniGen requirements and dependencies.

We’ll switch back to our Command Prompt window. Still in the omnigen active virtual environment, at the prompt, we’ll copy and paste in Syntax 04, to download and install a bunch of required packages. Then we just press “Enter”. It’ll take a bit of time, to download all the required packages.

Once everything has completed successfully, the prompt will return to the starting prompt state. Don’t close the Command Prompt window, we’ll come back to it shortly.

OK. That’s the OmniGen requirements and dependencies downloaded and installed. Onto the next step.

Run OmniGen for the 1st Time

Step 7, is to run OmniGen for the first time.

We’ll switch back to our Command Prompt window. Still in the omnigen active virtual environment, at the prompt, we’ll copy and paste in Syntax 05, to launch OmniGen. Then we press “Enter”. It won’t take long to load. If a message saying, “INFO: Could not find files for the given patterns”, is shown, don’t worry about it, this isn’t an error, and presumably will be suppressed in future versions of OmniGen.

A local URL will be shown in the Command Prompt window. We just hold down our “Control” key on our keyboard, and click the URL link with our mouse.

If OmniGen opens in your browser, then congratulations, you’ve just installed OmniGen successfully.

Note that the large 15 and a half gig model, and some additional required files, haven’t yet been downloaded. This will all be done when you generate your very first image, so be prepared to wait a while on your first image generation. This only needs to be done one time. You can check the progress in the Command Prompt window.

If you have any issues with downloading the model, then you can always manually download it from the official Hugging Face page. I’ll leave the URL link in the description below for you.

And in case your wondering, the model is saved in the cache folder, under your username. I’ll leave the full PATH in the description below for you.

Anyway, make sure you don’t skip the next bonus step, to automate launching OmniGen in the future.

Run OmniGen Again with Automation Script

Step 8 is optional, but highly recommended, to save us time and hassle every time we want to launch OmniGen again. Launching OmniGen the manual way, is just too much hassle, plus we need to remember all that syntax. To run it manually, we would need to launch the Command Prompt window from within our omnigen folder, remember and type in the syntax to activate the omnigen conda environment, then finally remember and type in the syntax to run the app using the Gradio Web UI.

Instead of that, how about we just have a file, that we double-click, and all of the above manual steps are done for us. Let’s create just that.

We’ll open our omnigen folder. We’ll right-click in an empty space, select “New”, then “Text Document”. We’ll delete the document default name AND the “.txt” file extension, and rename it to “run.bat”. And then press “Enter”. On the warning window that pops up, we’ll click “Yes” to confirm we want to change the file extension.

We’ll right-click our new “run.bat” file, and select “Edit in Notepad”. Now we just copy and paste in Syntax 06. Depending on the location of YOUR OmniGen install, you’ll need to change the two locations on lines 2 and 3. Then we just save the file and close it.

Now when we want to run OmniGen, we just navigate to our omnigen folder, and double-click our new “run.bat” file. Of course, you could also just create a shortcut to this file on your Desktop, for added convenience.

Then when the local URL link appears in our Command Prompt window, we just “Control” and click it, to open OmniGen using the Gradio Web UI in our browser.

Nice and easy.

Conclusion

Now you know how to do a complete and clean install, of the fast 8-bit quantization OmniGen version, that currently runs on CUDA capable NVIDIA GPUs, with only 6GB or more VRAM.

Now you can get on with experiencing, what I honestly believe, is the future direction of image and photo editing and manipulation, using text prompts.

Anyway, hope you found this tutorial helpful, and I’ll catch you in the next one.

Resources

Links

Official NVIDIA GPU CUDA List:
https://developer.nvidia.com/cuda-gpus

NVIDIA Drivers Download Page:
https://www.nvidia.com/download/index.aspx

Git Download Page:
https://git-scm.com/downloads

Miniconda Download Page:
https://docs.anaconda.com/miniconda/

OmniGen Model Files:
https://huggingface.co/Shitao/OmniGen-v1/tree/main

Syntax

[SYNTAX 01]:
Clone OmniGen GitHub Repository:
git clone https://github.com/newgenai79/omnigen

[SYNTAX 02]:
Create omnigen Conda Environment:
conda create -n omnigen python=3.10

[SYNTAX 03]:
Activate omnigen Conda Environment:
conda activate omnigen

[SYNTAX 04]:
Install OmniGen Requirements and Dependencies:
pip install -r requirements.txt

[SYNTAX 05]:
Run OmniGen (in active virtual environment):
python app.py

[SYNTAX 06]:
OmniGen Automation Run Script:
@echo off
rem Step 1: Change directory to E:\omnigen
cd /d E:\omnigen
rem Step 2: Activate the omnigen conda environment
call conda activate omnigen
rem Step 3: Run OmniGen
python app.py

Paths

Miniconda Environments:
C:\Users\**YOUR USERNAME**\miniconda3\envs

OmniGen Model Saved To:
C:\Users\**YOUR USERNAME**\.cache\huggingface\hub\models–Shitao–OmniGen-v1\snapshots