Skip to content

Commit fe1cf68

Browse files
rrombrromb
committed
update examples for release
1 parent edb2eb9 commit fe1cf68

5 files changed

Lines changed: 10 additions & 23 deletions

File tree

‎README.md‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,13 @@ new checkpoints. The following list provides an overview of all currently availa
88

99
## News
1010

11-
**February X, 2023**
11+
**February 27, 2023**
12+
1213
*Stable UnCLIP 2.1*
13-
- New stable diffusion finetune (_Stable unCLIP 2.1_, [HuggingFace](https://huggingface.co/stabilityai/stable-unclip-2-1)) at 768x768 resolution,
14-
based on SD2.1-768. This model allows for image variations and mixing operations as described in TODO, and, thanks to its modularity, can be combined with other models
15-
such as [KARLO](https://github.com/kakaobrain/karlo). Documentation [here](doc/UNCLIP.MD).
14+
- New stable diffusion finetune (_Stable unCLIP 2.1_, [HuggingFace](https://huggingface.co/stabilityai/)) at 768x768 resolution,
15+
based on SD2.1-768. This model allows for image variations and mixing operations as described in [*Hierarchical Text-Conditional Image Generation with CLIP Latents*](https://arxiv.org/abs/2204.06125), and, thanks to its modularity, can be combined with other models
16+
such as [KARLO](https://github.com/kakaobrain/karlo). Documentation [here](doc/UNCLIP.MD). Comes in two variants: [*Stable unCLIP-L*](TODO) and [*Stable unCLIP-H*](TODO), which are conditioned on CLIP
17+
ViT-L and ViT-H image embeddings, respectively.
1618

1719

1820
**December 7, 2022**

‎assets/stable-samples/stable-unclip/unclip-variations.png‎

1.69 MB
Loading

‎assets/stable-samples/stable-unclip/unclip-variations_noise.png‎

1.47 MB
Loading

‎doc/UNCLIP.MD‎

Lines changed: 4 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,14 @@
11
### Stable unCLIP
2-
_++++++ NOTE: preliminary checkpoints for internal testing ++++++_
32

43
[unCLIP](https://openai.com/dall-e-2/) is the approach behind OpenAI's [DALL·E 2](https://openai.com/dall-e-2/),
54
trained to invert CLIP image embeddings.
65
We finetuned SD 2.1 to accept a CLIP ViT-L/14 image embedding in addition to the text encodings.
76
This means that the model can be used to produce image variations, but can also be combined with a text-to-image
87
embedding prior to yield a full text-to-image model at 768x768 resolution.
9-
We provide two models, trained on OpenAI CLIP-L and OpenCLIP-H image embeddings, respectively, available
10-
_[TODO: +++prelim private upload on HF+++]_ from [https://huggingface.co/stabilityai/stable-unclip-preview](https://huggingface.co/stabilityai/stable-unclip-preview).
8+
We provide two models, trained on OpenAI CLIP-L and OpenCLIP-H image embeddings, respectively, available from [https://huggingface.co/stabilityai/](TODO).
119
To use them, download from Hugging Face, and put and the weights into the `checkpoints` folder.
1210
#### Image Variations
13-
![image-variations-l-1](../assets/stable-samples/stable-unclip/houses_out.jpeg)
14-
![image-variations-l-2](../assets/stable-samples/stable-unclip/plates_out.jpeg)
15-
16-
_++TODO: Input images from the DIV2K dataset. check license++_
11+
![image-variations-l-1](../assets/stable-samples/stable-unclip/unclip-variations.png)
1712

1813
Run
1914

@@ -24,16 +19,7 @@ to launch a streamlit script than can be used to make image variations with both
2419
These models can process a `noise_level`, which specifies an amount of Gaussian noise added to the CLIP embeddings.
2520
This can be used to increase output variance as in the following examples.
2621

27-
**noise_level = 0**
28-
![image-variations-l-3](../assets/stable-samples/stable-unclip/oldcar000.jpeg)
29-
30-
**noise_level = 500**
31-
![image-variations-l-4](../assets/stable-samples/stable-unclip/oldcar500.jpeg)
32-
33-
**noise_level = 800**
34-
![image-variations-l-6](../assets/stable-samples/stable-unclip/oldcar800.jpeg)
35-
36-
22+
![image-variations-noise](../assets/stable-samples/stable-unclip/unclip-variations_noise.png)
3723

3824

3925
### Stable Diffusion Meets Karlo
@@ -51,7 +37,7 @@ wget https://arena.kakaocdn.net/brainrepo/models/karlo-public/v1.0.0.alpha/0b623
5137
wget https://arena.kakaocdn.net/brainrepo/models/karlo-public/v1.0.0.alpha/85626483eaca9f581e2a78d31ff905ca/prior-ckpt-step%3D01000000-of-01000000.ckpt
5238
cd ../../
5339
```
54-
and the finetuned SD2.1 unCLIP-L checkpoint _[TODO: +++prelim private upload on HF+++]_ from [https://huggingface.co/stabilityai/stable-unclip-preview](https://huggingface.co/stabilityai/stable-unclip-preview), and put the ckpt into the `checkpoints folder`
40+
and the finetuned SD2.1 unCLIP-L checkpoint from [https://huggingface.co/stabilityai/](https://huggingface.co/stabilityai/TODO), and put the ckpt into the `checkpoints folder`
5541

5642
Then, run
5743

‎scripts/streamlit/stableunclip.py‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -276,7 +276,6 @@ def load_model_from_config(config, ckpt, verbose=False, vae_sd=None):
276276
version = st.selectbox("Model Version", list(VERSION2SPECS.keys()), 0)
277277
use_karlo = version in ["Stable unCLIP-L"] and st.checkbox("Use KARLO prior", False)
278278
state = init(version=version, load_karlo_prior=use_karlo)
279-
st.info(state["msg"])
280279
prompt = st.text_input("Prompt", "a professional photograph")
281280
negative_prompt = st.text_input("Negative Prompt", "")
282281
scale = st.number_input("cfg-scale", value=10., min_value=-100., max_value=100.)

0 commit comments

Comments
 (0)