Skip/Reset CLIP position_ids FIX

This post was reserved from

Tensor CLIP problem fix (fix for dropped chunks distortons, hands, faces, embeddings using, etc)

The problem legend:

This is an int64 tensor that has the values from 0 to 76, merging will convert these to float and introduce errors. For example in AnythingV3 the value 76 has become 75.9975, which is cast back to int64 when loaded by the webui, resulting in 75.

What we got if we use model with 'broken' CLIP:

The model has no reactions to chunks. For example: prompt 'smile woman sleepy', the network will ignore 'smile' chunk and you need to use (smile:1.5) or something like that so that the network can possibly (not guaranteed) begin to respond to the chunk.

Original research:

How to check tensors in the finished model:

You can check any of your model/mixes for broken CLIPs through this small useful python script
Download link:

Or you can use extension for WEB-UI:

You need:

  1. Installed python (by default we use python: 3.10.6)
  2. Installed pip3 (by default packaged with Python installation)
  3. Installed torch package (command: pip3 install torch torchvision torchaudio --extra-index-url )
  4. Installed safetensors (command: pip3 install safetensors )

After installation open your command line and check how it works with steps:

  1. python
  2. import torch
    x = torch.rand(5, 3)

Everything is normal if you see something like:
tensor([[0.3380, 0.3845, 0.3217],
[0.8337, 0.9050, 0.2650],
[0.2979, 0.7141, 0.9069],
[0.1449, 0.1132, 0.1375],
[0.4675, 0.3947, 0.1426]])

How to use python tensor check script:

  1. Place in folder with your models
  2. Open your Terminal app
  3. Type for CKPT model: python YOURMODENAME.ckpt
    Type for SAFETENSORS model: python YOURMODENAME.safetensors
  4. Press Enter
    (optional: if you want to load the model into VRAM you need to add 'cuda' in the end, for example:
    python \models\Stable-diffusion\sd-1-5.ckpt cuda
    By default models load into RAM)

Optional using:

  1. Place script in stable-diffusion-webui folder
  2. Open Terminal and type: .\venv\scripts\activate
  3. Type: python \models\Stable-diffusion\YOURMODELNAME.[ckpt or safetensors]

After running you will see the tensor matrix. Your model health is perfect if you see something like on a screenshot (a sequence of integers from 0 to 76). If you see different values - you need to fix CLIP in your model.

How to fix:

If you merge your model:

For merging with more precision method with finetune weight blocks use webui extension and press checkbox Skip/Reset CLIP position_ids to FORCE RESET

If you need to fix finished model which you downloaded:

(Option 1) CLIP import: use :

WARNING: It may complete change your CLIP .pt file training data. If you see 'strange' results on renders after importing CLIP - use another CLIP fix method.

  1. Install extension
  2. Extract reference CLIP model without bad tensors from model you want (check tensors before extracting), for example you can use standard SD model v1-5-pruned-emaonly.ckpt:
    a) Choose v1-5-pruned-emaonly.ckpt in Source, press Load
    b) Advanced tab
    c) Architecture: SD-v1, Class CLIP-v1, Component CLIP-v1-SD
    d) Name the file below (by default, click Export
    e) Clear loaded model
  3. CLIP fix the model with bad tensors:
    a) Choose YOUR MODEL in Source, press Load
    b) Advanced tab
    c) Architecture: SD-v1, Class CLIP-v1, Component CLIP-v1-SD
    d) Action -> choose extracted reference file (, press Import
    e) Name your new model at left upper corner, press Save
    f) Check your new model with tensor check script

(Option 1a) Fix CLIP ids only:
a) Choose only in components (without import .pt): Architecture: SD-v1, Class CLIP-v1, Component CLIP-v1-SD, then save
b) Go to WEBUI settings, Model Toolkit, Click on checkbox 'Fix broken CLIP position IDs'. After that you can just load model and save it without manual choosing components in option 'a'.

(Option 2) Fix finished model through :

  1. Choose Preset Weights 'ALL_A'
  2. Click on checkbox Save as safetensors
  3. Click on checkbox Skip/Reset CLIP position_ids -> FORCE RESET
  4. Choose Model A -> YOUR MODEL
  5. Choose Model B -> YOUR MODEL
  6. Name your fixed model in Output Model Name
  7. Press Run Merge
  8. Check new model through in Tensor checker script


Illustrative example of the power of CLIP fix.

Picture 1 - AshMix base model with bad tensors
Picture 2 - fixed AshMix model
Prompt: loli
Negative: -
Seed: 1


Picture 1 - AshMix base model with bad tensors
Picture 2 - fixed AshMix model
Prompt: woman
Negative: -
Seed: 1


Pub: 21 Jan 2023 12:10 UTC
Edit: 15 Mar 2023 06:10 UTC
Views: 10484