zer0int

CLIP-fine-tune. Fine-tuning code for CLIP models

ComfyUI-HunyuanVideo-Nyan. Text Encoders finally matter 🤖🎥 - scale CLIP & LLM influence! + a Nerdy Transformer Shuffle node

Long-CLIP. Scripts for use with LongCLIP, including fine-tuning Long-CLIP

ComfyUI-Long-CLIP. ComfyUI implementation of Long-CLIP, including Flux node: LongCLIPTextEncodeFlux

CLIP-fine-tune-registers-gated. Vision Transformers Needs Registers. And Gated MLPs. And +20M params. Tiny modality gap ensues!

ComfyUI-Nuke-a-Text-Encoder. For SDXL, SD1.5, Flux. Nuke T5 and let CLIP guide Flux.1 on its own! Or let let random guide Flux.1! Or load a CLIP crazy opinion embedding about your image and let that guide the AI!

CLIP-XAI-GUI. CLIP GUI - XAI app ~ explainable (and guessable) AI with ViT & ResNet models

CLIP-Interrogator-LongCLIP-hallucinwords. CLIP Interrogator, fully in HuggingFace Transformers 🤗, with LongCLIP & CLIP's own words and / or *your* own words!

CLIP-SAE-finetune. Sparse Autoencoders (SAE) vs CLIP fine-tuning fun.

CLIP-ViT-visualization. What do CLIP Vision Transformers learn? Feature Visualization can show you!

CLIP-text-image-interpretability. Get CLIP ViT text tokens about an image, visualize attention as a heatmap.

CLIP-txt2img-diffusers-scripts. Example scripts for using [my] fine-tuned CLIP models with HuggingFace 🤗

ComfyUI-CLIP-Flux-Layer-Shuffle. Comfy Nodes (and a CLI script) for shuffling around layers in transformer models, creating a curious confusion.

ComfyUI-LLama3-Layer-Shuffle-Prompting. Shuffle LLama-3.2 layers and have it prompt an image. Node works with any model - Flux, SD3, SDXL...

CLIP-DeepDream. Deep Dreaming with CLIP Vision Transformers

Auto-GPT. Auto-GPT + CLIP vision for stable v0.3.1

CLIP-gradient-ascent-embeddings. Use CLIP to create matching texts + embeddings for given images; useful for XAI, adversarial training

CLIP-Direct-Ascent-Synthesis. Like a CLIP + VQGAN. Except without a VQGAN.

bitsandbytes-windows. Windows bitsandbytes 0.46.0 for Python 3.10, CUDA 12.8

ComfyUI-GPT2-Layer-Shuffle-Prompting. Shuffle GPT-2's layers and have it prompt an image. Node works with any model - Flux, SD3, SDXL...

CLIP-Layer-Deck-Shuffle. Experimental removal / shuffling of layers in CLIP ViT + Text Transformer

ComfyUI-workflows. Workflows to implement fine-tuned CLIP Text Encoders with ComfyUI / SD, SDXL, SD3

CLIP-tokenizer. A simple CLIP tokenizer encode / decode script. For when you need to know CLIP's tokenization and / or token IDs for some reason.

CLIPInversion. What do we learn from inverting CLIP models? And what does a CLIP 'see' in an image?

GPT-OSS-20B-Windows-16GB-RTX4090. No Hopper architecture (RTX 5090, etc.) required! <16 GB VRAM, Windows.

OpenVision. OpenVision: A Fully-Open, Cost-Effective Family of Advanced Vision Encoders for Multimodal Learning. + A closer look in PyTorch!

Inf-CLIP. Geometric Parametrization GmP-Inf-CLIP modification of: "Breaking the Memory Barrier: Near Infinite Batch Size Scaling for Contrastive Loss". A super memory-efficiency CLIP training scheme.

CLIP-test-time-registers. Official Pytorch implementation of "Vision Transformers Don't Need Trained Registers"

CLIP-attention-entropy. A small script for CLIP attn entropy plots

CLIP-generative-adversarial. Projected Gradient Descent (PGD), inverted and amplified -> prompt & generate images with CLIP

sam_faster. SAM: Sharpness-Aware Minimization (PyTorch)

CLIP_Surgery. CLIP Surgery for all CLIP models including custom fine-tuned CLIP

vit-visualization. vit-visualization with large CLIP ViT models

CLIP-text-to-image. CLIP guiding self towards an image, from text prompt

CLIP-vs-human-cosine-similarity-game. How much do YOU align to an AI / CLIP?

CLIP-HeadHunter. Head-Hunter: A Visual Bias Explorer. Attention Head Max Visualization to find, rank, and visualize heads; map bias; see what a CLIP 'sees'.

plop-for-CLIP. PLoP applied to CLIP

CLIP-ResNet-classic-DeepDream. Classic original Inception style DeepDream, but with CLIP ResNet. And CLIP ViT for comparison.

LLMorse. Talk Morse code to multimodal LLM using your voice. Beep-boop!

CLIP-ringelpiez. Plotting a CLIP vs. a CLIP in arbitrary ways.

CLIP-second_order_lens. Jupyter Notebook

text2image_safety. Python

VITAL-CLIP-class-concept-visualization. CLIP class/concept/facet/inner neuron visualization (CNN and ViT); plus: king-man+woman=queen in Vision Transformers