feat: add IP-Adapter support for SD 1.5 and SDXL

[fszontagh]

Adds IP-Adapter image-prompt conditioning for SD 1.5 and SDXL.

A reference image is encoded with a CLIP-Vision (ViT-H/14) encoder, projected into image tokens, and injected through a decoupled cross-attention added to each attn2 layer of the UNet. It composes with Control Net, so appearance (reference image) and pose (e.g. OpenPose) can be combined.

Usage

--ip-adapter <path>            IP-Adapter weight file
--ip-adapter-image <path>      reference image
--ip-adapter-strength <float>  injection strength (default 1.0)

Requires a CLIP-Vision ViT-H encoder via --clip_vision (the vit-h IP-Adapter variants reuse it). See docs/ip_adapter.md.

Implementation

• image_proj adapter runner; token count and dims auto-detected from the weights, so SD 1.5 (768) and SDXL (2048) share one path.
• Decoupled cross-attention added to CrossAttention (attn2 only), lazily created when to_k_ip/to_v_ip are present, so plain models are unchanged.
• Image tokens applied on the conditional pass only (skipped on the unconditional passes), matching the diffusers IP-Adapter CFG behavior.
• name_conversion maps the IP-Adapter tensors onto the UNet attn2 layers; the SDXL index-to-layer table is generated from the transformer-depth layout.

Verification

• SD 1.5 (ip-adapter_sd15) and SDXL (ip-adapter_sdxl_vit-h): all adapter tensors bind, reference content transfers, strength behaves monotonically.
• Verified combined with OpenPose Control Net (appearance + posed control).

Notes

• Vanilla IP-Adapter only; Plus (Resampler) and FaceID are follow-ups.
• SDXL VAE decode at 1024 is memory heavy; use --vae-tiling on limited VRAM.