sketchkit.colorization.manga_ninja.MangaNinjaModel¶

class sketchkit.colorization.manga_ninja.MangaNinjaModel(device: str = 'cuda', pretrained_model_name_or_path: str = 'runwayml/stable-diffusion-v1-5', image_encoder_path: str = 'openai/clip-vit-large-patch14', controlnet_model_name_or_path: str = 'lllyasviel/control_v11p_sd15_lineart', torch_dtype: dtype = torch.float16, auto_download: bool = True, **kwargs: Any)[source]¶

MangaNinja model for reference-based line art colorization using SD 1.5.

Paper: “MangaNinja: Line Art Colorization with Precise Reference Following” (CVPR 2025) Repository: https://github.com/ali-vilab/MangaNinjia

Note: MangaNinja is built on SD 1.5, not SDXL. It uses a reference image (instead of text prompts) to guide colorization, achieving remarkable consistency with the reference.

pipeline¶

The loaded custom pipeline.

Type:: MangaNinjiaPipeline

device¶

Computation device.

Type:: str

preprocessor¶

Line art detection preprocessor.

Type:: BatchLineartDetector

__init__(device: str = 'cuda', pretrained_model_name_or_path: str = 'runwayml/stable-diffusion-v1-5', image_encoder_path: str = 'openai/clip-vit-large-patch14', controlnet_model_name_or_path: str = 'lllyasviel/control_v11p_sd15_lineart', torch_dtype: dtype = torch.float16, auto_download: bool = True, **kwargs: Any)[source]¶

Initializes the MangaNinja pipeline.

Parameters:

device (str) – Hardware device to run the model on. Defaults to “cuda”.
pretrained_model_name_or_path (str) – HuggingFace hub path or local path for the base SD 1.5 model.
image_encoder_path (str) – HuggingFace hub path or local path for the CLIP image encoder.
controlnet_model_name_or_path (str) – HuggingFace hub path or local path for the ControlNet model.
torch_dtype (torch.dtype) – PyTorch data type for model weights.
auto_download (bool) – Whether to automatically download missing weights. Defaults to True.
**kwargs (Any) – Additional kwargs passed to diffusers loading methods.

Raises:

FileNotFoundError – If weights are missing and auto_download is False.
RuntimeError – If model initialization fails.

Methods

`__init__`([device, ...])	Initializes the MangaNinja pipeline.
`generate`(control_image, reference_image[, ...])	Generates a colorized image from line art using a reference image.