sketchkit.colorization.manga_ninja package

Subpackages

Module contents

MangaNinja Model for Reference-based Line Art Colorization (SD 1.5 Version)

Paper: “MangaNinja: Line Art Colorization with Precise Reference Following” (CVPR 2025) Repository: https://github.com/ali-vilab/MangaNinjia

Note: MangaNinja is built on SD 1.5, not SDXL. It uses a reference image (instead of text prompts) to guide colorization, achieving remarkable consistency with the reference.

class sketchkit.colorization.manga_ninja.MangaNinjaModel(device: str = 'cuda', pretrained_model_name_or_path: str = 'runwayml/stable-diffusion-v1-5', image_encoder_path: str = 'openai/clip-vit-large-patch14', controlnet_model_name_or_path: str = 'lllyasviel/control_v11p_sd15_lineart', torch_dtype: dtype = torch.float16, auto_download: bool = True, **kwargs: Any)[source]

Bases: object

MangaNinja model for reference-based line art colorization using SD 1.5.

Paper: “MangaNinja: Line Art Colorization with Precise Reference Following” (CVPR 2025) Repository: https://github.com/ali-vilab/MangaNinjia

Note: MangaNinja is built on SD 1.5, not SDXL. It uses a reference image (instead of text prompts) to guide colorization, achieving remarkable consistency with the reference.

pipeline

The loaded custom pipeline.

Type:

MangaNinjiaPipeline

device

Computation device.

Type:

str

preprocessor

Line art detection preprocessor.

Type:

BatchLineartDetector

_ensure_hf_models_cached(pretrained_model_name_or_path, image_encoder_path, controlnet_model_name_or_path)[source]
_load_pipeline(pretrained_model_name_or_path, image_encoder_path, controlnet_model_name_or_path, device, annotator_ckpts_path, manga_denoising_unet_path, manga_reference_unet_path, manga_controlnet_path, manga_point_net_path)[source]
generate(control_image: Image, reference_image: Image, prompt: str = '', size: int | Tuple[int, int] | None = None, negative_prompt: str = '', num_inference_steps: int = 50, guidance_scale: float = 7.5, guidance_scale_ref: float = 9.0, guidance_scale_point: float = 15.0, is_lineart: bool = False, seed: int | None = None, point_ref: Tensor | None = None, point_main: Tensor | None = None, **kwargs: Any) → Image[source]

Generates a colorized image from line art using a reference image.

Parameters:

  • control_image (Image.Image) – The input line art image to colorize.

  • reference_image (Image.Image) – The reference image providing color guidance.

  • prompt (str) – Not used by MangaNinja (kept for interface compatibility).

  • size (Optional[Union[int, Tuple[int, int]]]) – Target resolution. MangaNinja operates at 512x512 internally. Defaults to 512.

  • negative_prompt (str) – Not used by MangaNinja (kept for interface compatibility).

  • num_inference_steps (int) – Number of denoising steps. Defaults to 50.

  • guidance_scale (float) – Not used directly; guidance_scale_ref and guidance_scale_point are used instead.

  • guidance_scale_ref (float) – Guidance scale for reference image influence. Higher values make the model follow the reference more closely. Defaults to 9.0.

  • guidance_scale_point (float) – Guidance scale for point control influence. Higher values make the model follow point guidance more closely. Defaults to 15.0.

  • is_lineart (bool) – If True, the input is already a line art image and no additional line art extraction is needed. Defaults to False.

  • seed (Optional[int]) – Random seed for deterministic generation.

  • point_ref (Optional[torch.Tensor]) – Point map on the reference image for point-guided colorization. Shape: (1, 1, H, W).

  • point_main (Optional[torch.Tensor]) – Point map on the line art image for point-guided colorization. Shape: (1, 1, H, W).

  • **kwargs (Any) – Additional arguments.

Returns:

The colorized image.

Return type:

Image.Image

Raises:

RuntimeError – If image generation fails during pipeline execution.

sketchkit.colorization.manga_ninja._check_weights_available() → bool[source]
sketchkit.colorization.manga_ninja._download_file_with_fallback(hf_hub_download, repo_id, filename, local_dir)[source]
sketchkit.colorization.manga_ninja._download_via_subprocess(repo_id, filename, local_dir, endpoint=None)[source]
sketchkit.colorization.manga_ninja._download_weights()[source]
sketchkit.colorization.manga_ninja._get_weights_dir() → str[source]
sketchkit.colorization.manga_ninja._is_network_error(exc: BaseException) → bool[source]
sketchkit.colorization.manga_ninja._prefetch_hf_models()[source]
sketchkit.colorization.manga_ninja._with_clean_env(func, remove_proxy=False, remove_endpoint=False)[source]