sketchkit.colorization package

Subpackages

Submodules

sketchkit.colorization.colorizer module

Colorizer for sketch colorization tasks.

class sketchkit.colorization.colorizer.Colorizer(method: str = 'controlnet_lineart', device: str = 'cuda', **kwargs: Any)[source]

Bases: object

A class for sketch colorization using various AI methods.

This class provides a unified interface to colorize vector sketches or rasterized linearts using ControlNet-based models or reference-based methods like MangaNinja.

method

The colorization method being used (e.g., ‘controlnet_lineart’).

Type:

str

device

The computation device (‘cuda’ or ‘cpu’).

Type:

str

_model

The underlying AI model instance initialized based on the method.

AVAILABLE_METHODS = ['controlnet_lineart', 'controlnet_scribble', 'manga_ninja']
_generate_controlnet(control_image: Image, prompt: str, size: int | Tuple[int, int] | None = None, **kwargs: Any) → Image[source]

Generates an image using ControlNet-based models.

Parameters:

  • control_image (Image.Image) – The processed conditioning image.

  • prompt (str) – Textual prompt.

  • size (Optional[Union[int, Tuple[int, int]]]) – Final output size.

  • **kwargs (Any) – Additional control parameters.

Returns:

The generated image.

Return type:

Image.Image

_generate_manga_ninja(control_image: Image, reference_image: Image, size: int | Tuple[int, int] | None = None, **kwargs: Any) → Image[source]

Generates a colorized image using MangaNinja.

Parameters:

  • control_image (Image.Image) – The input line art image.

  • reference_image (Image.Image) – The reference image for color guidance.

  • size (Optional[Union[int, Tuple[int, int]]]) – Processing resolution.

  • **kwargs (Any) – MangaNinja-specific parameters including: - is_lineart (bool): Whether input is already line art. - guidance_scale_ref (float): Reference guidance scale. - guidance_scale_point (float): Point guidance scale. - num_inference_steps (int): Denoising steps. - seed (int): Random seed. - point_ref (torch.Tensor): Point map on reference image. - point_main (torch.Tensor): Point map on line art image.

Returns:

The colorized image.

Return type:

Image.Image

_generate_with_control_image(control_image: Image, prompt: str, size: int | Tuple[int, int] | None = None, reference_image: Image | None = None, **kwargs: Any) → Image[source]

Generates the final image using the underlying model.

Parameters:

  • control_image (Image.Image) – The processed conditioning image.

  • prompt (str) – Textual prompt.

  • size (Optional[Union[int, Tuple[int, int]]]) – Final output size.

  • reference_image (Optional[Image.Image]) – Reference image for MangaNinja.

  • **kwargs (Any) – Additional control parameters.

Returns:

The generated image.

Return type:

Image.Image

_prepare_control_image(image: ndarray | Image) → Image[source]

Standardizes the control image format to a PIL Image.

Parameters:

image (Union[np.ndarray, Image.Image]) – The raw input image array or PIL object.

Returns:

The standardized PIL Image in the correct color mode.

Return type:

Image.Image

Raises:

TypeError – If the input type is unsupported.

_prepare_reference_image(image: Image | ndarray) → Image[source]

Standardizes the reference image format to a PIL RGB Image.

Parameters:

image (Union[Image.Image, np.ndarray]) – The reference image.

Returns:

The standardized PIL Image in RGB mode.

Return type:

Image.Image

_run_with_image(image: Image | ndarray, prompt: str, size: int | Tuple[int, int] | None, input_size: Tuple[int, int] | None, reference_image: Image | ndarray | None = None, **kwargs: Any) → Image[source]

Processes a raster image (PIL or numpy array) for colorization.

Parameters:

  • image (Union[Image.Image, np.ndarray]) – The input raster image.

  • prompt (str) – Textual prompt for colorization.

  • size (Optional[Union[int, Tuple[int, int]]]) – Output image size.

  • input_size (Optional[Tuple[int, int]]) – Target size to resize the input image.

  • reference_image (Optional[Union[Image.Image, np.ndarray]]) – Reference image for MangaNinja method.

  • **kwargs (Any) – Additional generation parameters.

Returns:

Generated colorized image.

raises ValueError:

If the numpy array shape is invalid.

Return type:

Image.Image

_run_with_sketch(sketch: Sketch, prompt: str, size: int | Tuple[int, int] | None, input_size: Tuple[int, int] | None, reference_image: Image | ndarray | None = None, **kwargs: Any) → Image[source]

Processes a vector Sketch object for colorization.

Parameters:

  • sketch (Sketch) – The input vector sketch.

  • prompt (str) – Textual prompt for colorization.

  • size (Optional[Union[int, Tuple[int, int]]]) – Output image size.

  • input_size (Optional[Tuple[int, int]]) – Target size for rasterization.

  • reference_image (Optional[Union[Image.Image, np.ndarray]]) – Reference image for MangaNinja method.

  • **kwargs (Any) – Additional generation parameters.

Returns:

Generated colorized image.

Return type:

Image.Image

_sketch_to_raster(sketch: Sketch, size: int | Tuple[int, int] = 1024, background_color: Tuple[float, float, float] = (1, 1, 1), stroke_width: int = 3, fit_canvas: bool = True) → ndarray[source]

Converts a vector Sketch to a rasterized numpy array.

Parameters:

  • sketch (Sketch) – The vector sketch to render.

  • size (Union[int, Tuple[int, int]]) – Canvas dimensions. Defaults to 1024.

  • background_color (Tuple[float, float, float]) – RGB background color. Defaults to white.

  • stroke_width (int) – Thickness of the rendered strokes. Defaults to 3.

  • fit_canvas (bool) – Whether to scale paths to fit the canvas. Defaults to True.

Returns:

The rasterized image as a numpy array.

Return type:

np.ndarray

run(input_data: Sketch | Image | ndarray, prompt: str = 'masterpiece, best quality, vibrant colors, flat color, highly detailed, anime illustration style', size: int | Tuple[int, int] | None = None, input_size: Tuple[int, int] | None = None, reference_image: Image | ndarray | None = None, **kwargs: Any) → Image[source]

Executes the colorization process on the input sketch or image.

Parameters:

  • input_data (Union[Sketch, Image.Image, np.ndarray]) – The input lineart. Can be a vector Sketch, a PIL Image, or a numpy array.

  • prompt (str) – Text description to guide the colorization. Defaults to a high-quality anime illustration prompt. Note: Not used by ‘manga_ninja’ method (reference image is used instead).

  • size (Optional[Union[int, Tuple[int, int]]]) – The desired output size. If None, it infers the size from the input. For ‘manga_ninja’, the internal processing resolution is 512x512.

  • input_size (Optional[Tuple[int, int]]) – Optional resolution to resize the input to before processing.

  • reference_image (Optional[Union[Image.Image, np.ndarray]]) – Reference image for colorization guidance. Required for ‘manga_ninja’ method. Ignored for ControlNet-based methods.

  • **kwargs (Any) – Additional generation parameters. Common parameters include: - num_inference_steps (int): Number of denoising steps. - guidance_scale (float): CFG scale for text guidance. - seed (int): Random seed for deterministic generation. - negative_prompt (str): Negative text prompt. For ‘manga_ninja’ method, additional parameters include: - is_lineart (bool): If True, input is already line art. - guidance_scale_ref (float): Guidance scale for reference image influence. - guidance_scale_point (float): Guidance scale for point control influence. - point_ref (torch.Tensor): Point map on reference image (1,1,H,W). - point_main (torch.Tensor): Point map on line art image (1,1,H,W).

Returns:

The resulting colorized image.

Return type:

Image.Image

Raises:

  • TypeError – If the input_data type is not supported.

  • ValueError – If ‘manga_ninja’ method is used without a reference_image.

Module contents

Sketch Colorization

This module provides a unified API for colorizing vector sketches.

class sketchkit.colorization.Colorizer(method: str = 'controlnet_lineart', device: str = 'cuda', **kwargs: Any)[source]

Bases: object

A class for sketch colorization using various AI methods.

This class provides a unified interface to colorize vector sketches or rasterized linearts using ControlNet-based models or reference-based methods like MangaNinja.

method

The colorization method being used (e.g., ‘controlnet_lineart’).

Type:

str

device

The computation device (‘cuda’ or ‘cpu’).

Type:

str

_model

The underlying AI model instance initialized based on the method.

AVAILABLE_METHODS = ['controlnet_lineart', 'controlnet_scribble', 'manga_ninja']
_generate_controlnet(control_image: Image, prompt: str, size: int | Tuple[int, int] | None = None, **kwargs: Any) → Image[source]

Generates an image using ControlNet-based models.

Parameters:

  • control_image (Image.Image) – The processed conditioning image.

  • prompt (str) – Textual prompt.

  • size (Optional[Union[int, Tuple[int, int]]]) – Final output size.

  • **kwargs (Any) – Additional control parameters.

Returns:

The generated image.

Return type:

Image.Image

_generate_manga_ninja(control_image: Image, reference_image: Image, size: int | Tuple[int, int] | None = None, **kwargs: Any) → Image[source]

Generates a colorized image using MangaNinja.

Parameters:

  • control_image (Image.Image) – The input line art image.

  • reference_image (Image.Image) – The reference image for color guidance.

  • size (Optional[Union[int, Tuple[int, int]]]) – Processing resolution.

  • **kwargs (Any) – MangaNinja-specific parameters including: - is_lineart (bool): Whether input is already line art. - guidance_scale_ref (float): Reference guidance scale. - guidance_scale_point (float): Point guidance scale. - num_inference_steps (int): Denoising steps. - seed (int): Random seed. - point_ref (torch.Tensor): Point map on reference image. - point_main (torch.Tensor): Point map on line art image.

Returns:

The colorized image.

Return type:

Image.Image

_generate_with_control_image(control_image: Image, prompt: str, size: int | Tuple[int, int] | None = None, reference_image: Image | None = None, **kwargs: Any) → Image[source]

Generates the final image using the underlying model.

Parameters:

  • control_image (Image.Image) – The processed conditioning image.

  • prompt (str) – Textual prompt.

  • size (Optional[Union[int, Tuple[int, int]]]) – Final output size.

  • reference_image (Optional[Image.Image]) – Reference image for MangaNinja.

  • **kwargs (Any) – Additional control parameters.

Returns:

The generated image.

Return type:

Image.Image

_prepare_control_image(image: ndarray | Image) → Image[source]

Standardizes the control image format to a PIL Image.

Parameters:

image (Union[np.ndarray, Image.Image]) – The raw input image array or PIL object.

Returns:

The standardized PIL Image in the correct color mode.

Return type:

Image.Image

Raises:

TypeError – If the input type is unsupported.

_prepare_reference_image(image: Image | ndarray) → Image[source]

Standardizes the reference image format to a PIL RGB Image.

Parameters:

image (Union[Image.Image, np.ndarray]) – The reference image.

Returns:

The standardized PIL Image in RGB mode.

Return type:

Image.Image

_run_with_image(image: Image | ndarray, prompt: str, size: int | Tuple[int, int] | None, input_size: Tuple[int, int] | None, reference_image: Image | ndarray | None = None, **kwargs: Any) → Image[source]

Processes a raster image (PIL or numpy array) for colorization.

Parameters:

  • image (Union[Image.Image, np.ndarray]) – The input raster image.

  • prompt (str) – Textual prompt for colorization.

  • size (Optional[Union[int, Tuple[int, int]]]) – Output image size.

  • input_size (Optional[Tuple[int, int]]) – Target size to resize the input image.

  • reference_image (Optional[Union[Image.Image, np.ndarray]]) – Reference image for MangaNinja method.

  • **kwargs (Any) – Additional generation parameters.

Returns:

Generated colorized image.

raises ValueError:

If the numpy array shape is invalid.

Return type:

Image.Image

_run_with_sketch(sketch: Sketch, prompt: str, size: int | Tuple[int, int] | None, input_size: Tuple[int, int] | None, reference_image: Image | ndarray | None = None, **kwargs: Any) → Image[source]

Processes a vector Sketch object for colorization.

Parameters:

  • sketch (Sketch) – The input vector sketch.

  • prompt (str) – Textual prompt for colorization.

  • size (Optional[Union[int, Tuple[int, int]]]) – Output image size.

  • input_size (Optional[Tuple[int, int]]) – Target size for rasterization.

  • reference_image (Optional[Union[Image.Image, np.ndarray]]) – Reference image for MangaNinja method.

  • **kwargs (Any) – Additional generation parameters.

Returns:

Generated colorized image.

Return type:

Image.Image

_sketch_to_raster(sketch: Sketch, size: int | Tuple[int, int] = 1024, background_color: Tuple[float, float, float] = (1, 1, 1), stroke_width: int = 3, fit_canvas: bool = True) → ndarray[source]

Converts a vector Sketch to a rasterized numpy array.

Parameters:

  • sketch (Sketch) – The vector sketch to render.

  • size (Union[int, Tuple[int, int]]) – Canvas dimensions. Defaults to 1024.

  • background_color (Tuple[float, float, float]) – RGB background color. Defaults to white.

  • stroke_width (int) – Thickness of the rendered strokes. Defaults to 3.

  • fit_canvas (bool) – Whether to scale paths to fit the canvas. Defaults to True.

Returns:

The rasterized image as a numpy array.

Return type:

np.ndarray

run(input_data: Sketch | Image | ndarray, prompt: str = 'masterpiece, best quality, vibrant colors, flat color, highly detailed, anime illustration style', size: int | Tuple[int, int] | None = None, input_size: Tuple[int, int] | None = None, reference_image: Image | ndarray | None = None, **kwargs: Any) → Image[source]

Executes the colorization process on the input sketch or image.

Parameters:

  • input_data (Union[Sketch, Image.Image, np.ndarray]) – The input lineart. Can be a vector Sketch, a PIL Image, or a numpy array.

  • prompt (str) – Text description to guide the colorization. Defaults to a high-quality anime illustration prompt. Note: Not used by ‘manga_ninja’ method (reference image is used instead).

  • size (Optional[Union[int, Tuple[int, int]]]) – The desired output size. If None, it infers the size from the input. For ‘manga_ninja’, the internal processing resolution is 512x512.

  • input_size (Optional[Tuple[int, int]]) – Optional resolution to resize the input to before processing.

  • reference_image (Optional[Union[Image.Image, np.ndarray]]) – Reference image for colorization guidance. Required for ‘manga_ninja’ method. Ignored for ControlNet-based methods.

  • **kwargs (Any) – Additional generation parameters. Common parameters include: - num_inference_steps (int): Number of denoising steps. - guidance_scale (float): CFG scale for text guidance. - seed (int): Random seed for deterministic generation. - negative_prompt (str): Negative text prompt. For ‘manga_ninja’ method, additional parameters include: - is_lineart (bool): If True, input is already line art. - guidance_scale_ref (float): Guidance scale for reference image influence. - guidance_scale_point (float): Guidance scale for point control influence. - point_ref (torch.Tensor): Point map on reference image (1,1,H,W). - point_main (torch.Tensor): Point map on line art image (1,1,H,W).

Returns:

The resulting colorized image.

Return type:

Image.Image

Raises:

  • TypeError – If the input_data type is not supported.

  • ValueError – If ‘manga_ninja’ method is used without a reference_image.