sketchkit.sketch2image.controlnet package

Module contents

ControlNet Model Implementation for Sketch-to-Image Conversion (SDXL Version)

This module encapsulates the diffusers StableDiffusionXLControlNetPipeline for generating images conditioned on sketch inputs.

class sketchkit.sketch2image.controlnet.ControlNetModel(device: str = 'cuda', controlnet_model: str = 'xinsir/controlnet-scribble-sdxl-1.0', sd_model: str = 'stabilityai/stable-diffusion-xl-base-1.0', vae_model: str = 'madebyollin/sdxl-vae-fp16-fix', torch_dtype: dtype = torch.float16, safety_checker: bool = False, **kwargs: Any)[source]

Bases: object

ControlNet model for scribble-conditioned sketch-to-image conversion using SDXL.

pipeline

The loaded diffusers pipeline.

Type:

StableDiffusionXLControlNetPipeline

controlnet_model

The name/path of the loaded ControlNet.

Type:

str

sd_model

The name/path of the base SDXL model.

Type:

str

device

Computation device (‘cuda’ or ‘cpu’).

Type:

str

generate(control_image: Image, prompt: str, size: int | Tuple[int, int] | None = None, negative_prompt: str = 'longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality', num_inference_steps: int = 30, guidance_scale: float = 7.5, controlnet_conditioning_scale: float = 1.0, seed: int | None = None, **kwargs: Any) → Image[source]

Generates an image from a scribble-conditioned sketch using SDXL.

Parameters:

  • control_image (Image.Image) – The input conditioning scribble/sketch image.

  • prompt (str) – Text prompt describing the desired output.

  • size (Optional[Union[int, Tuple[int, int]]]) – Target resolution. If None, it auto-scales proportionally based on a 1024x1024 baseline.

  • negative_prompt (str) – Text prompt detailing what to avoid.

  • num_inference_steps (int) – Number of denoising steps. Defaults to 30.

  • guidance_scale (float) – CFG scale for text guidance. Defaults to 7.5.

  • controlnet_conditioning_scale (float) – Strength of the ControlNet. Defaults to 1.0.

  • seed (Optional[int]) – Random seed for deterministic generation.

  • **kwargs (Any) – Additional arguments for the pipeline.

Returns:

The generated image.

Return type:

Image.Image

Raises:

RuntimeError – If image generation fails during pipeline execution.