sketchkit.sketch2model.methods package

Submodules

sketchkit.sketch2model.methods.doubao module

Doubao-Seed3D API implementation for sketch-to-3D model conversion.

sketchkit.sketch2model.methods.doubao.sketch_to_3d_doubao_impl(sketch_input: Sketch | str | ndarray | Image, output_path: str | None = None, api_key: str | None = None, api_base: str | None = None, **kwargs) → str[source]

Convert sketch to 3D model using Doubao-Seed3D API.

Parameters:

  • sketch_input – Input sketch, can be Sketch object, file path (str), PIL Image, or numpy array

  • output_path – Output file path (if None, will be auto-generated)

  • api_key – Doubao API key (if None, will read from environment variable DOUBAO_API_KEY)

  • api_base – API base URL (if None, uses default)

  • **kwargs – Additional parameters - model_name: Model name (default: “doubao-seed3d-1-0-250928”) - file_format: Output format (default: “glb”) - subdivision_level: Subdivision level (default: “medium”) - max_wait_time: Maximum wait time in seconds (default: 600) - poll_interval: Polling interval in seconds (default: 5)

Returns:

Generated 3D model file path

sketchkit.sketch2model.methods.sens_standalone module

SENS Standalone Inference using TorchScript models.

This module provides SENS inference without requiring the original SENS codebase. Only requires: torch, numpy, PIL, skimage

Models are automatically downloaded on first use, following the dataset download pattern.

class sketchkit.sketch2model.methods.sens_standalone.SENSStandaloneInference(model_dir: str | None = None, device: str = 'cpu', skip_integrity_check: bool = False)[source]

Bases: object

Standalone SENS inference using exported TorchScript models.

This provides the same functionality as the original SENS but without requiring the SENS source code.

Required files in model_dir:

  • sens_encoder.pt

  • sens_occ_former_full.pt

  • sens_from_gmm.pt

  • sens_affine.pt

  • sens_occ_head.pt

  • sens_config.pt

_augment_cropped_square(image: ndarray, res: int = 256) → ndarray[source]

Crop to content and resize to square.

_check_integrity() → bool[source]

Check if all required model files exist (following dataset pattern).

_download()[source]

Download SENS models from Google Drive (following dataset download pattern).

Downloads the entire folder and extracts it.

_find_model_dir(model_dir: str | None) → Path | None[source]

Find the model directory (legacy method, kept for compatibility).

_get_grid_samples(res: int) → Tensor[source]

Get grid sample coordinates.

_load_models()[source]

Load TorchScript models.

_mesh_adaptive_level(occ_np, voxel_size, voxel_origin)[source]

Fallback with adaptive level.

compute_occupancy_grid(z: Tensor, res: int = 128) → Tensor[source]

Compute occupancy grid.

encode_sketch(sketch: Tensor) → Tuple[Tensor, Tensor][source]

Encode sketch to latent representation.

infer(image_input: str | ndarray | Image, res: int = 128) → Tuple[ndarray, ndarray] | None[source]

Convert sketch to 3D mesh.

Parameters:

  • image_input – Image path, numpy array, or PIL Image

  • res – Occupancy grid resolution (higher = more detail, slower)

Returns:

(vertices, faces) tuple or None if failed

merge_zh_step_a(zh: Tensor, mu: Tensor, p: Tensor, phi: Tensor, eigen: Tensor) → Tensor[source]

Merge GMM information into zh (core SENS operation).

mesh_from_occupancy(occ_values: Tensor) → Tuple[ndarray, ndarray] | None[source]

Generate mesh using marching cubes.

prepare_features(zh: Tensor, out_cls: Tensor, thresh_mask: float = 0.0) → Tensor[source]

Prepare features with full GMM merge.

preprocess_sketch(image_input: str | ndarray | Image) → Tensor[source]

Preprocess sketch image for SENS.

Parameters:

image_input – Image path, numpy array, or PIL Image

Returns:

Preprocessed tensor [1, 1, 256, 256]

static save_mesh_obj(vertices: ndarray, faces: ndarray, output_path: str)[source]

Save mesh as OBJ file.

sketchkit.sketch2model.methods.sens_standalone.is_standalone_available(model_dir: str | None = None) → bool[source]

Check if SENS standalone models are available.

sketchkit.sketch2model.methods.sens_standalone.sketch_to_3d_sens_standalone(sketch_input: Sketch | str | ndarray | Image, output_path: str | None = None, model_dir: str | None = None, device: str = 'cpu', resolution: int = 128, **kwargs) → str[source]

Convert sketch to 3D model using SENS standalone inference.

Parameters:

  • sketch_input – Input sketch (Sketch object, file path, numpy array, or PIL Image)

  • output_path – Output OBJ file path (auto-generated if None)

  • model_dir – Directory containing TorchScript models

  • device – Computing device (‘cpu’ or ‘cuda’)

  • resolution – Occupancy grid resolution (default: 128)

Returns:

Path to output OBJ file

Return type:

str

sketchkit.sketch2model.methods.teddy_monstermash module

Teddy/MonsterMash sketch-to-3D conversion module.

This module provides functions to convert sketches to 3D models using the full MonsterMash/Teddy skeleton-based inflation algorithm.

sketchkit.sketch2model.methods.teddy_monstermash.generate_teddy_input_images(gray_sketch: ndarray, check_single_contour: bool = True) → Tuple[ndarray, ndarray][source]

Generate outline and region images for Teddy/MonsterMash sketch2model.

According to SKETCH2MODEL_USAGE.md: - outlineImg: White background (255) with black outline (0) - regionImg: Black background (0) with white filled region (255)

Parameters:

  • gray_sketch – Preprocessed grayscale image (H, W) as uint8 with white background and black lines

  • check_single_contour – If True, validate that only one main contour exists

Returns:

Tuple of (outline_img, region_img) both as uint8 arrays

Raises:

ValueError – If no contour found or multiple large contours detected

sketchkit.sketch2model.methods.teddy_monstermash.preprocess_raster_for_teddy(raster_sketch: Image | ndarray, target_size: int = 512) → ndarray[source]

Preprocess raster sketch for Teddy/MonsterMash processing.

Converts RGB image to grayscale, resizes to target size, and ensures correct polarity (white background, black lines).

Parameters:

  • raster_sketch – Input raster image as PIL Image or numpy array (H, W, 3) or (H, W) as uint8

  • target_size – Target size for resizing (default: 512)

Returns:

Preprocessed grayscale image (target_size, target_size) as uint8 with white background (255) and black lines (0)

sketchkit.sketch2model.methods.teddy_monstermash.sketch_to_3d_teddy_impl(sketch_input: Sketch | str | ndarray, output_path: str | None = None, **kwargs) → str[source]

Convert sketch to 3D model using Teddy/MonsterMash method.

This function provides a unified interface consistent with other sketch-to-3D methods.

Parameters:

  • sketch_input – Input sketch, can be Sketch object, file path (str), or numpy array (H, W) or (H, W, 3) uint8

  • output_path – Output OBJ file path (if None, will be auto-generated)

  • **kwargs – Additional parameters - triangle_opts: Triangle library options (default: “pqa25QYY”) - subs_factor: Subsampling factor (default: 2) - smooth_factor: Smoothing factor (default: 10) - default_inflation: Default inflation amount (default: 2.0) - region_inflations: List of inflation amounts per region (optional) - check_single_contour: Whether to check single-connected contour (default: True)

Returns:

Path to generated OBJ file

Return type:

str

Example

>>> from sketchkit.sketch2model import sketch_to_3d
>>> output_path = sketch_to_3d('sketch.png', method='teddy', output_path='output.obj')
>>> print(f"Model saved to: {output_path}")
sketchkit.sketch2model.methods.teddy_monstermash.teddy_from_raster(raster_sketch: Image | ndarray, output_path: str | None = None, temp_dir: str | None = None, triangle_opts: str = 'pqa25QYY', subs_factor: int = 2, smooth_factor: int = 10, default_inflation: float = 2.0, region_inflations: list | None = None, check_single_contour: bool = True) → str[source]

Convert raster sketch to 3D mesh using full Teddy/MonsterMash algorithm.

This function performs the complete pipeline: 1. Preprocess raster image (grayscale, resize, polarity check) 2. Generate outline and region images 3. Call C++ sketch2model function 4. Save result to OBJ file

Parameters:

  • raster_sketch – Input raster image as PIL Image or numpy array (H, W, 3) or (H, W) as uint8

  • output_path – Output OBJ file path. If None, generates a temp path

  • temp_dir – Temporary directory for output. If None, uses system temp

  • triangle_opts – Triangle library options (default: “pqa25QYY”)

  • subs_factor – Subsampling factor (default: 2)

  • smooth_factor – Smoothing factor (default: 10)

  • default_inflation – Default inflation amount (default: 2.0)

  • region_inflations – Optional list of inflation amounts per region

  • check_single_contour – If True, validate single connected contour

Returns:

Path to generated OBJ file

Raises:

  • RuntimeError – If C++ module is not available

  • ValueError – If preprocessing or validation fails

sketchkit.sketch2model.methods.teddy_monstermash.validate_teddy_inputs(outline_img: ndarray, region_img: ndarray) → bool[source]

Validate that outline and region images meet Teddy requirements.

Parameters:

  • outline_img – Outline image (should be white bg, black outline)

  • region_img – Region image (should be black bg, white region)

Returns:

True if valid, raises ValueError if invalid

Module contents

Sketch2Model Methods

Implementation modules for different sketch-to-3D model conversion methods.

class sketchkit.sketch2model.methods.SENSStandaloneInference(model_dir: str | None = None, device: str = 'cpu', skip_integrity_check: bool = False)[source]

Bases: object

Standalone SENS inference using exported TorchScript models.

This provides the same functionality as the original SENS but without requiring the SENS source code.

Required files in model_dir:

  • sens_encoder.pt

  • sens_occ_former_full.pt

  • sens_from_gmm.pt

  • sens_affine.pt

  • sens_occ_head.pt

  • sens_config.pt

_augment_cropped_square(image: ndarray, res: int = 256) → ndarray[source]

Crop to content and resize to square.

_check_integrity() → bool[source]

Check if all required model files exist (following dataset pattern).

_download()[source]

Download SENS models from Google Drive (following dataset download pattern).

Downloads the entire folder and extracts it.

_find_model_dir(model_dir: str | None) → Path | None[source]

Find the model directory (legacy method, kept for compatibility).

_get_grid_samples(res: int) → Tensor[source]

Get grid sample coordinates.

_load_models()[source]

Load TorchScript models.

_mesh_adaptive_level(occ_np, voxel_size, voxel_origin)[source]

Fallback with adaptive level.

compute_occupancy_grid(z: Tensor, res: int = 128) → Tensor[source]

Compute occupancy grid.

encode_sketch(sketch: Tensor) → Tuple[Tensor, Tensor][source]

Encode sketch to latent representation.

infer(image_input: str | ndarray | Image, res: int = 128) → Tuple[ndarray, ndarray] | None[source]

Convert sketch to 3D mesh.

Parameters:

  • image_input – Image path, numpy array, or PIL Image

  • res – Occupancy grid resolution (higher = more detail, slower)

Returns:

(vertices, faces) tuple or None if failed

merge_zh_step_a(zh: Tensor, mu: Tensor, p: Tensor, phi: Tensor, eigen: Tensor) → Tensor[source]

Merge GMM information into zh (core SENS operation).

mesh_from_occupancy(occ_values: Tensor) → Tuple[ndarray, ndarray] | None[source]

Generate mesh using marching cubes.

prepare_features(zh: Tensor, out_cls: Tensor, thresh_mask: float = 0.0) → Tensor[source]

Prepare features with full GMM merge.

preprocess_sketch(image_input: str | ndarray | Image) → Tensor[source]

Preprocess sketch image for SENS.

Parameters:

image_input – Image path, numpy array, or PIL Image

Returns:

Preprocessed tensor [1, 1, 256, 256]

static save_mesh_obj(vertices: ndarray, faces: ndarray, output_path: str)[source]

Save mesh as OBJ file.

sketchkit.sketch2model.methods.is_standalone_available(model_dir: str | None = None) → bool[source]

Check if SENS standalone models are available.

sketchkit.sketch2model.methods.sketch_to_3d_doubao_impl(sketch_input: Sketch | str | ndarray | Image, output_path: str | None = None, api_key: str | None = None, api_base: str | None = None, **kwargs) → str[source]

Convert sketch to 3D model using Doubao-Seed3D API.

Parameters:

  • sketch_input – Input sketch, can be Sketch object, file path (str), PIL Image, or numpy array

  • output_path – Output file path (if None, will be auto-generated)

  • api_key – Doubao API key (if None, will read from environment variable DOUBAO_API_KEY)

  • api_base – API base URL (if None, uses default)

  • **kwargs – Additional parameters - model_name: Model name (default: “doubao-seed3d-1-0-250928”) - file_format: Output format (default: “glb”) - subdivision_level: Subdivision level (default: “medium”) - max_wait_time: Maximum wait time in seconds (default: 600) - poll_interval: Polling interval in seconds (default: 5)

Returns:

Generated 3D model file path

sketchkit.sketch2model.methods.sketch_to_3d_sens_standalone(sketch_input: Sketch | str | ndarray | Image, output_path: str | None = None, model_dir: str | None = None, device: str = 'cpu', resolution: int = 128, **kwargs) → str[source]

Convert sketch to 3D model using SENS standalone inference.

Parameters:

  • sketch_input – Input sketch (Sketch object, file path, numpy array, or PIL Image)

  • output_path – Output OBJ file path (auto-generated if None)

  • model_dir – Directory containing TorchScript models

  • device – Computing device (‘cpu’ or ‘cuda’)

  • resolution – Occupancy grid resolution (default: 128)

Returns:

Path to output OBJ file

Return type:

str

sketchkit.sketch2model.methods.sketch_to_3d_teddy_impl(sketch_input: Sketch | str | ndarray, output_path: str | None = None, **kwargs) → str[source]

Convert sketch to 3D model using Teddy/MonsterMash method.

This function provides a unified interface consistent with other sketch-to-3D methods.

Parameters:

  • sketch_input – Input sketch, can be Sketch object, file path (str), or numpy array (H, W) or (H, W, 3) uint8

  • output_path – Output OBJ file path (if None, will be auto-generated)

  • **kwargs – Additional parameters - triangle_opts: Triangle library options (default: “pqa25QYY”) - subs_factor: Subsampling factor (default: 2) - smooth_factor: Smoothing factor (default: 10) - default_inflation: Default inflation amount (default: 2.0) - region_inflations: List of inflation amounts per region (optional) - check_single_contour: Whether to check single-connected contour (default: True)

Returns:

Path to generated OBJ file

Return type:

str

Example

>>> from sketchkit.sketch2model import sketch_to_3d
>>> output_path = sketch_to_3d('sketch.png', method='teddy', output_path='output.obj')
>>> print(f"Model saved to: {output_path}")
sketchkit.sketch2model.methods.teddy_from_raster(raster_sketch: Image | ndarray, output_path: str | None = None, temp_dir: str | None = None, triangle_opts: str = 'pqa25QYY', subs_factor: int = 2, smooth_factor: int = 10, default_inflation: float = 2.0, region_inflations: list | None = None, check_single_contour: bool = True) → str[source]

Convert raster sketch to 3D mesh using full Teddy/MonsterMash algorithm.

This function performs the complete pipeline: 1. Preprocess raster image (grayscale, resize, polarity check) 2. Generate outline and region images 3. Call C++ sketch2model function 4. Save result to OBJ file

Parameters:

  • raster_sketch – Input raster image as PIL Image or numpy array (H, W, 3) or (H, W) as uint8

  • output_path – Output OBJ file path. If None, generates a temp path

  • temp_dir – Temporary directory for output. If None, uses system temp

  • triangle_opts – Triangle library options (default: “pqa25QYY”)

  • subs_factor – Subsampling factor (default: 2)

  • smooth_factor – Smoothing factor (default: 10)

  • default_inflation – Default inflation amount (default: 2.0)

  • region_inflations – Optional list of inflation amounts per region

  • check_single_contour – If True, validate single connected contour

Returns:

Path to generated OBJ file

Raises:

  • RuntimeError – If C++ module is not available

  • ValueError – If preprocessing or validation fails