Mapping Textures to Shapes

Mapping Textures to Shapes

Texture mapping is a powerful way to add deformations to the surface of your objects. This is one of the advanced techniques to complete within the SDK.

During this tutorial we will:

(1) Introduce the operators used to map a texture to a surface

(2) Show how to build a shape graph using our suite of graph building functions.

image

The graph

The Metafold backend create shapes by consuming directed acyclic graphs where each node represents a series of mathematical computations or the initialization of source values. We can build graphs very quickly using the SDK. See here for details on current operations.

Texture mapping requires the use of 3 operators:

  1. LoadVolume is responsible for bringing the data that defines the texture into the graph.
  2. MapTexture creates a scalar field defined at every grid point. This field is generated by mapping the original input points x, y, z to the heightmap space u, v via the different mapping types: "Box", "Cylinder", "Plane", "Sphere". The values of the field at each point are the interpolated values of the heightmap at u, v.
  3. LinearFilter scales and adds the perturbation field created to the original implicit field of our shape.

The result is a new way to add topological complexity to your shape.

DAG created when mapping textures. The shape node can be replaced by any shape you are looking to texture map to. It is good practice to have a Redistance operator at the end of your graphs to ensure you generate true distances.
DAG created when mapping textures. The shape node can be replaced by any shape you are looking to texture map to. It is good practice to have a Redistance operator at the end of your graphs to ensure you generate true distances.

Process

We want to map the following texture onto a hand brace mold.

image
image

The first step is up upload the image and convert it to a heightmap using the convert_image_to_heightmap() job. This jobs takes in any coloured image, converts it to grayscale and normalizes the resulting values to the range [0, 1], where black receives a value of 0, and white receives 1. The output is written to a binary file of 32-bit floats. 

from metafold import MetafoldClient
from metafold.func_type import VolumeAsset, JSONEvaluator
from metafold.func import *
from metafold.util import xform

metafold = MetafoldClient(access_token="...", project_id="...")

image_filename = "octagon.jpg"
if existing := metafold.assets.list(q=f"filename:{image_filename}"):
    image = metafold.assets.update(existing[0].id, image_filename)
else:
    image = metafold.assets.create(image_filename)

heightmap_job = metafold.jobs.run(
    "convert_image_to_heightmap",
    {
       image_filename
    }
)

heightmap = heightmap_job.assets[0]
dim_x, dim_y = heightmap_job.meta["size"]
For details on working on assets see the tutorial here.

Stored in the metadata of the job are the dimensions of the image. We store these for later use.

Now that we have out heightmap we can apply it to a shape. 

# Assume we have defined our shape previously...
source = ...
brace = ...

We take advantage of the LoadVolume operator to take our new heightmap asset and bring it into our graph for evaluation. The resolution parameter must be set to the size of the image. Since this is a 2D texture we set the resolution in z to be 1. 

volume_asset = VolumeAsset()
volume_asset["path"] = heightmap.filename
load_volume = LoadVolume(volume_asset, {"resolution": [dim_x, dim_y, 1]})

We now use the MapTexturePrimitive operator. We use the xform utility function to define a affine transform to the underlying cylinder which we are mapping to. We then use scale to control the size of the mapping in the radial and angular coordinates. 

# We first define a transform we want to apply to the underling primtive.
xform_ = xform(rotation=[44.0, 0.0, 70.0], translation=[90.0, 0.0, 0.0])
pertubation_field = MapTexturePrimitive(
    source, 
    load_volume,
    {
        "shape_type": "Cylinder",
        "shape_xform": xform_,
        "scale": [0.2, 2],
    }
)

To now apply the deformations to our shape, we use the LinearFilter operator. We port the perturbation field through inputb of this operator. Hence we can control the level of deformation to the 0-level set with the scale_b parameter. 

punctured_brace = LinearFilter(shape, pertubation_field, {"scale_b": 2.0})

# We add a redistance to get true distance values. 
punctured_brace =  Redistance(punctured_brace))

graph = JSONEvaluator()
punctured_brace(graph)

We can now evaluate the graph as usual and get the final shape.

metafold.jobs.run(
    "evaluate_graph",
    {
        "graph": graph.json()
    }
)

We now have our final shape that we can render later.

image