Text-Conditioned CAD

Overview

“Prompt-to-part” is the natural interface to generative CAD: describe the part and get a parametric feature template. Here a text-conditioned CAD model maps a natural-language description to a bag-of-words embedding, then through a small MLP to a distribution over feature templates (Box, Difference(Box, Cylinder), …); the argmax template is then instantiated with its parameters (complementing the autoregressive CadSequenceGenerator).

using Rasmah

templates = ["Box", "Difference(Box, Cylinder)", "Cylinder"]
corpus = ["a solid box", "a box with a cylindrical hole", "a plain cylinder",
          "box", "block with a hole", "cylinder"]
labels = [1, 2, 3, 1, 2, 3]

model = TextConditionedCad(corpus, templates; hidden = [16])
θ = text_cad_fit(model, corpus, labels; epochs = 600, lr = 1e-2, seed = 0)

211-element Vector{Float64}:
 -0.09927851361184353
  0.5869530502047858
  0.29271508128540985
  0.5961057646415998
  0.19144739325253698
  0.016047914065004127
  0.12110368095184959
  0.12294255967641515
  0.08166486156723918
 -0.16779456782669724
  ⋮
 -0.7400303470325379
 -0.4991999002359841
  0.8260420535667067
 -0.687117635553187
  0.6304555279213475
  0.9402346956305264
  0.6617544295353608
 -0.46385834876580606
 -0.5972266921560696

Verification

The trained classifier maps each description to its template:

println("\"a box with a hole\" → ", text_to_feature(model, θ, "a box with a hole"))
println("\"a solid box\"       → ", text_to_feature(model, θ, "a solid box"))
println("\"a plain cylinder\"   → ", text_to_feature(model, θ, "a plain cylinder"))

"a box with a hole" → Difference(Box, Cylinder)
"a solid box"       → Box
"a plain cylinder"   → Cylinder

Next steps

• Graph generative design — the structure-level view of the same feature graph.
• Generative CAD — learned CAD feature sequences.