On the relationship of node-trees and syntactic expressions

First things first, you're probably thinking to yourself, "What the heck are node-trees and syntactic expressions?" and, "Why should I care if they're related?" . Have no fear, behind every big word is a longer explanation, hopefully consisting of smaller words.

"Node trees" are networks of operators, that have inputs and outputs which can be connected to create a complex procedure. Common uses of node trees involve shader operations as in Maya's Hypershade and the the Slim template system. The entire workflow of SideFX Houdini and Apple's Shake centers around node trees. Node trees very flexible and can give rise to an almost limitless number of effects from the same sets of building blocks. They are also easy to uderstand, as each input and output is rendered visually in the workspace.

"Syntactic Expression" is a fancy word for a valid statement in a programming language. For example, "2 + 1;" is a syntactic expression in C or rsl. Often the term "syntactic expression" is used to refer to statements in a what's called "prefix notation." Normal mathematics is written in "infix notation" which means the operator appears between the operands. In the example above '+' is the operator and '1' and '2' are the operands. Prefix notation moves the operator to the beginning of the statement so it becomes "(+ 2 1)". The paranthesis enclose statements written in this manner. Prefix notation is convenient to use because it makes statements very regular and allows any operand to be replaced by another syntatic expression without changing the meaning of the expression. "(* (+ (* 2 3) (/ 1 6)) 2)" is equivalent to "2 * (2 * 3 + 1 / 6)." This demonstrates that prefix notation also eliminates the need for established precedece rules such as "multiplication before addition and subtraction" and "division and multiplication in the order they appear".

Now that the terms are defined, how are they related? Any tree i=of nodes can be flipped upside down an written as a single syntactic expression, and any syntactic expression can be drawn as a node tree. This means tools like houdini, shake, or maya's hypershade can be used to generate ideas for code, and that almost anything that can be done with code can be done in terms of nodes, provided there is a 1 to 1 correspondence between nodes and functions.

As an example, let's look at the definition of the fBm function, that was used in the flame shader. in rsl it looks like:

float fBm (point p; float octaves, lacunarity, gain )
    float sum = 0, amp = 1;
    point pp = P;
    float i;
    for( i = 0; i < octaves; i += 1 )
        sum += amp * noise(pp);
	amp *= gain;
	pp *= lacunarity;
    return sum;

Okay so basically it just layers noise on a point. Let's start by rewriting the fBm function in LISP, a programming language designed around syntactic expressions and the use of prefix notation.

(defun fBm (P octaves lacunarity gain)
  (let ((sum 0) (amp 1) (pp P))
    (dotimes octaves
      (setf sum (+ sum (* amp (noise pp))))
      (setf amp (* amp gain))
      (setf pp (* pp lacunarity))

Don't let the line breaks fool you, this is a single syntactic expression (you can count the paranthsis if you don't believe me). It does the same thing as the code above, but now in a single statement tree. Anywhere we want to use fBm from now on, it can be called like so "(fBm pointP numOctves myLac aGain)". Let's try to convert this definition to a node tree. Each block of paranthesis beomes an single node named by the first word after the paranthesis. Every word after that becomes a connection to the nodes inputs. Any paranthesis enclosing a node indicate its outputs will connect to the enclosing node.

As you can see the expression maps out rather easily to a node tree. The iterate node looks a little funny as we have to assume it has some mechanism for determinig what it should repeat. And the let node maps oddly because it establishes a lexical scope in lisp code, but in a node structure it would just provide a few attributes for other nodes to manipulate. On the balance, it's a rather good approximation.

From here on out, we can visualize fBm as a single node with five inputs and single numeric output. That's the beauty of syntactic expressions, since they are atomic, it's easy to replace a function's expansion with its name to abstract away complexity. In Houdini, this would be the equivalent of collapsing a group of nodes into a subnet.

Let's use the new fBm node to construct a flame shader. Basically a flame shader is made up of three components, a color ramp from yellow to red, an opacity mask to make it wispy, and a displacement to deform the object. Each of these involves fBm to generate layers of noise. So we can visualize the bottom of the tree as our complete shader, and the three nodes immediately abouve it must be Ci (the color information), Oi (opacity), and displace. As a syntactic expression it might look like "(shader Ci Oi displace)".

The next step will be to expand each of those in to their components. Starting with Ci, we know we need to be able to use two colors, fBm, and the value of t, the vertical axis of texture space. So we can expand Ci to look like "(Ci color1 color2 fBm t)". Obviously fBm will need some parameters so the next level of expansion migh look like "(Ci color1 color2 (fBm P oct lac gain) t)".

Moving on to Oi, it can depends on t and fBm. Oi can use the same parameters for fBm, which will generate the same output. Oi becomes "(Oi t (fBm P oct lac gain))".

Displace depends on a cos moving along the t axis, and noise. It can be written as "(displace (cos t) (fBm P oct2 lac2 gain2))". Note that displace feeds different parameters to fBm, that will result in different output. It would appear in a node tree as a seperate instance of a fBm node, whereas Ci and Oi share an FBm node.

At this point the flame share looks like this

 (Ci color1 color2 (fBm P oct lac gain) t)
 (Oi t (fBm P oct lac gain))
 (displace (cos t) (fBm P oct2 lac2 gain2)))

The user of this shader will probably want to be able to play with the settings so we should probably add another element called user parameters to the syntatcic expression. At this point we can also draw it as a complete node tree

(shader (user-parameters)
 (Ci color1 color2 (fBm P oct lac gain) t)
 (Oi t (fBm P oct lac gain))
 (displace (+ (cos t) (fBm P oct2 lac2 gain2))))

At this point the only things left to do are actually define, how the functions Ci, Oi, and displace work and convert the shader definition to renderman shading language so we can use it.

(defun displace (amt)
  (calculatenormal (+ P (* (normalize N) amt))))

(defun Ci (color1 color2 noise_val height)
  (mix color1 color2 (smoothstep 0 1 (- noise_val height))))

(defun Oi (height noise_val)
  (let ((central_trans (clamp .2 1 (- 1 (dotproduct (faceforward N I) (* -1 I))))))
    (* (- 1 (smoothstep 0 1 (- height noiseval))  central_trans))))

(defun shader (userparamters Color Opacity Normal)
  (shade (* Color Opacity) Normal))

And in rsl

color flame_color( color color1, color2; float noise_val, height)
     return mix(color1, color2, smoothstep(0, 1, noise_val + height));

float opacity( float height, noise_val; normal n; vector i )
     float central_trans = clamp(.2, 1, 1 - n.normalize(-i));
     return (1 - smoothstep(0, 1, height - noise_val)) * central_trans;

normal flame_displace( float noise_val, height, offset; point p; normal n )
     float wave_disp = cos((height + offset) * 3.14) * (1 + cos(height + offset) * .01);
     float disp = (wave_disp + noise_val) * .4; 
     p = p + normalize(n) * disp;
     return calculatenormal( p );

surface flame(
     float offset = .5;
     color color1 = (.8, .5, .1);
     color color2 = (1, 0, 0);
     float flame_octaves = 4;
     float flame_lacunarity = 1.7;
     float flame_gain = 1.3;
     point p = transform( "object", P);
     float flame_noise = (fBm(p, filterwidthp(p), flame_octaves, flame_lacunarity, flame_gain) / 2 );
     N = flame_displace( 2 * flame_noise, t, offset, P, normalize(N) );
     Oi = Os * opacity( t, flame_noise, N, I );
     Ci = Cs * Oi * flame_color( color1, color2, flame_noise, t );

Ideally, this has shown that despite the apparent complexity of renderman code, it can be visualized in the same node based manner as houdini and shake procedure trees. The intermediary conversion to Lisp is merely for didactic purposes and to illustrate the mapping of nodes to functions calls. If you're interested, though, you can learn more about Lisp by visiting Successful Lisp. In both houdini and shake, nodes have controls built in to them. To simulate this, just add more parameters to your functions and place arbitrary, or better yet, user editable values in those spots.