Tim Jones
Twitter GitHub RSS feed

Introducing StitchUp: "Generating Shaders from HLSL Shader Fragments" implemented in XNA 4.0

or "A stitch in time saves lots of mucking about with shader permutation explosion..."

Want to skip to the end?

A bit of background

HLSL shaders have been around for a while now. There are a multitude of different effects that you can create with shaders. They are very powerful, and taken on their own, fairly easy to work with. However, problems start to creep in when you're writing a game with several different shaders (what I'm really talking about are effects, which combine vertex and pixel shaders into a single compilable unit). As soon as you start working with multiple effects, you hit the usual problems:

This boils down to "how do I handle shader permutations?" Of course, there are many ways of solving this problem, ranging from hard-coded #if statements, through to fully-fledged node-based shader graphs.

What does it do?

StitchUp is one possible solution of the problem of shader permutation explosion. It is an implementation, using an XNA 4.0 Content Pipeline extension, of Shawn Hargreaves' article Generating Shaders From HLSL Fragments. I see StitchUp as sitting somewhere in the middle between hard-coded #if statements and node-based shader graphs. It's far more flexible than preprocessor defines or includes, but it's less powerful (and also simpler to work with as a developer) than a shader node graph.

For much more detail and background on HLSL fragments, go have a read of the article this project is based on. I have implemented everything in the article except for the last section on "Adaptive Fragments". I have also changed the syntax used to define fragments, as you'll see below.

StitchUp allows you to create effects by defining individual shader fragments. These fragments are stitched together to form the complete effect. Fragments can include a vertex shader part, a pixel shader part, or both. They can specify that they require effect parameters, vertex attributes, interpolated attributes, and textures. Fragments can communicate with each other by exporting and importing named values.

What does it look like?

Here's an example fragment:

fragment base_texture;

[textures]
texture2D color_map;

[vertexattributes]
float2 uv;

[interpolators]
float2 uv;

[ps 2_0]
__hlsl__
void main(INPUT input, inout OUTPUT output)
{
    output.color = tex2D(color_map, input.uv);
}
__hlsl__

Here's a simple vertex fragment. This fragment shows a bit more of the fragment definition syntax - it's pretty close to HLSL, including semantics.

fragment vertex_transform;

[parameters]
matrix wvp : WORLDVIEWPROJECTION;

[vertexattributes]
float3 position : POSITION;
float3 normal : NORMAL;

[interpolators]
float3 normal;

[vs 2_0]
__hlsl__
void main(INPUT input, inout OUTPUT output)
{
    output.position = mul(float4(input.position, 1), wvp);
    output(normal, input.normal);
}
__hlsl__

All variable names will get mangled in the final effect file, to ensure they are unique across all fragments. This happens behind the scenes, and you shouldn't need to worry about it (until something goes wrong and then you can log an issue).

How does it work?

StitchUp is implemented as an XNA 4.0 Content Pipeline extension. In your Content project, you have many .fragment files, and one or more .stitchedeffect files. These files are associated with StitchUp-specific importers and processors. At compile-time, StitchUp will take the fragments and compile them into an effect. From here on it's as though the effect came from the standard Effect Importer / Processor, and you can load it into your XNA game as you would any other effect.

StitchUp also comes with a custom Model Importer. This importer lets you specify which effect to use to render the model, and will build and load that effect instead of the standard BasicEffect.

Can I see a demo?

Sure! There's a demo project in the repository. This project is based on the Chase Camera sample from App Hub. The fragments here combine to form a simplified version of BasicEffect (please forgive the nonsensical logic of this sentence).

The fragments are included in the demo content project like this:

image

The file properties for a .fragment file look like this:

image

The file properties for a .stitchedeffect file look like this:

image

The file properties for one of the models file look like this:

image

At runtime, the effect parameter values are set in the DrawModel method in ChaseCameraGame.cs. Because the parameter names for each fragment will get mangled to ensure they are unique, it's slightly awkward to refer to parameters by name (this is an area I'd like to improve in the future). So I generally use semantics, if I know I'll only use a fragment once per effect.

Matrix wvp = transforms[mesh.ParentBone.Index] * world * camera.View * camera.Projection;
effect.Parameters.GetParameterBySemantic("WORLD").SetValue(transforms[mesh.ParentBone.Index] * world);
effect.Parameters.GetParameterBySemantic("WORLDVIEWPROJECTION").SetValue(wvp);  
effect.Parameters.GetParameterBySemantic("CAMERA_POSITION").SetValue(camera.Position);

effect.Parameters.GetParameterBySemantic("AMBIENT_LIGHT_DIFFUSE_COLOR").SetValue(new Vector4(0.1f, 0.1f, 0.1f, 1.0f));
effect.Parameters.GetParameterBySemantic("LIGHT_DIRECTION").SetValue(Vector3.Normalize(new Vector3(0.2f, -0.9f, 0.2f)));
effect.Parameters.GetParameterBySemantic("LIGHT_DIFFUSE_COLOR").SetValue(new Vector4(0.5f, 0.6f, 0.4f, 1.0f));

effect.Parameters.GetParameterBySemantic("DIFFUSE_TEXTURE_ENABLED").SetValue(true);
effect.Parameters.GetParameterBySemantic("SPECULAR_COLOR").SetValue(Vector3.One);
effect.Parameters.GetParameterBySemantic("SPECULAR_POWER").SetValue(16);
effect.Parameters.GetParameterBySemantic("SPECULAR_INTENSITY").SetValue(1);

Over to you...

If you do decide to give StitchUp a try, I'd appreciate any feedback you might have, positive or otherwise. Either leave a comment here or log an issue in the github repository. I'll gladly accept push requests for bug fixes.

The future?

Acknowledgements

If it wasn't obvious already, this whole project owes its existence to Shawn Hargreaves' article Generating Shaders From HLSL Fragments, which first appeared in ShaderX3 and is now available from his website. Shawn also helped with some technical questions I had.

9 comments

Gravatar Guy Sherman
Nov 15, 2010
20:11

Looks like a great idea. When I was at university, a friend of mine and I built a prototype of something similar for GLSL with this. It was pretty rough though. I will certainly give Stitch-Up a try.

Gravatar Michael B. McLaughlin
Nov 15, 2010
21:30

Very cool! I look forward to trying this when I get a chance.

Gravatar Tim Jones
Nov 16, 2010
04:38

Thanks for the feedback!

`Guy I think the great thing about the HLSL Fragments approach is its simplicity. It's really just string manipulation, so it's (relatively) easy to understand what's going on.

`Michael Thank you - I have removed all references to SM 1.1, from here and the code itself.

Gravatar Alejandro
Nov 17, 2010
11:10

Awesome man! I have read through your blog/github posts, Shawn's article and spent a few hours toying and trying to understand the concept behind it. Still struggling though.

Although, must say that this is insanely powerful stuff.

I started making a vertex "fragment" that just outputs position and a pixel fragment with a constant color: - In the [params] body, how can I set a starting default value? like: [params] float4 color : DIFFUSE_COLOR; // = { 1,1,1,1}; ?

Then made another vertex fragment that outputs only the interpolated uv texcoords and another pixel fragment that reads from a texture using those uv. - I wasn't able to decipher how do you use the texture coordinates. - The PositionNormalTexture fragment doesn't have an uv output anywhere. - Yet the BasicMaterial fragment doesn't import anything but has a [vertex] attribute and interpolator. Is there some magic wand going around behind the scenes? Like: "No one output-ed an uv and you need it, so I'll just give to you"?.

I was totally blown away by the compose-ability then available in the .stitchedeffect file. Although a simple example, it is now possible to generate a position only effect, or a textured effect without rewriting anything... I have written a fair amount of post processing effects before and more than half of the body of the file is repeated code: output uv, halfpixel offset, etc etc etc.

Some questions/thoughs: - Is it still possible to #include an .fxh with basic functions and methods? I tried but failed big time. Or the correct practice would be to think about "Helper Fragments" and you would be actually "importing the methods"? (for things like random, poisson things, etc). - Is it possible to create several techniques inside the same .stitched file? For example, to switch between different shadowing techniques, but inside the same .fx file to keep the same parameters (like camera, bias, etc) and only load one file in XNA (it would be a hassle to load as many files as there are "techniques"). - As an example, if I understand some of the overwhelming world of HLSL, the bool enable_texture of the BasicMaterial.fragment should be an static one, ideally passed as uniform bool parameter to the PixelShader() method or assembled under another technique. Am I right to say that the way it is now the final .fx file will evaluate both branches and discard the unselected one? - Is there a way to see the stitched .fx file (ouput somewhere in some folder). I only see it when there is an error (usually name mismatch). Visual studio keeps it in a temporal folder. - To understand a little bit more: the interface() { } body of Shawn's article has been completely replaced by the [] attributes right?

  • As a side note, to me a node based editor is just a few steps ahead, if some modular enough .fragments are pre-written, the nodes would only be calling (or string creating) those fragments. Like color channel of a node -> uv, it would be like saying import(uv, color = float4(uv, 1, 1)); I'm likely oversimplifying (still under the influence of the hype rush that this triggered).

Some problems found, likely because of Visual Studio: When assigning Visual C++ as the .fragment text editor (to get a little bit of syntax highlighting) commenting a line outside of the hlsl body (via ctrl+e, c) will always crash Visual Studio. (Is there a way to assign NShader as the syntax highlighter for the .fragment? that would be awesome)

All in all, thank you very much for the effort put on this work, I will definitely be looking for it. (Sorry for this long post).

Gravatar Tim Jones
Nov 18, 2010
15:21

@alejandro Great feedback, thank you! I'm glad you like it - I'd be interested to hear more if / when you try writing some fragments.

I have answered many of your points in a new post here.

I didn't answer everything there though, so here are my answers for your remaining questions:

  1. Probably I did the texture coordinates in a cack-handed way - probably PositionNormalTexture should define "uv" in the [vertex] and [interpolators] blocks. It could then export it for the BasicMaterial to use.

  2. Techniques - this is an interesting question. After reading your comment I've done some thinking about it, and I'm not sure what the ideal implementation would look like. Are you thinking of having "DrawShadow" and "UseShadow" techniques? Or "ShadowPSSM" and "ShadowVSM" techniques? Or both? The "Adaptive Fragments" section of Shawn's article is targeted at this sort of area, but I think it's meant for a different purpose.

  3. The enable_texture parameter is going to get written out to the final .fx file, and become available to set at runtime. I think this is static branching?

  4. Yes, the interface() { } block from Shawn's article is replaced by the [params], [textures], [vertex] and [interpolators] blocks.

  5. I think you might be on to something - judicious use of export / import could perhaps approximate node graphs. But because it's all in code it's going to be difficult to expose it in a graphical editor, which I think is the main benefit of those kinds of systems.

Gravatar Alejandro Martinez
Nov 19, 2010
06:51

You're welcome!. I read the new post before noticing this reply. I'll gladly offer any feedback as soon as they come!.

  1. Thats what I though mainly because of the naming suggestion of the PositionNormalTexture file. Although come to think of it, it could perfectly be also that the material asks for required vertex attributes i.e. If a single color material is assigned then the Normal and Texture coordinates will go wasted (likely to be wiped by the compiler and driver though when it sees that no one is using it, as long as it is never exported from the Vertex Shader to the PixelShader). So I find it perfectly valid to start with the minimum required (i.e. position) and the materials assigned start adding the required attributes (texture? -> UV, normalMapping? -> welcome the Tangents, etc) Perhaps it boils down to whatever flavor you like more.

  2. It could be both, as long as the DrawShadow and UseShadow would benefit from sharing the same parameters. Let's see, take PCF and simple ShadowMap are perhaps better examples, both will need the depthMapTexture, depthBias, LightViewProjection, etc. Some of those parameters could be set only once and never be touched again (i.e. a static light). I find this a perfect candidate for having different techniques, what happens if at runtime you which to change to a simple ShadowMap? Because the "shadows" are too far away? Rather than changing the effect file of the model and setting the parameter values of the previous effect, a change of technique would suffice. Even though internally techniques/passes are indeed different effects, on the developer side it implies loading 3 different effects and passing around previous params.

I read again the Adaptive Fragments: - The interpolator part (depending on pixel shader profile) perhaps is a little bit overkill and not needed nowadays (this is likely a shortsighted statement). - The PerPixelLighting part (ppl, !ppl) would work great as long as ppl is an exposed parameter variable. I don't fully understand it, $ambient, $color, $etc are interpolators grouped under $color that get created only if vs(!ppl) and ps(ppl)? If thats the case then it is a powerful selector. However in his implementation ppl is discarded at the compiled file.

I kinda think of this -> suppose MultiEffect.stitchedeffect: (plz excuse the lazyness about thinking a good .fragments separation) [technique] PCFShadow ..\VertexOutput.fragment ..\DepthRead.fragment ..\ProjectionMap.fragment ..\PCFTest.fragment ..\FinalColor.fragment

[technique] SimpleShadow ..\VertexOutput.fragment ..\DepthRead.fragment ..\ProjectionMap.fragment ..\ShadowTest.fragment ..\FinalColor.fragment

Does that make sense?

  1. Whether it is static branching or not I believe it depends on the graphics drivers and compiler versions (perhaps they are clever enough). However the more hints you give to the compiler the better. In this .pdf file http://developer.amd.com/media/gpu_assets/Dark_Secrets_of_shader_Dev-Mojo.pdf , they point out that "bool" versus "static bool" will yield different instruction counts because "bool" will evaluate both branches. Note that this paper is old enough and perhaps some points do not apply anymore. Although as I pointed out in your next post, [params] with static variables kinda bends some purposes, [headercode] sections are perhaps better suited for this (along methods, structs, array constants, #define PI 3.1415..), and will be handled automatically since it is declared inside hlsl body.

  2. I kinda lean towards brackets. I don't fully understand the interace() of Shawn's block, although that's likely because I never had the chance to play with it.

  3. How about the other way around?. (Paint me a crazy face) The graphical editor generates the code that the effect code generator will use to generate the effect file! a code generator layer over a code generator layer! What I see with this is that responsibilities are very separable along .fragment files and very composable. The same things nodes offer. Suppose a "TIME" node, an "UV Node" and a "ADDNODE". "ADDNODE" gets two inputs, add them, outputs the result. TIME would define a float and export, UV would define a float2 and export, ADDNODE would import both of them (here comes trouble, what should it export? the type with most components? i.e. a float2?), define an output and export. To be honest, I really don't know about the practical implementations of node based UI designs. So I'll just update my thinking in the matter. Node based UI seems way closer with what you have here, but you never know...

Again, thank you very much for this and the time taken to answer questions. Glad to hear that my feedback was appreciated.

Gravatar Tim Jones
Nov 20, 2010
19:01

@Alejandro

  1. Please vote on StitchUp's first issue :) https://github.com/roastedamoeba/stitchup/issues/issue/1

  2. Do you mean you prefer the interface() block from Shawn's article? If so, can I ask why? I did spend some time thinking about the syntax, and the result is intended to be familiar to HLSL programmers, but with as little overhead as possible. I'm certainly willing to listen to specific suggestions though :)

  3. Interesting ideas :) I'm still not sure how much value the current StitchUp implementation would add to that kind of system though. It would probably be easier to work with something built-for-purpose. (A little bit of history: I actually wrote what became StitchUp 2 years ago. It was mostly finished, and then I got distracted by shader node graph systems. I started to write one, but quickly got bogged down with the sheer scale of the project [although attempting to write the node definitions in C# and compile them to HLSL might have had something to do with that]. I abandoned both projects until a few weeks ago.)

Gravatar Alejandro Martinez
Nov 21, 2010
02:04

@Tim

  1. Up-Voted! (with some comments)

  2. Pardon the mind-talking-mess. Nope, I lean towards your brackets syntax, got it from the very beginning looking at your samples. [params] [vertexattributes] [headercode] etc point out very well (at least to me) what they mean. In some posts however some people might not like the hlsl body separation, I do like it, it implies that everything inside will be handled as any other effect files and the rules there are hlsl, methods, structs, inputs, etc, #defines, you name it. Perhaps it should just be [hlsl]? (Why hlsl ?, is that somekind of syntax practice <- I'm really noob in this area).

  3. I wouldn't really know!. You have made one point clear, thats one overwhelming and complex project to pull off (to me StitchUp is ambitious enough, let alone both!). To be honest I'm really not interested in using a node-based graph system, I'm just overly curious on how to tackle such a complex task. So I would love to hear more on the matter and how/if you come up with solutions. (Did you unburied both projects or just the StitchUp?).

Gravatar Tim Jones
Nov 21, 2010
16:21

`Alejandro

  1. Thank you.

  2. The two ideas behind hlsl is that (a) it needs to be something that doesn't ever appear in user code, and (b) it needs to appear both at the beginning and end of the code block. It would be awkward for me to use [hlsl] because that would conflict with block declarations (i.e. [vs]). The particular syntax I chose is borrowed from (C++'s method for embedding blocks of inline assembly (actually Microsoft use asm, but other C compilers use __asm, and I think it looks nicer.)

  3. I did do a little work on "SharpShader", as I called it, quite recently, but I don't think I'll do more on it in the foreseeable future. Not for lack of interest - just too many other other priorities. I have uploaded a pic of the editor I was working on - as you can see I didn't get far - I got further on the back-end code, but it's nowhere near distributable.

Make a comment

Sorry, commenting has been temporary disabled because of spam. If you have any questions, you can email me, and you can also find me on Twitter.