Adding Global shaders to UE4

Intro

On UE4, a global shader is one that doesn’t operate on a material or a mesh - for example, a full-screen quad, a post-processing feature, a compute shader, etc. There are times when the built-in global shaders do not provide enough functionality and a custom shader pass is required. Doing this is relatively simple, which will be covered in this post.

USF Files

UE4 reads USF (Unreal Shader Files) off the Engine/Shaders folder, so any new shaders need to supply their source files there. As of 4.17, shaders can also be read from a plugin (Plugin/Shaders).

Tip

I’d recommend enabling r.ShaderDevelopmentMode=1 in your ConsoleVariables.ini file for ease of development! Check out https://docs.unrealengine.com/latest/INT/Programming/Rendering/ShaderDevelopment/ for more information!

How-to

Let’s start by adding a new USF file in your Engine/Shaders folder; let’s call it MyTest.usf and add a simple pass-through Vertex Shader, and a Pixel Shader that returns a custom color:

// MyTest.usf

// Simple pass-through vertex shader
void MainVS(
in float4 InPosition : ATTRIBUTE0,
out float4 Output : SV_POSITION
)
{
Output = InPosition;
}

// Simple solid color pixel shader
float4 MyColor;
float4 MainPS() : SV_Target0
{
return MyColor;
}

Now, in order to get UE4 to pick up the shader and start compiling it, we need to declare a C++ class; let’s start with the Vertex Shader:

// This can go on a header or cpp file
class FMyTestVS : public FGlobalShader
{
DECLARE_EXPORTED_SHADER_TYPE(FMyTestVS, Global, /*MYMODULE_API*/);

FMyTestVS() { }
FMyTestVS(const ShaderMetaType::CompiledShaderInitializerType& Initializer)
: FGlobalShader(Initializer)
{
}

static bool ShouldCache(EShaderPlatform Platform)
{
return true;
}
};

You can see some required things here:

1. This is a subclass of FGlobalShader; as such it will end up in the Global Shader Map, (which means we don’t need a Material to find it).
2. Usage of the DECLARE_EXPORTED_SHADER_TYPE() macro; this will generate exports required for serialization of the shader type, etc; the third parameter is a type for external linkage for the code module where the shader module will live, if required (eg the C++ code doesn’t live in the Renderer module).
3. Two constructors, both the default and the serialization, are required.
4. The ShouldCache() function, needed to decide if this shader should be compiled under certain circumstances (eg we might not want to compile a compute shader on a non-compute shader capable RHI).

With the class declared, we can now register the Shader Type to UE4’s list:

// This needs to go on a cpp file

IMPLEMENT_SHADER_TYPE(, FMyTestVS, TEXT("MyTest"), TEXT("MainVS"), SF_Vertex);

This macro maps the type (FMyTestVS) to the USF file (MyTest), the shader entry point (MainVS) and the frequency/shader stage (SF_Vertex); it also causes the shader to be added to the compilation list, as long as its ShouldCache() method returns true.

Note: Whatever module you end adding your FGlobalShader has to be loaded before the actual engine starts, or you will get an assert like “"Shader type was loaded after engine init, use ELoadingPhase::PostConfigInit on your module to cause it to load earlier.”; i.e. we current do not allow a dynamic module loaded in the middle of the game or editor to add its own shader type.

Let’s now declare the more interesting Pixel Shader:

class FMyTestPS : public FGlobalShader
{
DECLARE_EXPORTED_SHADER_TYPE(FMyTestPS, Global, /*MYMODULE_API*/);

FShaderParameter MyColorParameter;

FMyTestPS() { }
FMyTestPS(const ShaderMetaType::CompiledShaderInitializerType& Initializer)
: FGlobalShader(Initializer)
{
MyColorParameter.Bind(Initializer.ParameterMap, TEXT("MyColor"), SPF_Mandatory);
}

static void ModifyCompilationEnvironment(EShaderPlatform Platform, FShaderCompilerEnvironment& OutEnvironment)
{
FGlobalShader::ModifyCompilationEnvironment(Platform, OutEnvironment);
// Add your own defines for the shader code
OutEnvironment.SetDefine(TEXT("MY_DEFINE"), 1);
}

static bool ShouldCache(EShaderPlatform Platform)
{
// Could skip compiling for Platform == SP_METAL for example
return true;
}

// FShader interface.
virtual bool Serialize(FArchive& Ar) override
{
bool bShaderHasOutdatedParameters = FGlobalShader::Serialize(Ar);
Ar << MyColorParameter;
return bShaderHasOutdatedParameters;
}

void SetColor(FRHICommandList& RHICmdList, const FLinearColor& Color)
{
SetShaderValue(RHICmdList, GetPixelShader(), MyColorParameter, Color);
}
};
// Same source file as before, different entry point
IMPLEMENT_SHADER_TYPE(, FMyTestPS, TEXT("MyTest"), TEXT("MainPS"), SF_Pixel);

In this class we are now exposing the shader parameter MyColor from the USF file:

1. The first thing to note is the FShaderParameter MyColorParameter member added to the class: this will hold information for the runtime to be able to find the bindings so the value of the parameter can be set at runtime.
2. In the serialization constructor we Bind() the parameter to the ParameterMap by name, this has to match the USF name.
3. The new ModifyCompilationEnvironment() function, this is used in the cases where we might have the same C++ class define different behaviors and be able to set up #define values in the shader.
4. The Serialize() method, required as this is where the compile/cook time information from the shader’s binding (matched during the serialization constructor) gets load and stored at runtime.
5. Finally we have a custom SetColor() method, which shows how to set the MyColor parameter at runtime with a specified value.

Let’s now write a simple function to draw a fullscreen quad using these shader types:

void RenderMyTest(FRHICommandList& RHICmdList, ERHIFeatureLevel::Type FeatureLevel, const FLinearColor& Color)
{
// Get the collection of Global Shaders
auto ShaderMap = GetGlobalShaderMap(FeatureLevel);
// Get the actual shader instances off the ShaderMap
TShaderMapRef MyVS(ShaderMap);
TShaderMapRef MyPS(ShaderMap);
// Declare a bound shader state using those shaders and apply it to the command list
static FGlobalBoundShaderState MyTestBoundShaderState;
SetGlobalBoundShaderState(RHICmdList, FeatureLevel, MyTestBoundShaderState, GetVertexDeclarationFVector4(), *MyVS, *MyPS);
// Call our function to set up parameters
MyPS->SetColor(RHICmdList, Color);

// Setup the GPU in prep for drawing a solid quad
RHICmdList.SetRasterizerState(TStaticRasterizerState<FM_Solid, CM_None>::GetRHI());
RHICmdList.SetBlendState(TStaticBlendState<>::GetRHI());
RHICmdList.SetDepthStencilState(TStaticDepthStencilState<false, CF_Always>::GetRHI(), 0);

// Setup the vertices
FVector4 Vertices[4];
Vertices[0].Set(-1.0f, 1.0f, 0, 1.0f);
Vertices[1].Set(1.0f, 1.0f, 0, 1.0f);
Vertices[2].Set(-1.0f, -1.0f, 0, 1.0f);
Vertices[3].Set(1.0f, -1.0f, 0, 1.0f);
// Draw the quad
DrawPrimitiveUP(RHICmdList, PT_TriangleStrip, 2, Vertices, sizeof(Vertices[0]));
}

If you want to actually test this in your codebase, you can try something like this; I declared a console variable so it can be toggled at runtime:

static TAutoConsoleVariable<int32> CVarMyTest(
TEXT("r.MyTest"),
0,
TEXT("Test My Global Shader, set it to 0 to disable, or to 1, 2 or 3 for fun!"),
ECVF_RenderThreadSafe
);

void FDeferredShadingSceneRenderer::RenderFinish(FRHICommandListImmediate& RHICmdList)
{
[...]
// ***
// Inserted code, just before finishing rendering, so we can overwrite the screen’s contents!
int32 MyTestValue = CVarMyTest.GetValueOnAnyThread();
if (MyTestValue != 0)
{
FLinearColor Color(MyTestValue == 1, MyTestValue == 2, MyTestValue == 3, 1);
RenderMyTest(RHICmdList, FeatureLevel, Color);
}
// End Inserted code
// ***
FSceneRenderer::RenderFinish(RHICmdList);
[...]
}

At this point you should be able to run your project, then pull up a console using tilde (~) and typing r.MyTest 1, then r.MyTest 2 and/or r.MyTest 3 to change colors. Use r.MyTest 0 to disable the pass.

Debugging the generated source

Take a look at the blog post https://www.unrealengine.com/blog/debugging-the-shader-compiling-process if you want to be able to debug the compilation of your USF file and/or see the processed file.

Bonus!

You can modify the USF file while an uncooked game/editor is running, and then press Ctrl+Shift+. (or type recompileshaders changed in the console) to pick up and rebuild your shaders, for quick iteration!

Enjoy!