Adding Global shaders to UE4 v2.0

(Updated post for 4.18 and above!)

Intro

In Unreal Engine 4, global shaders are shaders that can be used from the C++ side to render post-processing effects, dispatch compute shaders, clear the screen, etc. (i.e., shaders that don’t operate on a material or a mesh). On occasion, more advanced functionality is necessary to achieve a desired look, and a custom shader pass is required. Doing this is relatively simple, as I will explain here.

USF Files

UE4 reads .usf (Unreal Shader Files) off the Engine/Shaders/Private folder. Any new shaders need their source files placed here, or if working on a plugin, PluginName/Shaders/Private).

Tip

I’d recommend enabling r.ShaderDevelopmentMode=1 in your ConsoleVariables.ini file for ease of development! Check out the documentation for more information!

How-to

Let’s start by adding a new .usf file in your Engine/Shaders/Private folder; let’s call it MyTest.usf and add a simple pass-through Vertex Shader, and a Pixel Shader that returns a custom color:

// MyTest.usf

// Simple pass-through vertex shader
void MainVS(
in float4 InPosition : ATTRIBUTE0,
out float4 Output : SV_POSITION
)
{
Output = InPosition;
}

// Simple solid color pixel shader
float4 MyColor;
float4 MainPS() : SV_Target0
{
return MyColor;
}

Now, in order to get UE4 to pick up the shader and start compiling it, we need to declare a C++ class; let’s start with the Vertex Shader:

// This can go on a header or cpp file
class FMyTestVS : public FGlobalShader
{
DECLARE_EXPORTED_SHADER_TYPE(FMyTestVS, Global, /*MYMODULE_API*/);

FMyTestVS() { }
FMyTestVS(const ShaderMetaType::CompiledShaderInitializerType& Initializer)
: FGlobalShader(Initializer)
{
}

static bool ShouldCache(EShaderPlatform Platform)
{
return true;
}


static bool ShouldCompilePermutation(const FGlobalShaderPermutationParameters& Parameters)
{
// Useful when adding a permutation of a particular shader
return true;
}
};

There are a few requirements here:

1. This is a subclass of FGlobalShader. As such, it will end up in the Global Shader Map, (which means we don’t need a material to find it).
2. Usage of the DECLARE_EXPORTED_SHADER_TYPE() macro will generate the exports required for serialization of the shader type, etc. The third parameter is a type for external linkage for the code module where the shader module will live, if required (e.g. the C++ code doesn’t live in the Renderer module).
3. Two constructors, both the default and the serialization.
4. The ShouldCache() function, needed to decide if this shader should be compiled under certain circumstances (e.g. we might not want to compile a compute shader on a non-compute shader capable RHI).
5. The ShouldCompilePermutation() function, needed when a permutation of a global shader is required. This is a slightly more advanced topic outside the scope of this post.

With the class declared, we can now register the Shader Type to UE4’s list:

// This needs to go on a cpp file

IMPLEMENT_SHADER_TYPE(, FMyTestVS, TEXT("/Engine/Private/MyTest.usf"), TEXT("MainVS"), SF_Vertex);

This macro maps the type (FMyTestVS) to the .usf file (/Engine/Private/MyTest.usf), the shader entry point (MainVS) and the frequency/shader stage (SF_Vertex). It also causes the shader to be added to the compilation list as long as its ShouldCache() and ShouldCompilePermutation() methods both returns true.

Note: Whichever module you add your FGlobalShader to has to be loaded before the actual engine starts, or you will get an assert like, “"Shader type was loaded after engine init, use ELoadingPhase::PostConfigInit on your module to cause it to load earlier.”. We current do not allow a dynamic module that is loaded after a game or editor to add its own shader type.

Let’s now declare the more interesting Pixel Shader:

class FMyTestPS : public FGlobalShader
{
DECLARE_EXPORTED_SHADER_TYPE(FMyTestPS, Global, /*MYMODULE_API*/);

FShaderParameter MyColorParameter;

FMyTestPS() { }
FMyTestPS(const ShaderMetaType::CompiledShaderInitializerType& Initializer)
: FGlobalShader(Initializer)
{
MyColorParameter.Bind(Initializer.ParameterMap, TEXT("MyColor"), SPF_Mandatory);
}

static void ModifyCompilationEnvironment(const FGlobalShaderPermutationParameters& Parameters, FShaderCompilerEnvironment& OutEnvironment)
{
FGlobalShader::ModifyCompilationEnvironment(Parameters, OutEnvironment);
// Add your own defines for the shader code
OutEnvironment.SetDefine(TEXT("MY_DEFINE"), 1);
}

static bool ShouldCache(EShaderPlatform Platform)
{
// Could skip compiling for Platform == SP_METAL for example
return true;
}

static bool ShouldCompilePermutation(const FGlobalShaderPermutationParameters& Parameters)
{
// Useful when adding a permutation of a particular shader
return true;
}

// FShader interface.
virtual bool Serialize(FArchive& Ar) override
{
bool bShaderHasOutdatedParameters = FGlobalShader::Serialize(Ar);
Ar << MyColorParameter;
return bShaderHasOutdatedParameters;
}

void SetColor(FRHICommandList& RHICmdList, const FLinearColor& Color)
{
SetShaderValue(RHICmdList, GetPixelShader(), MyColorParameter, Color);
}
};
// Same source file as before, different entry point

IMPLEMENT_SHADER_TYPE(, FMyTestPS, TEXT("/Engine/Private/MyTest.usf"), TEXT("MainPS"), SF_Pixel);

In this class we are now exposing the shader parameter MyColor from the .usf file:

1. The FShaderParameter MyColorParameter member is added to the class, which will hold information for the runtime to be able to find the bindings, allowing the value of the parameter to be set at runtime.
2. In the serialization constructor we Bind() the parameter to the ParameterMap by name, this has to match the .usf file's name.
3. The new ModifyCompilationEnvironment() function is used when the same C++ class defines different behaviors and be able to set up #define values in the shader.
4. The Serialize() method is required. This is where the compile/cook time information from the shader’s binding (matched during the serialization constructor) gets loaded and stored at runtime.
5. Finally, we have a custom SetColor() method, which shows how to set the MyColor parameter at runtime with a specified value.

Let’s now write a simple function to draw a fullscreen quad using these shader types:

void RenderMyTest(FRHICommandList& RHICmdList, ERHIFeatureLevel::Type FeatureLevel, const FLinearColor& Color)
{
 // Get the collection of Global Shaders
 auto ShaderMap = GetGlobalShaderMap(FeatureLevel);
 // Get the actual shader instances off the ShaderMap
 TShaderMapRef<FMyTestVS> MyVS(ShaderMap);
 TShaderMapRef<FMyTestPS> MyPS(ShaderMap);
 
 // Declare a pipeline state object that holds all the rendering state
 FGraphicsPipelineStateInitializer PSOInitializer;

 PSOInitializer.PrimitiveType = PT_TriangleStrip;
PSOInitializer.BoundShaderState.VertexDeclarationRHI = GetVertexDeclarationFVector4();
 PSOInitializer.BoundShaderState.VertexShaderRHI = MyVS->GetVertexShader();
 PSOInitializer.BoundShaderState.PixelShaderRHI = MyPS->GetPixelShader();
 PSOInitializer.RasterizerState = TStaticRasterizerState<FM_Solid, CM_None>::GetRHI();
 PSOInitializer.BlendState = TStaticBlendState<>::GetRHI();
 PSOInitializer.DepthStencilState = TStaticDepthStencilState<false, CF_Always>::GetRHI();
 
 // Apply it
 SetGraphicsPipelineState(RHICmdList, PSOInitializer);
 
 // Call our function to set up parameters. This has to happen AFTER the PSO has been applied!
 MyPS->SetColor(RHICmdList, Color);
 
 // Setup the vertices
 FVector4 Vertices[4];
 Vertices[0].Set(-1.0f, 1.0f, 0, 1.0f);
 Vertices[1].Set(1.0f, 1.0f, 0, 1.0f);
 Vertices[2].Set(-1.0f, -1.0f, 0, 1.0f);
 Vertices[3].Set(1.0f, -1.0f, 0, 1.0f);
 
 // Draw the quad
 DrawPrimitiveUP(RHICmdList, PT_TriangleStrip, 2, Vertices, sizeof(Vertices[0]));
}

If you want to actually test this in your codebase, you can try something like this; I declared a console variable so it can be toggled at runtime:

static TAutoConsoleVariable<int32> CVarMyTest(
TEXT("r.MyTest"),
0,
TEXT("Test My Global Shader, set it to 0 to disable, or to 1, 2 or 3 for fun!"),
ECVF_RenderThreadSafe
);

void FDeferredShadingSceneRenderer::RenderFinish(FRHICommandListImmediate& RHICmdList)
{
[...]
// ***
// Inserted code, just before finishing rendering, so we can overwrite the screen’s contents!
int32 MyTestValue = CVarMyTest.GetValueOnAnyThread();
if (MyTestValue != 0)
{
FLinearColor Color(MyTestValue == 1, MyTestValue == 2, MyTestValue == 3, 1);
RenderMyTest(RHICmdList, FeatureLevel, Color);
}
// End Inserted code
// ***
FSceneRenderer::RenderFinish(RHICmdList);
[...]
}

At this point you should be able to test out your new global shader! Run your project, then pull up a console using tilde (~) and type r.MyTest 1, then r.MyTest 2 and/or r.MyTest 3 to change colors. Use r.MyTest 0 to disable the pass.

Debugging the generated source

Take a look at the blog post https://www.unrealengine.com/blog/debugging-the-shader-compiling-process if you want to be able to debug the compilation of your .usf file and/or see the processed file.

Bonus!

You can modify the USF file while an uncooked game/editor is running, and then press Ctrl+Shift+. (period) or type recompileshaders changed in the console to pick up and rebuild your shaders for quick iteration!

Enjoy!