C# on Kampter

URP屏幕空间Sobel边缘检测

Mon, 10 Apr 2023 20:25:54 +0800

什么是边缘检测

根据Wiki说明，边缘检测是一种用于识别图像变化的明显的区域的技术。因此这种基础技术被广泛应用与图像处理和计算机视觉中用以查找出物体的边缘。边缘检测算法的目的是发掘图像中不连续火图像亮度急剧变化的点。而常见的几种算法用于便边缘检测技术包括Sobel， Prewitt, Roberts 和 Canny。其中Canny 边缘检测器是罪被广泛应用的算法，它使用多阶算法结合，于1986年被John F. Canny最先开发出来并且随后广泛使用。

为什么边缘检测重要

边缘检测是图像处理的重要组成部分，因为它有助于计算识别和将图像分离成不同的部分。它用于图像处理、计算机视觉和机器视觉等领域的图像分割和数据提取。检测图像亮度的急剧变化的目的是捕捉重要事件和属性的变化。边缘是与图像相关的最重要的特征之一。我们通过图像的边缘了解图像的底层结构。因此，计算机视觉处理管道广泛使用边缘检测。

Sobel 边缘检测

简单地说，我们对每个片段的四个边邻居进行采样，并对采样属性的差异求和。小的差异表明表面是一致的，我们很可能不在边缘上，而大的差异表明采样属性的突然变化，我们很可能在边缘像素上。

我们将用来计算 Sobel 差分的函数可以表示为：

s为纹理样本，是原像素点的位置
- 中、上、下、左、右
multiplier 是一个标量输入控制参数
线性影响值 bias 是一个标量输入控制参数，用于减少最终结果中的噪声伪影

URP代码实现

URP中需要在custompass中开启Scene normal的支持，方法参考官方实现的SSAO效果。

1

ConfigureInput(ScriptableRenderPassInput.Normal);

在Shader中便可以对CameraNormalsTexture采样

1
2
3
4
5


#include "Packages/com.unity.render-pipelines.universal/ShaderLibrary/DeclareNormalsTexture.hlsl"
float4 SampleNormal(float2 uv)
{
 return SAMPLE_TEXTURE2D(_CameraNormalsTexture, sampler_CameraNormalsTexture, uv);
}

对法线和深度采样的方式几乎完全相同，采样法线和深度两种方式

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


float4 SobelSampleNormal(float2 uv, float3 offset)
{
 float4 pixelCenter = SampleNormal(uv);
 float4 pixelLeft = SampleNormal(uv - offset.xz);
 float4 pixelRight = SampleNormal(uv + offset.xz);
 float4 pixelUp = SampleNormal(uv + offset.zy);
 float4 pixelDown = SampleNormal(uv - offset.zy);

 return abs(pixelLeft - pixelCenter) +
 abs(pixelRight - pixelCenter) +
 abs(pixelUp - pixelCenter) +
 abs(pixelDown - pixelCenter);
}

只使用邻近的四个的原因是为了减少不必要的边缘检测

完成 SobelDepth 和SobelNormal的计算后，将两种采样结果合并

1

float sobelOutline = saturate(max(sobelDepth, sobelNormal));

此时还需要处理一些缺陷

物体与物体连接处没有边缘
视线与物体夹角较小，接近平视时因为像素的问题会导致一部分物体表面都会出现描边颜色而不是很细的线。

Reference

URP屏幕空间环境光遮蔽

Wed, 05 Apr 2023 14:35:17 +0800

Screen Space Ambient Occlusion: Enhancing Realism in Video Games

Introduction

Video games have come a long way since their inception, and the quality of graphics has improved significantly. One aspect that helps in achieving realistic visuals in video games is Screen Space Ambient Occlusion (SSAO). SSAO is a technique that enhances the depth and realism of the game by simulating the way light behaves in real life. In this blog post, we will discuss what SSAO is, how it works, its benefits in video games, and how it can be implemented.

What is Screen Space Ambient Occlusion?

Screen Space Ambient Occlusion is a technique used in video games to simulate the way light interacts with objects in real life. In video games, SSAO is used to enhance the depth and realism of the game by simulating the shadows that are created when light is blocked by objects. SSAO calculates the amount of ambient light that reaches a pixel on the screen by taking into account the occlusion of that pixel by surrounding objects.

How Does Screen Space Ambient Occlusion Work?

SSAO works by analyzing the depth and normal information stored in the game’s frame buffer. The technique calculates the amount of ambient light that should be reaching a pixel by analyzing the surrounding pixels. SSAO simulates the way light behaves in real life by taking into account the occlusion of a pixel by surrounding objects. By adding shadows to the game, SSAO enhances the depth and realism of the game’s graphics.

Benefits of Screen Space Ambient Occlusion

The use of SSAO in video games has several benefits. Firstly, it enhances the realism of the graphics by simulating the way light behaves in real life. This makes the game world feel more immersive, and the player feels more connected to the game. Secondly, SSAO adds depth to the game by simulating the way shadows are created when light is blocked by objects. This adds a sense of realism to the game world, making it feel more natural. Lastly, SSAO is a relatively inexpensive technique that can be used in real-time applications, making it a popular choice among developers.

Implementing SSAO in Video Games

Implementing SSAO in video games can be done using different techniques. Here are some of the most common methods used by game developers:

Screen Space Ambient Occlusion

Screen Space Ambient Occlusion is the most common method used by game developers. This technique uses the depth and normal information stored in the game’s frame buffer to calculate the amount of ambient light that should be reaching a pixel. SSAO is then applied to the pixel to simulate the way light behaves in real life.

Horizon-Based Ambient Occlusion

Horizon-Based Ambient Occlusion is a technique that takes into account the horizon of the game world. This technique calculates the amount of ambient light that should be reaching a pixel by analyzing the distance between the pixel and the horizon. This method is more computationally expensive than SSAO but provides more accurate results.

Multi-Scale Ambient Occlusion

Multi-Scale Ambient Occlusion is a technique that uses different scales to calculate the amount of ambient light that should be reaching a pixel. This method takes into account the distance between the pixel and the objects in the game world to provide more accurate results.

When implementing SSAO, it is essential to consider the limitations of the technique. SSAO may not be effective in simulating the way light behaves in open spaces or in situations where the light source is very bright. It can also create artifacts in the game world, such as halos around objects. To overcome these limitations, game developers can use other techniques such as Horizon-Based Ambient Occlusion or Multi-Scale Ambient Occlusion.

Limitations of SSAO

While SSAO is a useful technique for enhancing the realism of video games, it does have its limitations. SSAO is only effective in simulating the way light behaves in real life in certain situations. For example, SSAO is not effective in simulating the way light behaves in open spaces or in situations where the light source is very bright. In addition, SSAO can sometimes create artifacts in the game world, such as halos around objects.

Future of SSAO

Despite its limitations, SSAO is becoming an essential technique for game developers looking to create realistic graphics in their video games. As technology continues to improve, it is likely that SSAO will become more effective in simulating the way light behaves in real life. With the increasing demand for realistic graphics in video games, game developers will continue to rely on SSAO and other techniques to enhance the depth and realism of their games.

Conclusion

Screen Space Ambient Occlusion is a technique that enhances the realism of video games by simulating the way light behaves in real life. SSAO works by analyzing the depth and normal information stored in the game’s frame buffer and calculating the amount of ambient light that should be reaching a pixel. The use of SSAO in video games has several benefits, including enhancing the realism and depth of the game world. While SSAO does have its limitations, it is becoming an essential technique for game developers looking to create realistic graphics in their video games. With its increasing popularity, it is important for game developers to continue to explore and experiment with SSAO to create even more realistic and immersive gaming experiences for players.

闲言闲语

以上内容使用notion AI创作完成，刚好近期AIGC火的一塌糊涂，诸如Midjourney, Stable diffusion, 还有ChatGPT等等，以及很多大厂在GDC上面分享了由AI帮助生产的技术。我们也来凑个热闹好了。其实不难看出，AI在讲述一个完全未知效果的时候还挺好用的。一板一眼，条条框框，逻辑十分正确，至少比我强多了。而SSAO到底是什么，以及怎么实现，其实在上述内容已经讲的很清楚了。这里用中文列出来：

获取屏幕空间深度
根据相机的世界位置，深度，就可以重建整个世界空间
URP下面没有法线图，构建法线的内容参考

https://wickedengine.net/2019/09/22/improved-normal-reconstruction-from-depth/

在法线的方向的半球内随机撒点，点尽可能靠近像素中心位置。
构建像素点至随机点的向量，并转化至View 空间
对比depth 和构建的向量的深度，计算这个向量是否被遮挡。
加权平均半球内的全部随机点结果，取值缩放至 0 - 1范围内，即遮蔽效果的强度。

Reference

URP屏幕空间雾效

Tue, 04 Apr 2023 14:25:03 +0800

Unity URP Depth Fog

Introduction

Unity is a powerful game engine that has been used by game developers around the world to create high-quality games with various tools and features. One of these features is the Universal Rendering Pipeline (URP), which provides a lightweight rendering pipeline that is optimized for mobile and low-end hardware devices. In this blog post, we will discuss how to implement depth fog in Unity URP, which is a visual effect that creates a sense of depth and distance in a scene.

What is Depth Fog?

Depth fog is an atmospheric effect that makes distant objects appear hazy or blurry. This effect can be used to create a sense of scale in a scene and to make objects appear more realistic. Depth fog is a useful tool for game developers who want to create immersive environments that feel more natural.

How to Implement Depth Fog in Unity URP

Unity URP provides a built-in Depth Texture that can be used to implement depth fog. The Depth Texture is a grayscale texture that represents the distance between the camera and the objects in the scene. To implement depth fog in Unity URP, follow these steps:

Create a new material and set its Shader to the URP/Lit shader.
Enable the Depth Texture option in the material.
Adjust the Depth Fade Start and Depth Fade Distance properties to control the distance at which the fog starts and how far it extends.
Set the fog color in the material.

Once you have created the material with the depth texture enabled, apply it to the objects in your scene that you want to be affected by depth fog. The objects closer to the camera will be less affected by the fog, while the objects further away will appear more hazy.

Benefits of Using Depth Fog

Depth fog is a powerful tool for game developers who want to create immersive environments. By using depth fog, developers can create a sense of depth and distance in a scene, which can make the scene feel more natural and realistic. Depth fog can also be used to create a sense of scale in a scene, which is important for games that involve large environments, such as open-world games.

Conclusion

Depth fog is a useful visual effect that can be used to create a sense of depth and distance in a scene. Unity URP provides a built-in Depth Texture that can be used to implement depth fog. By following the steps outlined in this blog post, you can easily add depth fog to your Unity URP project. By using depth fog, game developers can create immersive environments that feel more natural and realistic, making for a more engaging gaming experience.

技术总结

还是用 Notion AI帮忙生成一下对于Depth Fog的介绍。

采样Depth
重建世界空间
三种雾效计算方式

$$ f\;=\;\frac{d_{max}\;-\;\left|z\right|}{d_{max}\;-\;d_{ming}} $$

$d_{max}$和$d_{min}$分别表示受雾影响的最小距离和最大距离。

$$ f\;=\;e^{-d\cdot\left|z\right|} $$

d是控制雾的浓度的参数。

$$ f\;=\;e^{-\left(d\;\cdot\;\left|z\right|\right)^2} $$

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


half4 frag(Varyings input): SV_Target
{
 // 采样depth 并且重建世界空间
 half2 uv = input.positionCS.xy / _ScaledScreenParams.xy;
 float rawDepth = SampleSceneDepth(uv);
 float3 positionWS = ComputeWorldSpacePosition(uv, rawDepth, UNITY_MATRIX_I_VP);

 // 高度雾效系数 = （终止高度 - 当前像素高度）/（终止高度 - 起始高度）
 float fogDensity = (_FogEnd - positionWS.y) / (_FogEnd - _FogStart);
 // 计算浓度 saturate：截取到0-1
 fogDensity = saturate(fogDensity * _FogDensity);

 // 原始颜色
 half4 finalColor = SAMPLE_TEXTURE2D(_MainTex, sampler_MainTex, uv);
 // 插值原始颜色，与雾效颜色，使用雾效系数作为参数
 finalColor.rgb = lerp(finalColor.rgb, _FogColor.rgb, fogDensity);

 return finalColor;
}

Reference

URP 14 RenderFeature 使用方式

Mon, 20 Mar 2023 10:47:50 +0000

URP 14

URP 14 对应Unity 2022.2对于2021lts版本的URP 12中后处理的API有些许改变，需要参考官方文档的改变。而其中最大的改变来自于RTHandle的引入。

以一个DesaturateFeature为例。在DesaturateFeature.cs中改变比较大的是加入一个SetupRenderPasses来帮助创建Materail和传递参数，而不是传统的写法在Create()中。

1
2
3
4
5
6


 public override void SetupRenderPasses(ScriptableRenderer renderer, in RenderingData renderingData)
 {
 Material material = CoreUtils.CreateEngineMaterial(shader);
 desaturateRenderPass.ConfigureInput(ScriptableRenderPassInput.Color);
 desaturateRenderPass.Setup(material, "DesaturatedRenderPass",renderingData);
 }

然后写内部的DesaturateRenderPass 类。

变化一是RenderTargetHandle.Init("_CustomPassHandle")方式的改变为

1
2
3
4


var colorCopyDescriptor = renderingData.cameraData.cameraTargetDescriptor;
 colorCopyDescriptor.depthBufferBits = (int) DepthBits.None;
 RenderingUtils.ReAllocateIfNeeded(ref m_CopiedColor, colorCopyDescriptor, name: "_FullscreenPassColorCopy");

变化二：command buffer的GetTemporaryRT和ReleaseTemporaryRT被弃用。
官方推荐使用Blitter.BlitCameraTexture方法把原始的RT添加我们自定义的material然后再传递回去。代码实现如下

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


CommandBuffer cmd = CommandBufferPool.Get();
var cameraData = renderingData.cameraData;
using (new ProfilingScope(cmd, profilingSampler))
{
 var source =cameraData.renderer.cameraColorTargetHandle;
 passMaterial.SetTexture(m_BlitTextureShaderID, copiedColor);
 Blitter.BlitCameraTexture(cmd, source, copiedColor, passMaterial, 0);
 Blitter.BlitCameraTexture(cmd, copiedColor, source);
 context.ExecuteCommandBuffer(cmd);
 cmd.Clear();
}

参考官方的案例，完整写一个ColorTint的实例

Shader的部分可以十分简化，Vert在Blit.hlsl中官方已经帮忙处理好，这里只需要写frag的逻辑部分，太香了。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


Shader "ColorBlit"
{
 SubShader
 {
 Tags { "RenderType"="Opaque" "RenderPipeline" = "UniversalPipeline"}
 LOD 100
 ZWrite Off Cull Off
 Pass
 {
 Name "ColorTintPass"
 HLSLPROGRAM
 #include "Packages/com.unity.render-pipelines.core/Runtime/Utilities/BlitColorAndDepth.hlsl"
 #pragma vertex Vert
 #pragma fragment frag

 float4 _ColorTint;

 half4 frag (Varyings input) : SV_Target
 {
 half4 color = FragColorAndDepth(input).color;
 color *= _ColorTint;
 return color;
 }
 ENDHLSL
 }
 }
}

完整的ColorTintRenderFeature部分。思路如下，用一个colorTintSettings做UI接住全部的设置内容。重写一个SetupRenderPasses来传递以前在Create()中传递的内容，创建就好好创建，设置参数就好好设置参数，嗯恒河里。RenderFeature class基本就这样没什么可以说的，与urp 12就这么点区别。哦对，在传递相机的部分，用renderer.cameraColorTargetHandle而不是cameraColorTarget。换句话说，unity 换了RThandle这个API，跟着换就行，虽然老的也能用，但是Unity官方论坛的讨论结果是未来RThandle会成为主流，你现在学会对未来没坏处。

这里偷懒不想创建Volume，也不想多创建一个c#脚本来写colorTintPass，直接用一个intenal class来写吧。首先最大的区别就是用RThandle替代以前的API。重写OnCameraSetup()，调用ConfigureTarget()来设置当前Target，也就是刚刚传递过来的cameraColorTargetHandle。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94


using UnityEngine;
using UnityEngine.Rendering;
using UnityEngine.Rendering.Universal;

public class ColorTintRenderFeature : ScriptableRendererFeature
{
 [System.Serializable]
 public class ColorTintSettings
 {
 public Shader colorTintShader;
 public RenderPassEvent passEvent = RenderPassEvent.BeforeRenderingPostProcessing;
 public ScriptableRenderPassInput requirements = ScriptableRenderPassInput.Color;
 [Header("ColorTint Settings")] [Space(10)]
 public Color colorTint = Color.red;
 }

 public ColorTintSettings colorTintSettings = new ColorTintSettings();
 private ColorTintPass colorTintPass;

 public override void SetupRenderPasses(ScriptableRenderer renderer, in RenderingData renderingData)
 {
 colorTintPass.ConfigureInput(colorTintSettings.requirements);
 colorTintPass.Setup(renderer.cameraColorTargetHandle, colorTintSettings);
 }
 public override void Create()
 {
 colorTintPass = new ColorTintPass();
 colorTintPass.renderPassEvent = colorTintSettings.passEvent;
 }

 public override void AddRenderPasses(ScriptableRenderer renderer, ref RenderingData renderingData)
 {
 renderer.EnqueuePass(colorTintPass);
 }
 protected override void Dispose(bool disposing)
 {
 colorTintPass.Dispose();
 }

 public class ColorTintPass : ScriptableRenderPass
 {
 private ProfilingSampler profilingSampler = new ProfilingSampler(nameof(ColorTintPass));
 private ColorTintSettings colorTintSettings;
 private Material colorTintMaterial;
 private static readonly int ColorTintID = Shader.PropertyToID("_ColorTint");
 private static readonly int BlitTextureID = Shader.PropertyToID("_BlitTexture");
 private RTHandle source;
 private RTHandle copiedColor;

 public void Setup(RTHandle colorHandle, ColorTintSettings colorTintSettings)
 {
 this.colorTintSettings = colorTintSettings;
 Shader shader = colorTintSettings.colorTintShader;
 if (shader == null)
 return;
 colorTintMaterial = CoreUtils.CreateEngineMaterial(shader);
 source = colorHandle;
 }

 public void Dispose()
 {
 source.Release();
 CoreUtils.Destroy(colorTintMaterial);
 }

 public override void OnCameraSetup(CommandBuffer cmd, ref RenderingData renderingData)
 {
 ConfigureTarget(source);
 var colorCopyDescriptor = renderingData.cameraData.cameraTargetDescriptor;
 colorCopyDescriptor.depthBufferBits = (int) DepthBits.None;
 RenderingUtils.ReAllocateIfNeeded(ref copiedColor, colorCopyDescriptor, name: "_FullscreenPassColorCopy");
 }

 public override void Execute(ScriptableRenderContext context, ref RenderingData renderingData)
 {
 if (colorTintMaterial == null)
 return;

 CommandBuffer cmd = CommandBufferPool.Get();
 var cameraData = renderingData.cameraData;

 using (new ProfilingScope(cmd, profilingSampler))
 {
 colorTintMaterial.SetColor(ColorTintID, colorTintSettings.colorTint);
 Blitter.BlitCameraTexture(cmd, source, copiedColor);
 colorTintMaterial.SetTexture(BlitTextureID, copiedColor);
 Blitter.BlitCameraTexture(cmd, copiedColor, source, colorTintMaterial, 0);
 }
 context.ExecuteCommandBuffer(cmd);
 cmd.Clear();
 CommandBufferPool.Release(cmd);
 }
 }
}

RT的初始化方法

1

RenderingUtils.ReAllocateIfNeeded(ref copiedColor, colorCopyDescriptor, name: "_FullscreenPassColorCopy");

这里偷懒直接用Blitter.BlitCameraTexture(cmd, source, source, colorTintMaterial, 0) 直接把source当作最终的RT并写入colorTintMaterial，按照逻辑的话，写成就和之前cmd.blit写出来到一个暂时的RT上再写入原source一样了。

1
2
3


Blitter.BlitCameraTexture(cmd, source, temp, colorTintMaterial, 0);
colorTintMaterial.SetColor(ColorTintID, colorTintSettings.colorTint);
Blitter.BlitCameraTexture(cmd, temp, source);

最终效果如图

参考

Upgrading to URP 14 (Unity 2022.2) | Universal RP | 14.0.6 (unity3d.com)