Float precision and mobile GPUs

This post will be of academic interest to almost everyone but the floating point precision of some mobile GPUs can affect their ability to produce the correct results from shaders.

OpenGL ES defines three precision levels for float variables highp, mediump and lowp. Because highp isn’t supported in fragment shaders by some GPUs and lowp is very low precision, shaders get written to use mediump. mediump has to have a minimum accuracy of 2[sup]-10[/sup] - or 10 bits for precision. The only common GPU that doesn’t exceed this is the ARM Mali 400. NVIDIA Tegras (up to version 3 at least) support 2[sup]-13[/sup] which may be affected by this issue but are much less likely to than Mali 400s. Other mobile GPUs and desktop GPUs use 32 bit floats which have plenty of precision and won’t have this issue.

2[sup]-10[/sup] provides enough precision for shaders to access individual pixels in the game source texture so shaders that use nearest filtering will have less of an issue. Others need to be able to specify texture coordinates with much higher precision - e.g. crt-pi relies on linear filtering and precise texture coordinates in order to get even scan lines.

I don’t have a device with a Mali 400 but my desktop GPU does support 16 bit floats - which have the same 10 bits precision as Mali 400 GPUs. I’ve produced a version of crt-pi that uses 16 bit floats to demonstrate the problem. The image below shows a 1080 screen. half without curvature and half with.

You can see from the left side the issue is worse nearer the bottom of the image. This is because with floating point variables precision is relative to the value being held - errors in absolute terms are greater for larger values. e.g. 1% of 2 = 0.02, 1% of 200 = 2. The source texture coordinate is zero at the top and increases downwards so errors will be greater nearer the bottom. The right side shows greater issues because the calculation of the screen shape is introducing additional errors. Errors in calculations like this shows that shaders that use nearest neighbour filtered textures may still have problems.

Beyond academic interest, this is an issue for shader developers and users of Mali 400 (and possibly Tegra) GPUs to be aware of.

Excellent post. The inconsistency among GPUs/drivers is one of the more frustrating aspects of working on shaders, IMO.