I started writing a comprehensive guide to all the CRT masks but there’s just too much going on right now and I’m too damn lazy.
If you check the shader notes within the shader file, descriptions are given for each mask.
You need to consider the resulting TVL when using these masks at different resolutions and the TVL of different CRT displays. You can determine the TVL by determining the width of the phosphor triad in pixels and dividing the vertical resolution in pixels by the width of the phosphor triad in pixels.
Some quick notes. Numbers below refer to the masks in guest-dr-venom.
0 - cgwg aperture grille. Good for emulating a consumer-grade Trinitron at 720p, looks like a PVM at 1080p.
1 - weird slot mask, needs higher than 1080p. Not great since it results in chromatic aberration
2 - 3 line aperture grille. TVL is roughly equal to a consumer-grade Trinitron at 1080p. Not a great mask; results in chromatic aberration
3 - stretched dot mask. Doesn’t resemble any known CRT mask but it avoids chromatic aberration. 270 TVL at 1080p, 540 TVL at 4K
4 - coarse dot mask, needs higher than 1080p resolutions and results in CA
5 - 2 line aperture grille, same resolution recommendations as mask 0, except this one does some kind of brightness compensation that has an undesirable effect on gamma
6 - another variant of the 2 line aperture grille; becomes fainter over brighter colors (the opposite of what happens with real CRT phosphors). Also does weird stuff to gamma in my experience.
7 - yet another 2 line aperture grille that does unwanted stuff to gamma. Black and white.
So, basically, my recommendation is to use either mask 0 at 720p to emulate a consumer-grade Trinitron or at 1080p to emulate a PVM. Use mask 3 if you want something that more closely resembles a shadow mask CRT. The guest-dr-venom slotmask (the one with separate parameters) is also very good as it avoids chromatic aberration and can be adjusted to different sizes for different resolutions. There are also a lot of good masks for 4K displays in @hunterk’s giant mask shader function.