Modern replacement for png images in shaders?

Might be a stupid question but since size is a concern why not use a more modern image format like avif or webp ?

2 Likes

Yeah, I agree, it would be good to look into these and see what would be the best fit for Retroarch and see what the complexity is of adding one of the formats.

Do you have specific experience with one vs the other that you could share with us?

1 Like

I think the biggest potential issues would be: compatibility with the codebase (license and language) and the size/complexity of the decoder dependency.

2 Likes

Isn’t PNG a lossless format though?

1 Like

Special tools use a Floyd - Steinberg dithering algorithm by first converting the PNG to Indexed color, lowering the color count, optionally using dithering, and converting back to RGB.

3 Likes

I only have experience with modern image formats in a web context so I didn’t have to implement the format itself sorry. But the size difference is quite important compared to jpg and png.

1 Like

webp looks the most promising but it is not open source, I’m not sure how we would fit that into the RA scheme.

At the very least some license info would have to be distributed with RA. (According to the google docs.)

From what I’ve read AVIF is better than WebP in all aspects and has been designed to replace it. The AVIF library is BSD so maybe compatible with the codebase ?

1 Like

For some reason I was thinking it didn’t support an alpha channel… I was mistaken.

I just tested and AVIF produces a 4K image that is smaller than my PNG compressed 1080p image.

That is pretty mind blowing! :exploding_head:

3 Likes

My big takeaway on that reddit is the licensing issues @hunterk mentioned.

Speaking of takeaways, it might be beneficial to split the conversation about modern image formats so others can chime in.

1 Like

yes, we wouldn’t want to touch the hevc licensing, but AV1/AVIF looks to be good for it. libavif looks like a good fit, licensing- and code-wise, since it’s BSD and C99 (C89 is preferable, but C99 is still preferable to C++whatever). I haven’t pulled the codebase down to see how large it actually is, though, and I was a little concerned by this:

No AV1 codecs are enabled by default. Enable them by enabling any of the following CMake options:

  • AVIF_CODEC_AOM - requires CMake, NASM
  • AVIF_CODEC_DAV1D - requires Meson, Ninja, NASM
  • AVIF_CODEC_LIBGAV1 - requires CMake, Ninja
  • AVIF_CODEC_RAV1E - requires cargo (Rust), NASM

These libraries (in their C API form) must be externally available (discoverable via CMake’s FIND_LIBRARY ) to use them, or if libavif is a child CMake project, the appropriate CMake target must already exist by the time libavif’s CMake scripts are executed.

That is, even though this lib seems like a good fit, if it requires a cascade of other deps with funky “modern” build systems, etc., we’re still not going to be in good shape.

It looks like liboam is still a good option and is similar in code and licensing to libavif.

8 Likes

Since AVIF has an animated format… would adding the decoder open the door for animated images?

3 Likes

potentially but not necessarily. we support gif images for some things but animated gifs still don’t work.

I played around with using apngs as a sort of rudimentary video format for video snaps, but that didn’t work either, despite using a decoder that supports them (this stb_image fork/PR: https://github.com/nothings/stb/pull/625, IIRC)

3 Likes

Has there been any talk about this. I just converted a 4MB PNG and it was 10 times smaller as an avif image.