CMYK shader

RetroArch Additions Shaders
#1

UPDATE 3

Moved to https://github.com/aybe/RetroArch-shaders !

UPDATE 2 http://1drv.ms/1qHspnY

(unzip in your ‘shaders’ folder)

Two presets now :

  • cmyk.cgp -> standard conversion to CMYK (blacks are not entirely black)
  • cmykPureBlackWhite.cgp -> blacks and whites are untouched, some might prefer this

README included in the archive, make sure to read it for more details.

In order : original, cmyk.cgp, cmykPureBlackWhite.cgp

UPDATE 1

Final version :smiley:

http://1drv.ms/1omXeim

No more issues, no tricks used, no more weird fade outs, works flawlessly (there was an issue in the LUT being generated).

Installation : unzip as-is in your ‘shaders’ folder

NOTE : if you happen to see gray instead of white, you need to choose another shader preset and choose the preset again (switching shader with N/M keys won’t be enough). Currently I don’t know why this happens sometimes …

Original colors: [attachment=1:2z7jjn6q]RetroArch-0927-214722.png[/attachment:2z7jjn6q]

CMYK colors: [attachment=0:2z7jjn6q]RetroArch-0927-214739.png[/attachment:2z7jjn6q]

Hi,

This is something I was thinking about for quite some time and finally tested the idea thanks to Retroarch :smiley:

In order :

  • original RGB (GTU shader more exactly)
  • transformed to CMYK space

The effect looks really great on some games, but there’s a drawback which I haven’t been able to address yet : white colors.

You can clearly see the problem during fade outs, well that’s normal since CMYK color space cannot represent all RGB colors but I thought there might be a way to overcome this if we use the same logic as paper (subtractive), against a screen logic (additive); my attempt didn’t work for today, I’ll take another look later.

I’m posting it here if someone has an idea / wants to improve the concept.

NOTE : this is not an RGB to CMYK conversion using the formulas on the web, these simply translate between these spaces, i.e. going RGB -> CMYK -> RGB does absolutely no change using them (actually they are in the shader but they are not used). Instead I used a color profile and rendered 16.7M colors and converted to CMYK space, result is like it is in a sheet of paper.

download : http://1drv.ms/1rwvgpr (it is 12MB because there’s a 4096*4096 texture)

Waiting for your comments :smiley:

1 Like
#2

Very interesting! I see what you mean about the fadeouts. It drags across the hues, making a rainbow effect, instead of just getting darker.

However, my results are hued differently from yours:

768×448

#3

ah yes interesting,

I am not yet sure about how well does my ‘scale_type’ setting scales to any video settings, I’ll try with SF3 and see if I can fix it. (it seems to affect the returned coords, I don’t know why but I’ll try to find out)

EDIT : I’ve tried to run SF3 but it won’t run, whether using MAME and associated romset or using the roms from the first CPS3 emu.

for FBA

RetroArch: [libretro INFO] :: [FBA] Archive: sfiii
RetroArch [ERROR] :: [libretro ERROR] :: [FBA] ROM index 1 was not found ... CRC: 0xcfc9e45a
RetroArch [ERROR] :: Failed to load game.
RetroArch [ERROR] :: rarch_main_init_wrap() failed.

for MAME

sfiii_euro.29f400.u2 NOT FOUND (tried in sfiii sfiii)
cap-sf3-3.chd NOT FOUND
FATALERROR: ERROR: required files are missing, the game cannot be run.

I’ve read that there’s a CPS3 bios but I can’t find any, is this correct ?

Can you explain how you run CPS3 games ?

Thanks

#4

I use FBA, but the crazy colors are not specific to CPS3, it’s just what I had lying around for screenshots :slight_smile: It had the same results for me in SNES.

Since I’m getting crazy rainbow colors, I’m guessing that it’s sampling from the palette 90 degrees rotated relative to your sampling. FAKEDIT: hmm. Works as expected in Windows (both Win and Lin using AMD GPUs). Could be something weird with Mesa drivers. The fadeout effect is less of a problem in Windows, too. It just gets stuck on some dark colors before dropping all the way to black.

Your technique for color-shifting is really clever. I played around with a similar concept to try and make a LUT-based palette-switching shader for NES, etc., but I didn’t get very far. To generate your LUT, do you just take a full RGB palette and then change the color space/depth in Photoshop?

#5

UPDATE:

I’ve made some progress in fixing white and black hues, grays are still off; I guess I’ll have to use HSL to really fix things.

I was considering creating a texture that would hint for out of space colors but it’ll probably be a naive approach, I’ll just approach the problem using color-related formulas.

For the generation it’s really simple :

(C# WPF project)

int width = 4096;
int height = 4096;
var ints = new int[width*height];
int i;
i = 0;
for (int x = 0; x < 256; x++) {
	for (int y = 0; y < 256; y++) {
		for (int z = 0; z < 256; z++) {
			int i1 = (0xff << 24) | (x << 16) | (y << 8) | z;
			ints[i++] = i1;
		}
	}
}
BitmapSource bitmapSource = BitmapSource.Create(width, height, 96, 96, PixelFormats.Pbgra32, null, ints, width*4);
var formatConvertedBitmap = new FormatConvertedBitmap(bitmapSource, PixelFormats.Cmyk32, null, 0);

(x, y, z being r,g,b)

Currently I don’t know which color space they are using in .NET but I think it’s an optimum one. Using even better-tuned profiles from Photoshop such as US Coated SWOP is clearly something I will do later, when polishing :smiley: For now I’ve had really mixed results with NES games, some are good and some are really wrong. And the scaling/shader related stuff is still a little vague to me, apparently with different settings, the texcoords seems to be higher than 0 to 1 … I’ll understand it with time.

Regarding your issue, currently I don’t run a Linux distro but I’ll happily take a look when the shader is done. Right now my objective is to get a decent and pleasing conversion working well for most if not all games, then I’ll look at optimizing and fixing things so it runs well for most configs.

So, anyone, this is an early alpha, do not expect it to work OOB yet; FYI the configuration I am developing it on is Windows + an HD7850.

Also regarding color profiles, unfortunately there is no equation/formula to easily translate, ICC profiles are complex by nature and parsing them is hard (crazy). The simplest thing to do is to use the code above, then use a ColorConvertedBitmap in WPF which allows you to specify an ICC profile which for instance might come from Photoshop.

to be continued :smiley:

#6

UPDATE

I’ve been looking at the problem and here are my conclusions,

  • my trick of just passing non-saturated works but it’s not robust and really cheap :lol:
  • the path to take is just as PS does but more on this soon

The thing is to find/understand the algorithms that are below, because simply using a LUT is not enough.

https://wiki.archlinux.org/index.php/CMYK_support_in_The_GIMP

Intent
The color proof (rendering) intent can be one of the following:

perceptual
relative colorimetric
saturation
absolute colorimetric

This is how it is now, no tricks used, just using the LUT :

This is how it should be, perceptual mode

And for this task, the source code of the separate plugin shown in the page above should be very helpful =D

I’ll take a look at it when I have some free time.

#7

Hi mate,

Just uploaded the final version, all things work properly. (updated the first post) Check it out and tell me what you think !

#8

Moved to GitHub : https://github.com/aybe/RetroArch-shaders :smiley:

#9

Nice. I’ll check 'em out. :smiley:

I haven’t gotten to test out your latest on the CMYK in Windows yet, but would you mind uploading a LUT for full RGB color, as well? I was looking into using indexed color spaces in Photoshop with your shader to apply other palettes, but working from your CMYK LUT was throwing things off (as would be expected).

#10

Yes, I’ve just uploaded a BGR LUT : https://github.com/aybe/RetroArch-shaders/blob/master/shaders/bgrLUT.jpg

Formula for retrieving the color is here : https://github.com/aybe/RetroArch-shaders/blob/master/shaders/cmyk.cg#L45

There’s another shader I’ve started converting from HLSL, a surface blur shader like Photoshop, basically it’s a blur but with sharp edges so smooth blending but edges stay sharp. Currently it drops to 16 fps when in full-screen … I’ll see if I can optimize that and post it in the forum later :smiley:

[attachment=0:21vjk78d]snap0530.png[/attachment:21vjk78d]

#11

Got a chance to test your latest cmyk shader and it works perfectly :slight_smile: Really captures the look of promo screenshots from video game magazines!

Using that bgrLUT you uploaded, I used Photoshop’s indexed color mode with a 64-color NES palette I found online and it worked beautifully: So, while your shader obviously serves its intended purpose, it also has use as a general indexed color shader. All one needs to do is take the bgrLUT and convert it to their indexed palette using an *.act color palette file.

I tried your flickering scanlines shader, as well. It works great here, and it looks like it functions very similarly to the interlacing shader I wrote (misc/interlacing.cg in the common shaders repo) :slight_smile:

That last blur shader looks nice and bloomy. Good luck getting it performant enough for general usage.

#12

I have to say that I am rediscovering some of my games using CMYK :wink:

Your experiment looks good, makes me think that RetroArch is missing the EGA shader from DOSBox :smiley:

I think I’ve missed something in the process, my current setup (RetroArch-v1.0.0.2-64-bit.zip) doesn’t have it, neither the archives below:

RetroArch-20140105-Win32-MegaPack.zip
RetroArch-20140105-Win64-MegaPack.zip
RetroArch-v1.0.0.2-32bit.zip

Where is it ?

Regarding the surface blur, I think I’ll have to manually put the coefficients, currently mine uses a for…loop and while it is flexible it’s probably what kills performance, I’ll take a look !

#13

There’s a github repo with a bunch of Cg shaders: https://github.com/libretro/common-shaders They should probably be included in the megapacks in future versions…

#14

Okay, I’ll take a look, thanks !

#15

Hey thanks for making this. I couldn’t open this shader with .jpg, but it worked when I convert it to .PNG without compression. I was experimenting with the texture because I wanted to do a color correction for one of my shaders that needs a different red, green, and blue chroma color. It worked very well when I tested it. I have a question, is it possible to have the texture with lower resolution? Anyway, the shader looks good.

#16

I have started working on another version that uses a smaller (and different) texture but it’s not working for the moment :

[attachment=1:un552sv3]RetroArch-1003-174837.png[/attachment:un552sv3]

[attachment=0:un552sv3]HSL_LUT_CMYK.png[/attachment:un552sv3]

I guess I had to switch to HSL for having a smaller texture but as you can see it is not working, not yet sure if it will even work.

Regarding the resize, not sure, I have quickly tried rescaling the image in a editing software but it doesn’t seem good, it’s related to how the image is. As you can see there are blocks of hues which are of a certain height and when you down size, interpolation wouldn’t even help since it’s really different. I’ll see whether if I regenerate the bitmap from the source at a smaller scale will fix the issue.

For JPEG it’s weird because I’ve read that a build ‘might’ support PNG but I don’t know for JPEG. I was about to deliver the LUT in PNG but since it’s 64Mb I’ve decided not to. I do have one optimized with OptiPng at 13 Mb, I might put it later; thing is that the JPEG is barely different than the original one so I favored the download size for the time being.

I’ll try to take a look over the weekend :smiley:

#17

Yeah, I had to convert to png, as well, but I figured you were using jpg instead for bandwidth consideration.

#18

I guess you are Linux users, right ? :smiley:

(I’ll update the package with a PNG texture next time !)

Pokefan531:

Unfortunately a smaller texture doesn’t seem to work, I’ve tried to regenerate a 512*512 texture and update the shader :

I think it comes from the subtleties of the CMYK color space, if you look at the LUT closely, you will see darker tones between lighter tones and generating a smaller removes them … while on RGB it’s not much of a problem since you can safely interpolate, it’s not the case for CMYK. (someone correct me if I’m wrong)

left is 512px. right is 4096px

last hope might be the HSL approach I’ve explained … we’ll see

hunterk

I’ve been able to find a setting for the surface blur shader :

there are a few issues though,

  • small sections of white such as below Guy’s hand becomes black, that’s inherent to the algorithm
  • the shader runs too slowly when scaled, not sure if this is an issue unless a shader in a previous stage might enhance the input
  • the best results I’ve got are by mixing it with a bit from the original source

there seems to be an O(1) equivalent algorithm and apparently it would fix issue #1, will have to try that one !

#19

This seems likely to me. Pretty sure all interpolation will happen in the RGB space, which pretty much negates what you’re trying to do.

For the blur, you can try forcing a smaller size using an initial pass at 2x or whatever and then scaling it up after that, but it will negatively affect the quality of the blur. What algo is it using? Gaussian? If so, you might be able to utilize some of TroggleMonkey’s extensive collection of blurs (in the common shaders repo, under the ‘blurs’ subdirectory).

#20

After looking again at all these files, my conclusion is that it will never work for CMYK and the current LUT layout, it’s just not doable unless another layout is adopted.

Actually a surface blur is just a gaussian blur with an additional distance normalization factor which enhances the edges. I’ll take a look at the shaders you’ve mentioned !

Have you tried the Technicolor shaders I’ve uploaded ? on some games they really look cool even though it is cheap at times :lol: