Feedback-based PREV in shaders (ideas)

RetroArch Additions Shaders
#1

So right now, PREV is FIR-based. Having an IIR-model would improve flexibility, but complexity will increase dramatically. I’m considering adding this, but it needs to be very restrictive and optional.

One thing I’m thinking of is an optional FEEDBACK. In cgp/glslp, you can designate a single pass (FBO) which will be fed back into next frame. (FIne for motion blur, etc).

Feeding back multiple passes, and allow feeding back several frames from same pass is ofc possible in theory, but horribly complex and possibly very niche (temporal higher-order IIR, ye, totally batshit :P) A compromise here would be allowing multiple feedback frames, but only from one pass. In that case, it’d be something like PREV now (FEEDBACK, FEEDBACK1, FEEDBACK2, …)

#2

When you’re saying multiple feedback from one pass, do you think memory could be an issue if the chosen FBO is the last pass (biggest)?

#3

So, the choice is: one filtered frame available to multi-pass shaders or multiple filtered frames available to a single-pass-only shader? If so, I’m thinking the first would be more useful. I’m not coming up with any use cases for the second off the top of my head, but I’m inexperienced with this sort of thing and would be very interested in hearing some applications.

#4

If I’m not wrong, the two options are:

1-

2-

#5

#1 is more correct there. Yes, memory could be an issue, which is why I have to restrict things. Multiple frames of one pass, means that you can take one pass, say PASS2, and have it be fed back into next frames, which means PASS2 would get its own ring buffer of FBOs.

#6

I can’t see why you’d need more than the previous frame filtered. It’s enough for motion blur. I don’t see any benefits of having access to filtered FBO from FRAME 3, for example. Even if I want to improve animation by comparing previous frames, I think RAW ones are enough for this task and much less demanding.

So, I think approach #2 could be more useful. And probably use less memory. And you could just call it FEED# (FEED1, FEED2, etc). Where # indicates the PASS number.

#7

You could have a use case where you want FIR-behavior, but for a particular pass (and not ORIG). Basically, PREV is ORIGs history, and FEEDBACK could be PASSn’s history. To make it leaner memory wide, you could have something in the cgp like: shaderN_feedback_frames = 1, etc.

I’m not sure when you’d need the previous frame only for all passes. That would only accomodate motion blur.

#8

What about having at least two filtered passes from previous frame, where one of them couldn’t have previous history?

I can see a useful application if I have access to two filtered passes from previous frame: save some computations.

#9

That sounds kinda arbitrary. Once you go over 1, any fixed number feels very hardcoded (PREV does this, but there was no good way to avoid it at the time). Can you describe that particular use case?

#10

Something like this:

#11

Ok, no need to be so arbitrary. In that example I could read previous RAW to get PASS1 parameters.

#12

Since byuu seems to be considering the same capability, it might be a good idea to not set anything in stone until we see what he settles on. That way, if it’s good, we’re (conceptually) compatible and if it’s lacking, we won’t make the same mistake(s) :wink:

#13

Why should we?

He has had many opportunities to build bridges with us and make his spec somewhat compatible with ours - he has never shown any interest in playing ball or cooperating. Hell, I include bsnes/higan now with RetroArch Android - did he ever try to promote it to people? Did he ever recommend it to people? Hell no. He seems to be trying to do his damnedest best to pretend like we don’t exist and don’t matter. Why he does this I don’t know - all I know is that he is not a reliable or trustful person at this point. Not to mention he abandons APIs and specs faster than I change my underwear.

I say that we do our own stuff independent of what he does with Higan. Not wanting to be dependent on byuu’s code mode swings is why we forked libsnes in the first place.

#14

In my experience, if someone else is trying to do the same thing as me, I wait and see how it turns out for them, rather than make their same silly mistakes. It’s a labor/risk-saving strategy. This is a rare opportunity to let someone else be your guinea pig, but I can appreciate the satisfaction of being a first-mover, as well.

Either way, it doesn’t really matter to me, since I won’t be the one implementing it. :stuck_out_tongue:

#15

Well, maister can decide what he wants to do since I won’t be implementing it either.

#16

Yes, I have zero confidence that any shader spec from higan will be stable. With the latest additions to ruby, it’s basically equivalent feature-wise to RetroArch’s spec, so I hope I won’t have to play any “catch-up game”. That would be hilarious. <_< The reasoning here is that if you have something less compatible, but more featureful, you’ll foster an “elitist” attitude around it, which would be very bad. A lot of confidence would be lost in the shader specs.

Compared to RA right now, it’s basically: Pros:

  • Geometry shaders (GL 3.2+ only, not GLES3)
  • More flexible FBO formats (10-bit, 12-bit, 16-bit float in addition to RGBA8 and 32-bit float) (GL3+, GLES3+ only).
  • NPOT only (can be simpler to write certain shaders, potential less bandwidth usage, see cons …)

Cons:

  • Poor mix-and-match model. While the BML manifest works very similarily to .cgp/.glslp, it’s fully designed for shader folders. Also, shader files are split up into .vs/.fs, which imo makes it very awkward to mix-and-match from a GUI of any kind.
  • Lack of GLES2, GLES3 (!), GL2.1 support. This one is very glaring … Won’t even run on Mesa atm at all due to this requirement. So much for open source.
  • A concept of “default shader passes”. Not forcing all shader passes to be present is a bad idea as it adds a certain “magic” element to the spec. This also severely hurts understanding of how it’s all tied together. This was the cause of the recent AMD bug for instance. In the end you have to write the passes yourself anyways, and we have a useless redundant part. Also, changing the “built-in” shader could be devastating for compat.
  • NPOT only (worse precision which can affect certain shaders, not strictly GLES2 compatible, requires extension).
  • Hard to target for cross-compilation. The texture uniform objects are arranged in arrays. This makes cgc cross compilation much harder.

Anyways, I’ll probably wait and see what happens. I have enough to do outside RetroArch atm.

#17

The default shader passes thing is indeed mystifying. From an end-user perspective, it makes no difference how many files are in the folder, since they wouldn’t be poking around in there anyway. From a shader author perspective, it’s confusing and you have to dig into the higan source code to actually know wtf is going on if anything goes wrong.

#18

Ye, it’s obviously a relic from the legacy fixed-function-style GL shaders, where you had a well defined “default” vertex shader for example. It makes no sense whatsoever in modern GL.