It's about time we start busting the doors down on the shaders, too bad we don't have anyone really experienced in HLSL, I'm doing my best right now and it's slowly turning into logic in my head, but I'm suffering!
At the moment I'm trying to figure out how to throw a blur effect on the fog used by the View Range slider. I looked through both sets of shader files, twice, but didn't find anything that actually modifies the stupid fog. I used no_lighting.vsh and bad_vertex_shader.psh to figure out where the shaders are really being used (I made a ton of copies of them, renamed them to the other shader files, so I get the same effect of the ones I listed on the other ones, thus I get a bright pink surface on whatever used the .psh for example).
In the end, I did find something about the fog, it's actually in the vertex functions.inc, a small function for where the fog should be created (distance to player), but that's it. It's pretty frustrating, I thought I got somewhere.
The odd thing about it is, there is no trace of the function in the pixel shaders anywhere, so the pixel shader isn't using it, yet it's everywhere in the .vsh files, some bump maps (If I remember correctly) and other crap as an output. This is why at first I thought it's not gonna be the fog I'm talking about, but I know better now.
I talked to someone on IRC who knows a tad of HLSL, but not too much, if it was possible, because that was my first idea, to copy the function over to the pixel functions.inc, then toy with the 2d_blur.psh perhaps and apply it to the fog, but he said it wasn't possible because the pixel shader doesn't see it, but I'm not sure, he wondered how the fog is even made in the first place if it's not used by the pixel shaders.
I just hope it's not hidden away, I mean how the hell did the ILM guy achieve this then:
I've been asking about help on different forums, some interesting stuff but nothing final yet, after today's 'revelations', I might get better help, we'll see.
Otherwise, I don't get how shadows are modified either, anything I did with shadowvolume didn't really do anything, perhaps it needs to specifically be in asm, my 'test' files I used were in HLSL.
While looking for the fog, I did stumble on some obvious but I thought interesting things, I didn't look into all the spec files, and this is just a first impression, some of it might be wrong, I'll update it over the next few weeks:
I hope this will get some brainstorming going on, we have to do this already, it's been long enough.
Edit: Just so it's 'on file'; Old GDC .pdf about SWG rendering and the shader: http://developer.amd.com/media/gpu_assets/D3DTutorial06_jgrills.pdf
At the moment I'm trying to figure out how to throw a blur effect on the fog used by the View Range slider. I looked through both sets of shader files, twice, but didn't find anything that actually modifies the stupid fog. I used no_lighting.vsh and bad_vertex_shader.psh to figure out where the shaders are really being used (I made a ton of copies of them, renamed them to the other shader files, so I get the same effect of the ones I listed on the other ones, thus I get a bright pink surface on whatever used the .psh for example).
In the end, I did find something about the fog, it's actually in the vertex functions.inc, a small function for where the fog should be created (distance to player), but that's it. It's pretty frustrating, I thought I got somewhere.
The odd thing about it is, there is no trace of the function in the pixel shaders anywhere, so the pixel shader isn't using it, yet it's everywhere in the .vsh files, some bump maps (If I remember correctly) and other crap as an output. This is why at first I thought it's not gonna be the fog I'm talking about, but I know better now.
I talked to someone on IRC who knows a tad of HLSL, but not too much, if it was possible, because that was my first idea, to copy the function over to the pixel functions.inc, then toy with the 2d_blur.psh perhaps and apply it to the fog, but he said it wasn't possible because the pixel shader doesn't see it, but I'm not sure, he wondered how the fog is even made in the first place if it's not used by the pixel shaders.
I just hope it's not hidden away, I mean how the hell did the ILM guy achieve this then:
I've been asking about help on different forums, some interesting stuff but nothing final yet, after today's 'revelations', I might get better help, we'll see.
Otherwise, I don't get how shadows are modified either, anything I did with shadowvolume didn't really do anything, perhaps it needs to specifically be in asm, my 'test' files I used were in HLSL.
While looking for the fog, I did stumble on some obvious but I thought interesting things, I didn't look into all the spec files, and this is just a first impression, some of it might be wrong, I'll update it over the next few weeks:
.VSH - Tested by replacing it with: no_lighting.vsh
water_pass - controlls water waves/lights/movement
stars - controlls brightness (keeps them fully sized I believe) needs further checking out
terrain_dot3_vs20_blend - controls ground lighting
procedural_clouds - controls procedural cloud (duh) lighting... but I'm not 100% sure
gradient_sky - controls the 'filter' for the sky and the horizon, it has a 'fog' effect
cloudlayer - controls cloud movement, maybe shading like I wrote for procedural
e_planet_cloud - might control cloud shading, needs to be checked again
a_simple - Controls lighting on trees, some clothing and other stuff
2d - handles some interface 'coloring', like above and below the loading screen image, and other weird stuff
.PSH - Tested by replacing it with: bad_vertex_shader.psh
water_pass - controls the water surface (on/off, movement?)
vertex_color - controls sun light hue if you look at it, also controls some UI colors
texren_copy_c1a1 - facial shading, not sure what else
terrain_dot3_ps20_blend - Handles something on the ground textures
gradient_sky - What it sounds like, handles the sky color/shader (above the clouds)
h_color2_pp_ps20 - handles lighting on some pieces of clothing, like artisan shirt
cloudlayer - Not 100% sure what it does, but it controls the whole cloudlayer
a_specmap_pp_ps20 - Handles some random static objects, some roofs
a_emismap_ps20 - Handles lighting on some windows
a_envmask_specmap_ps20 - Handles other windows
a_modulate - Handles trees (foilage) and things in the sky (planets/StarDestroyers)
a_simple - Handles trees (wood), creatures, flags, random static objects
a_spec_pp_ps20 - Handles some roofs, bluefrog, I'd imagine random stati stuff
a_alpha_envmask - Street lights (not the glow though, it just shows the actual light itself)
a_detail_specmap_ps20 - Handles lighting on crafting stations
a_alpha_pp_ps20 - Handles some structure textures and tree foilage (not sure what's different to a_modulate yet)
Note: If you look into them, most files are in pairs of 3. As far as I know, they're split into different qualities, here is an example:
a_2blend.vsh - Written in ASM, lowest quality
a_2blend_vs11.vsh - Written in VS1.1, medium quality ?
a_2blend_vs20.vsh - Written in VS2.0, best quality
I hope this will get some brainstorming going on, we have to do this already, it's been long enough.
Edit: Just so it's 'on file'; Old GDC .pdf about SWG rendering and the shader: http://developer.amd.com/media/gpu_assets/D3DTutorial06_jgrills.pdf