john/cell
1
0
Fork 0
Code Issues 31 Pull Requests Packages Projects Releases 2 Wiki Activity

heavy overhaul of documentation organizaton

Browse Source
This commit is contained in:
John Alanbrook
2025-02-08 01:45:51 -06:00
parent 81b42eec67
commit c389e0744a
25 changed files with 591 additions and 1540 deletions
Download Patch File Download Diff File Expand all files Collapse all files

152
todo/graphics.md Normal file
View File

@@ -0,0 +1,152 @@
# Graphics drawing
## Terminology
Texture - a set of bytes on the GPU, not directly accessible
Surface - a set of bytes in RAM, modifiable
rect - a rectangle of {x,y,width,height}
Image - combination of a texture and rect, where the rect defines the UV coordinates on the texture to draw
# Drawing, cameras, viewports, logical size, and so on
A camera is a view into the game world. A camera can be "rendered", which means it renders the world, and what it can see in the world. A camera may draw to a surface, or to the main window. Objects in the world will render so that if their position is equal to the camera position, that is in the center of the screen. HUD functions always render so [0,0] is the bottom left of the camera's view.
Cameras always draw to their own render target. Then, they draw that render target to the framebuffer.
# COORDINATES
Screen coordinates start in the upper left corner at [0,0] and extend to the bottom right, in pixels. Raw mouse coordinates are in these.
# RENDERING PIPELINE
In prosperon, you call graphics rendering functions at well defined hook points. These are interleaved as necessary with predefined created objects, like sprites, 3d world models, and so on.
The engine stores a command buffer. When you issue "draw" commands, they are recorded into the command buffer. These are batched as much as they can be; if there is no significant state change between, the draw commands can be coalesced into one. Then, for each camera, the draw commands are executed.
# RENDERING COMPONENTS
## MATERIALS
A material defines the inputs to a shader.
## PIPELINES
Pipelines are how the rendering engine is set up. Switching pipelines can be done for special effects.
## SPECIAL EFFECTS
Sometimes you want a special effect. While there are many objects in prosperon you can create and have the engine handle for you, a special effect typically requires a bit of code.
# LAYERS
All things that draw have a layer. If no layer is set, the implicit layer is "0". Even draw and hud functions have a layer. To draw a draw function on a specific layer, set that function's "layer". ie,
this.draw = function() { render.rect(); }
this.draw.layer = -5;
Now that layer will draw at the -5 layer.
# CAMERAS
Everything is drawn via cameras. Cameras can draw directly to the screen, or they can draw to an offscreen render target. By default, everything is drawn to all cameras. There will eventually be a tag that lets you filter what is drawn to specifc cameras.
Cameras have a resolution they draw at, "size".
## TEXTURES
Anatomy of rendpering an image render.image(path)
Path can be a file like "toad"
If this is a gif, this would display the entire range of the animation
It can be a frame of animation, like "frog.0"
If it's an aseprite, it can have multiple animations, like "frog.walk.0"
file^ frame^ idx
render.image("frog.walk.0",
game.image("frog.walk.0") ==> retrieve
image = {
texture: "spritesheet.png",
rect: [x,y,w,h],
time: 100
},
frames: {
toad: {
x: 4,
y: 5,
w: 10,
h: 10
},
frog: {
walk: [
{ texture: spritesheet.png, x: 10, y:10, w:6,h:6, time: 100 },
{ texture: spritesheet.png, x:16,y:10,w:6,h:6,time:100} <--- two frame walk animation
],
},
},
}
texture frog {
texture: {"frog.png"}, <--- this is the actual thing to send to the gpu
x:0,
y:0,
w:10,
h:10
},
## RENDER MODES
/* rendering modes
ps1
gouraud
diffuse // 16 bit color, 5-5-5
7 dynamic lights, 1 ambient
textures are affine
no vertex skinning
256x256 texture max (generally 128x128)
320x240, variable up to 640x480
n64
gouraud
diffuse
combiner // a secondary texture sometimes used to combine
7 dynamic lights, 1 ambient
320x240, or 640x480
sega saturn
gouraud
diffuse
320x240 or 640x480
ps2
phong
diffuse
combiner // second texture for modulation of diffuse
combine_mode // int for how to combine
dreamcast
phong
diffuse
combiner // second texture; could be an environment map, or emboss bump mapping
fog
640x480
640x448, special mode to 1280x1024
gamecube
phong
diffuse
+7 textures // wow!
8 dynamic lights
640x480
*/
/* meshes
position (float3)
color (rgba)
uv
*/
/* materials, modern pbr
any object can act as a "material". The engine expects some standardized things:
diffuse - base color texture
bump - a normal map for dot3 bump maping used in phong shading
height - a grayscale heightmap
occlusion - ambient occlusion texture
emission - texture for where model emits light
bump2 - a second normal map for detail
metallic - a metal/smoothness map
specular - specular map, alternative for the metallic workflow
*/

186
todo/ops.md Normal file
View File

@@ -0,0 +1,186 @@
# RENDERING PIPELINE
The basic flow for developing graphics here:
1) develop a render graph
2) decide what to draw
The render graph is the "big idea" of how the data flows through a render; inside the execution, you utilize "what to draw".
Prosperon provides you with functions to facilitate the creation of rendering pipelines. For example, you could use "shadow_vol" function to create buffer geometry with shadow volume data.
Unity has a "graphics.rendermesh" function that you can call, and that unity automatically calls for renderer components. It is the same here. But there are a handful of other types to draw, particularly for 2d.
## 2D
### Anatomy of a 2d renderer
Traditionally, 2d rendering is a mix of tilemaps and sprites. Today, it is still more cost effective to render tilemaps, but we have a lot more flexibility.
NES
Nes had 1 tilemap and up to 8 sprites per scanline.
SNES
Up to 4 tilemap backgrounds, with priority, and flipping capability. 32 sprites per scanline, and by setting the priority correctly, they could appear behind background layers.
GB
One background layer, 10 sprites per scanline/40 per frame.
GBA
Up to 4 layers, sprites with affine transforms!
DS
Up to 4 layers; many sprites; and a 3d layer!
Sega saturn
This and everything else with generic vertex processing could do as many background layers and sprites as desired. This is what you get with prosperon on most modern computers. For more limited hardware, your options become limited too!
### Prosperon rendering
Layers
Every drawable 2d thing has a layer. This is an integer that goes from -9223372036854775808 to 9223372036854775808.
!!! On hardware that supports only a limited number of layers, this value must go from 0 to (layer #).
Layer sort
Within a layer, objects are sorted based on a given criteria. By default, this is nothing, and the engine may reorder the draws to optimize for performance. Instead, you can choose to sort by their y axis position, for example.
Parallax
Layers can have a defined parallax value, set at the engine level. Anything on that layer will move with the provided parallax. Each layer has an implicit parallax value of "1", which means it moves "as expected". Below 1 makes it move slower (0 makes it not move at all), 2 makes it move twice as fast, etc.
Tilemaps
These are highly efficient and work just like tilemaps on old consoles. When you submit one of these to draw, Prosperon can efficientally cull what can't be seen by the camera. You can have massive levels with these without any concern for performance. A tilemap is all on its own layer.
Tiles can be flipped; and the entire tilemap can have an affine transformation applied to it.
Sprites each have their own layer and affine transform. Tilemaps are just like a large sprite.
In addition to all of this, objects can have a "draw" event, wherein you can issue direct drawing commands like "render.sprite", "render.text", "render.circle", and so on. This can be useful for special effects, like multi draw passes (set stencil -> draw -> revert stencil). In this case, it is the draw event itself with the layer setting.
## 3D
3d models are like 3d sprites. Add them to the world, and then the engine handles drawing them. If you want special effects, its "draw" command can be overridden.
As sprites and 3d models are sent to render, they are added to a list; sorted; and then finally rendered.
## THE RENDERER
## Fully scriptable
The render layer is where you do larger scale organizing. For example, for a single outline, you might have an object's draw method be the standard:
- draw the model, setting stencil
- draw a scaled up model with a single color
But, since each object doing this won't merge their outlines, you need a larger order solution, wherein you draw *all* models that will be outlined, and then draw *all* scaled up models with a single color. The render graph is how you could do that. The render graph calls draw and render functions; so with a tag system, you can essentially choose to draw whatever you want. You can add new shadow passes; whatever. Of course, prosperon is packed with some standard render graphs to utilize right away.
Each graphical drawing command has a specific pipeline. A pipeline is a static object that defines every rendering detail of a drawing command.
A drawing command is composed of:
- a model
- a material
- a pipeline
The engine handles sorting these and rendering them effectively. There exist helper functions, like "render.image" which will in turn create a material and use the correct model.
You execute a list of drawing commands onto a render target. This might be the computer screen; it might be an offscreen target.
The material's properties are copied into the shader on a given pipeline; they also can have extra properties like "castshadows", "getshadows", and so on.
An *image* is a struct {
texture: GPU texture
rect: UV coordinates
}
## 2D drawing commands
The 2d drawing commands ultimately interface with a VERY limited subset of backend knowledge, and so are easily adaptable for a wide variety of hardware and screen APIs.
The basic 2D drawing techniques are:
Sprite - arbitrarily blit a bitmap to the screen with a given affine transformation and color
Tiles - Uniform squares in a grid pattern, drawn all on a single layer
Text - Generates whatever is needed to display text wrapped in a particular way at a particular coordinate
Particles - a higher order construction
Geometry - programmer called for circles or any other arbitrary shape. Might be slow!
## Effects
An "effect" is essentially a sequence of render commands. Typically, a sprite draws itself to a screen. It may have a unique pipeline for a special effect. But it might also have an "effect", which is actually a sequence of draw instructions. An example might be an outline scenario, where the sprite draws a black version of it scaled 1.1x, and then draws with the typical pipeline afterwards.
## A frame
During a frame, the engine finds everything that needs rendered. This includes enabled models, enabled sprites, tilemaps, etc. This also includes programmer directions inside of the draw() and hud() functions.
This high level commands are culled down, accounting for off screen sprites, etc, into a more compact command queue. This command queue is then rendered in whichever way the backend sees fit. Each "command queue" maps roughly into a "render pass" in vulkan. Once you submit a command queue, the data is sorted, required data is uploaded, and a render pass draws it to the specified frame.
A command is kicked off with a "batch" command.
var batch = render.batch(target, clearcolor) // target is the target buffer to draw onto
target must be known when the batch starts because it must ensure the pipelines fed into it are compatible. If clearcolor is undefined, it does not erase what is present on the target before drawing. To disable depth, simply do not include a depth attachment in the target.
batch.draw(mesh, material, pipeline)
This is the most fundamental draw command you can do. In modern parlance, the pipeline sets up the GPU completely for rendering (stencil, blend, shaders, etc); the material plugs data into the pipeline, via reflection; the mesh determines the geometry that is drawn. A mesh defines everything that's needed to kick of a draw call, including if the buffers are indexed or not, the number of indices to draw, and the first index to draw from.
batch.viewport()
batch.sprite
batch.text // a text object. faster than doing each letter as a sprite, but less flexible
// etc
batch.render(camera)
Batches can be saved to be executed again and again. So, one set of batches can be created, and then drawn from many cameras' perspectives. batch.render must take a camera
Behind the scenes, a batch tries to merge geometry, and does reordering for minimum pipeline changes behind the scenes.
Each render command can use its own unique pipeline, which entails its own shader, stencil buffer setup, everything. It is extremely flexible. Sprites can have their own pipeline.
ULTIMATELY:::
This is a much more functional style than what is typically presented from graphics APIs. Behind the scenes these are all translated to OpenGL or whatever; being functional at this level helps to optimize.
IMPORTANT NOTE:
Optimization only happens at the object level. If you have two pipelines with the exact same characteristics, they will not be batched. Use the exact same pipeline object to batch.
## SCENARIOS
BLOOM BULLETS
You want to draw a background; some ships; and some bullets that have glow to them. This amounts to two ideas:
1) draw the background and ships
2) draw bullets to a texture
3) apply bloom on the bullet
4) draw bullets+bloom over the background and ships
Steps 1, and 2-3, can be done in parallel. They constitute their own command queues. When both are done, the composite can then happen.
var bg_batch = render.batch(surf1, camera);
bg_batch.draw(background)
bg_batch.draw(ships)
bg_batch.end()
var bullet_batch = render.batch(surf2, camera);
bullet_batch.draw(bullets)
bullet_batch.end()
var bloom = render.batch(surf3, postcam)
bloom.draw(bullet_batch.color, bloom_pipeline)
bloom.end()
var final = render.batch(swapchain)
final.draw(bg_batch.color)
final.draw(bloom.color)
final.end()
When 'batch.end' is called, it reorders as needed, uploads data, and then does a render pass.
3D GAME WITH DIRECTIONAL LIGHT SHADOW MAP
var shadow_batch = render.batch(shadow_surf, dir_T)
shadow_batch.draw(scene, depth_mat) // scene returns a list of non culled 3d obejcts; we force it to use depth_mat
shadow_batch.end()
base_mat.shadowmap = shadow_batch.color;
var main_batch = render.batch(swapchain, camera)
main_batch.draw(scene)
main_batch.end()
FIERY LETTERS
This pseudo code draws a "hello world" cutout, with fire behind it, and then draws the game's sprites over that
var main = render.batch(swapchain, 2dcam)
main.draw("hello world", undefined, stencil_pipeline)
main.draw(fire)
main.draw(fullscreen, undefined, stencil_reset)
main.draw(game)
main.end()