Introduce structs for StorageBufferattribute and StorageInstancedBufferAttribute

[Spiri0]

Related issue: #29760

Description

With this extension the storageBufferAttributes get structs. This means that the information about the data structuring is always directly accessable from the storageBufferAttributes themselves. This seems to me to be a more effective an natural solution than bypass codes, which would have to run through the code in parallel to transport the struct informations about the storageBuffers. This way seems also more robust to me.

But please don't merge yet. I would like to take this PR as an opportunity to include any suggestions for the final improvement. In addition, I haven't yet understood the pointer in the WGSLNodeBuilder sufficiently. Otherwise I would have expanded the WGSLNodeBuilder straight away, because the necessary WGSL code can be generated easily and with little effort using the struct information in the storageBufferAttribute. Type and atomic are optional. The default type is created automatically from itemSize and typeClass if nothing is specified. Based on the comparison of array.count and the length of the structs, the storageBufferAttribute also recognizes whether it is a single struct or a struct array. This is necessary to be able to build the correct wgsl code because of the "array" flag. If the ratio is not an integer, an error message appear. A future struct node, as the top user level, can later take over data creation and struct creation and thus the entries in the storageBufferAttribute and the storageBufferAttribute structs. But it also works very well without a struct node."

toDo: So I still have to learn to understand the pointer in WGSLNodeBuilder.

How to use:

//example from my code

const meshletInfo = this.getMeshletsInfo(this.meshlets);

this.meshletInfoStorage = new THREE.StorageBufferAttribute( meshletInfo , 4 );

this.meshletInfoStorage.name = "MeshletInfo";
this.meshletInfoStorage.setStructs([
   { name: 'cone_apex' },
   { name: 'cone_axis', type: 'vec3<f32>' },
   { name: 'cone_cutoff', type: 'f32' },
   { name: 'boundingSphere' },
   { name: 'parentBoundingSphere' },
   { name: 'error', type: 'f32' },
   { name: 'parentError' },
   { name: 'lod', type: 'f32' },
   { name: 'bboxMin' },
   { name: 'bboxMax' }        
]);

getMeshletsInfo(meshlets) {
   const meshletCount = meshlets.length;
   const countPerMeshlet = 40;
      
   const meshletInfoArray = new Float32Array(meshletCount * countPerMeshlet);
      
   for (let i = 0; i < meshletCount; i++) {
      const meshlet = meshlets[i];
      const offset = i * countPerMeshlet;
      
      const bv = meshlet.boundingVolume;
      const pbv = meshlet.boundingVolume;
      
      meshletInfoArray.set([
         0, 0, 0, 0,
         0, 0, 0, 0,
         0, 0, 0, 0,
         bv.center.x, bv.center.y, bv.center.z, bv.radius,
         pbv.center.x, pbv.center.y, pbv.center.z, pbv.radius,
         meshlet.clusterError, 0, 0, 0,
         meshlet.parentError, 0, 0, 0,
         meshlet.lod, 0, 0, 0,
         ...meshlet.bounds.min.toArray(), 0,
         ...meshlet.bounds.max.toArray(), 0
      ], offset);
   }

   return meshletInfoArray;
}

In my example getWGSLStructString produce:

struct MeshletInfo {
cone_apex: vec4<f32>,
cone_axis: vec3<f32>,
cone_cutoff: f32,
boundingSphere: vec4<f32>,
parentBoundingSphere: vec4<f32>,
error: f32,
parentError: vec4<f32>,
lod: f32,
bboxMin: vec4<f32>,
bboxMax: vec4<f32>,
};

I can use this string together with the other information from the storageBuffer to build the final WGSL code in the WGSLNodeBuilder, but as mentioned I still have to familiarize myself with the pointer

[github-actions]

📦 Bundle size

Full ESM build, minified and gzipped.

	Before	After	Diff
WebGL	692.22 171.45	692.22 171.45	+0 B +0 B
WebGPU	825.46 222.91	827.47 223.26	+2.01 kB +360 B
WebGPU Nodes	824.58 222.7	826.59 223.06	+2.01 kB +366 B

🌳 Bundle size after tree-shaking

Minimal build including a renderer, camera, empty scene, and dependencies.

	Before	After	Diff
WebGL	464.58 112.26	464.58 112.26	+0 B +0 B
WebGPU	545.63 147.78	545.63 147.78	+0 B +0 B
WebGPU Nodes	501.51 137.48	501.51 137.48	+0 B +0 B

[Spiri0]

Today I tried to extend the WGSLNodeBuilder.js because the pointer doesn't seem to play a role in the implementation.
@sunag @aardgoose @RenaudRohlinger the standard structure for the wgsl storage code snippet looks like this and this is how it works:

struct MeshletInfo {
	nodeUniform1 : array< vec4<f32> >

};
@binding( 2 ) @group( 0 )
var<storage, read> NodeBuffer_556 : MeshletInfo;

For structs we need the following layout in order to be able to use multiple values:

struct MeshletInfo {
	nodeUniform1 : vec4<f32> 

};
@binding( 2 ) @group( 0 )
var<storage, read> NodeBuffer_556 : array<MeshletInfo>;

But this currently leads to this error message:

127.0.0.1/:1 Error while parsing WGSL: :150:42 error: cannot index into expression of type 'array<MeshletInfo>'
  compute( &NodeBuffer_552.nodeUniform0, &NodeBuffer_556.nodeUniform1, &NodeBuffer_557.nodeUniform2, instanceIndex, object.nodeUniform3 );

could the reason for this lie in how the compute shader is currently generated?

_getWGSLComputeCode( shaderData, workgroupSize )

[sunag]

Complex snippets are always generated by NodeBuilder, i.e. WGSLNodeBuilder in this case.

[sunag]

this.meshletInfoStorage.setStructs([
{ name: 'cone_apex' },
{ name: 'cone_axis', type: 'vec3' },
{ name: 'cone_cutoff', type: 'f32' },
{ name: 'boundingSphere' },
{ name: 'parentBoundingSphere' },
{ name: 'error', type: 'f32' },
{ name: 'parentError' },
{ name: 'lod', type: 'f32' },
{ name: 'bboxMin' },
{ name: 'bboxMax' }
]);

The default type of TSL should always be defined similar to what we see in GLSL, instead of f32 in TSL it should be float.

I think that #29760 (comment) will have less code repetition, and we can use the structure for other purposes as well as uniform buffer.

const myStruct = struct({
    cone_axis: 'vec3', 
    cone_cutoff: 'float',
    error: 'float',
    ...
});

// buffer size calculated automatically.
const structBuffer = structArray( myStruct, count );

// buffer size calculated manually.
const structBuffer = storageStruct( buffer, myStruct, count );

[Spiri0]

According to your ideas. Wouldn't it be practical if the bufferAttributes carried the struct information (name, type, atomic) for each element?
But if you already have concrete ideas, I'll hold back.

[sunag]

I think if we had a Node to take care of the structs it would be better, the other variations could be extended from other nodes and not only attributes, the structure could be used in the return of a function, or in a uniform buffer or TSL array() for use in compute.

[Spiri0]

Then for special cases in which atomic is important, a case distinction would be needed between [key: value] and [key: {type: value, atomic: true}] This would be important here, for example:

//single struct
const myDrawStruct = ({
   vertexCount: { type: 'uint', atomic: true },
   instanceCount: { type: 'uint', atomic: true },
   firstVertex: 'uint',
   firstInstance: 'uint',
});

Now that I've looked into it a little more closely, the whole thing probably isn't as huge an effort as I thought. I thought I was almost there because I also generated the wgsl code in the WGSLNodeBuilder. You will certainly be able to do the node variant better than me, because I currently don't see how you imagine the connection between the data and the structArray in order to bring it to its destination in the WGSLNodeBuilder. But I'm excited because it's definitely going to be good. This PR is then obsolete. I will then close it later with a reference to yours

[Spiri0]

Ahhh, I think I understand you a lot better now. I need some time to understand the code better. We have storage and storageObject. Should storageObject already have been an attempt to implement structs? I ask because when I use storageObject I see in its parameters under node bufferType: "struct". I took a closer look at this when I created storageStruct. I would prefer the name storageStruct.

I then managed to get my test compute shader to run

const computeShader = wgslFn(`
	fn compute(
		meshletInfo: ptr<storage, array<MeshletInfo>, read>,
		index: u32
	) -> void {

		var meshlet = meshletInfo[10u];
		var boundingSphere = meshlet.boundingSphere;
		var coneApex = meshlet.cone_apex;
                var error = meshlet.error;
	}
`);

However, only because I deactivated the shaderData.flow in _getWGSLComputeCode( shaderData, workgroupSize ) because that leads to an error for structs. Of course that's not a solution because it's an essential part for everything else. If I misspell a name as a test, I get an error message. So the struct access seems to work properly.
However, I now have to learn to understand the shaderData.flow better.

[Spiri0]

@sunag I think I now largely understand how you imagine it. My test shader now also works according to your scheme. I'll work this out carefully now.

[sunag]

storageObject is only used in the WebGL2 fallback, but it's a bit confusing, I'm thinking of updating this method in this release.