Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KHR_implicit_shapes extension Draft Proposal #2370

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open
151 changes: 151 additions & 0 deletions extensions/2.0/Khronos/KHR_collision_shapes/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
# KHR_collision_shapes

## Contributors

* Eoin Mcloughlin, Microsoft, <mailto:[email protected]>
* Rory Mullane, Microsoft, <mailto:[email protected]>
* George Tian, Microsoft, <mailto:[email protected]>
* Aaron Franke, Godot Engine, <mailto:[email protected]>

## Status

Draft

## Dependencies

Written against glTF 2.0 spec.

## Overview

This extension adds the ability to specify collision primitives inside a glTF asset. This extension does not mandate any particular behaviour for those objects aside from their collision geometry. These types are to be used in combination with other extensions that reference these collision primitives.

## glTF Schema Updates

### Shapes

This extension provides a set of document-level objects, which can be referenced by objects in the scene. The precise usage of these primitives should be specified by the extensions which utilize the shapes. The intended purpose of these these objects are to specify geometry which can be used for collision detection, which has informed the set of shapes which we have defined. Typically, the geometry specified by the shape will be simpler than any render meshes used by the node or its children, enabling real-time applications to perform queries such as intersection tests.

Shapes are defined within a dictionary property in the glTF scene description file, by adding an `extensions` property to the top-level glTF 2.0 object and defining a `KHR_collision_shapes` property with a `shapes` array inside it.

Each shape defines a mandatory `type` property which designates the type of shape, as well as an additional structure which provides parameterizations specific to that type. The following example defines a sphere.

```javascript
"extensions": {
"KHR_collision_shapes" : {
"shapes": [
{
"sphere": { "radius": 0.1 },
"type": "sphere"
}
]
}
}
```

Shapes are parameterized in local space of the node they are associated with. If a shape is required to have an offset from the local space of the node the shape is associated with (for example a sphere _not_ centered at local origin or a rotated box,) a child node should be added with the desired offset applied, and the shape properties added to that child.

To describe the geometry which represents the object, shapes must define at most one of the following properties:

| |Type|Description|
|-|-|-|
|**sphere**|`object`|A sphere centered at the origin in local space.|
|**box**|`object`|An axis-aligned box centered at the origin in local space.|
|**cylinder**|`object`|A cylinder centered at the origin and aligned along the Y axis in local space, with potentially different radii at each end.|
|**capsule**|`object`|A capsule (cylinder with hemispherical ends) centered at the origin and defined by two "capping" spheres with potentially different radii, aligned along the Y axis in local space.|
|**mesh**|`object`|A shape generated from a `mesh` object.|

The sphere, box, capsule, and cylinder all represent convex objects with a volume. However, the mesh type presents two options; when the `convexHull` property is `false`, the shape represents the surface of a referenced mesh. When `convexHull` is `true`, the shape represents the convex hull of a referenced mesh. The input mesh to a convex hull is not required to be convex itself, nor is there any requirement for the geometry to be closed. An implementation must generate a convex hull from the input mesh.

As the mesh shape references a `mesh`, it additionally allows for optional `skin` and `weights` properties, which have the same semantics and requirements enforced by the properties of the same name associated with a `node`. When specified on a mesh whose `convexHull` property is `true`, the resulting collision shape should be the convex hull of the deformed mesh. As collision detection is typically performed on CPU, the performance impact of deforming a mesh in such a use-case is typically higher than inside a vertex shader. As such, use of this functionality should be given careful consideration with respect to performance.

Degenerate shapes are prohibited. A sphere must have a positive, non-zero radius. A box shape must have positive non-zero values for each component of `size`. The cylinder and capsule shapes must have a positive, non-zero `height` and both `radiusTop` and `radiusBottom` should be positive; at least one of the radii should be non-zero. For mesh shapes whose `convexHull` property is `false`, the referenced mesh must contain at least one non-degenerate triangle primitive. For mesh shapes whose `convexHull` property is `true`, the referenced mesh must contain contain primitives with at least four non-coplanar vertices.

A nodes scale presents some special cases that must be handled. In accordance with the glTF specification on [https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#transformations](transformations), a shape referenced by a node whose scale is zero on all three axes should be considered disabled. When an analytical shape is referenced by a node whose whose scale is negative on one or more axes, the resulting shape size should be the absolute value of the nodes scale applied to the input shape parameters. i.e. a box with `size` of `[1.0, 1.0, 1.0]` associated with a node whose scale is `[1.0, -2.0, 3.0]` should result in a box of size `[1.0, 2.0, 3.0]` in world space.

### JSON Schema

* **JSON schema**: [glTF.KHR_collision_shapes.schema.json](schema/glTF.KHR_collision_shapes.schema.json)

### Object Model

With consideration to the glTF 2.0 Asset Object Model Specification document, the following pointer templates represent mutable properties defined in this extension.

| Pointer | Type|
|-|-|
| `/extensions/KHR_collision_shapes/shapes/{}/box/size` | `float3`|
| `/extensions/KHR_collision_shapes/shapes/{}/capsule/height` | `float`|
| `/extensions/KHR_collision_shapes/shapes/{}/capsule/radiusBottom` | `float`|
| `/extensions/KHR_collision_shapes/shapes/{}/capsule/radiusTop` | `float`|
| `/extensions/KHR_collision_shapes/shapes/{}/cylinder/height` | `float`|
| `/extensions/KHR_collision_shapes/shapes/{}/cylinder/radiusBottom` | `float`|
| `/extensions/KHR_collision_shapes/shapes/{}/cylinder/radiusTop` | `float`|
| `/extensions/KHR_collision_shapes/shapes/{}/sphere/radius` | `float`|
| `/extensions/KHR_collision_shapes/shapes/{}/mesh/weights/{}` | `float`|
| `/extensions/KHR_collision_shapes/shapes/{}/mesh/weights` | `float[]`|

Additional read-only properties

| Pointer | Type|
|-|-|
| `/extensions/KHR_collision_shapes/shapes.length` | `int`|
lexaknyazev marked this conversation as resolved.
Show resolved Hide resolved
| `/extensions/KHR_collision_shapes/shapes/{}/mesh/weights.length` | `int`|
| `/extensions/KHR_collision_shapes/shapes/{}/mesh/convexHull` | `boolean`|

## Known Implementations

[Blender importer/exporter](https://github.com/eoineoineoin/glTF_Physics_Blender_Exporter)

[Babylon.js importer](https://github.com/eoineoineoin/glTF_Physics_Babylon)

[Godot importer](https://github.com/eoineoineoin/glTF_Physics_Godot_Importer)

## Appendix: Full Khronos Copyright Statement

Copyright 2021-2023 The Khronos Group Inc.

This specification is protected by copyright laws and contains material proprietary
to Khronos. Except as described by these terms, it or any components
may not be reproduced, republished, distributed, transmitted, displayed, broadcast,
or otherwise exploited in any manner without the express prior written permission
of Khronos.

This specification has been created under the Khronos Intellectual Property Rights
Policy, which is Attachment A of the Khronos Group Membership Agreement available at
https://www.khronos.org/files/member_agreement.pdf. Khronos grants a conditional
copyright license to use and reproduce the unmodified specification for any purpose,
without fee or royalty, EXCEPT no licenses to any patent, trademark or other
intellectual property rights are granted under these terms. Parties desiring to
implement the specification and make use of Khronos trademarks in relation to that
implementation, and receive reciprocal patent license protection under the Khronos
IP Policy must become Adopters under the process defined by Khronos for this specification;
see https://www.khronos.org/conformance/adopters/file-format-adopter-program.

Some parts of this Specification are purely informative and do not define requirements
necessary for compliance and so are outside the Scope of this Specification. These
parts of the Specification are marked as being non-normative, or identified as
**Implementation Notes**.

Where this Specification includes normative references to external documents, only the
specifically identified sections and functionality of those external documents are in
Scope. Requirements defined by external documents not created by Khronos may contain
contributions from non-members of Khronos not covered by the Khronos Intellectual
Property Rights Policy.

Khronos makes no, and expressly disclaims any, representations or warranties,
express or implied, regarding this specification, including, without limitation:
merchantability, fitness for a particular purpose, non-infringement of any
intellectual property, correctness, accuracy, completeness, timeliness, and
reliability. Under no circumstances will Khronos, or any of its Promoters,
Contributors or Members, or their respective partners, officers, directors,
employees, agents or representatives be liable for any damages, whether direct,
indirect, special or consequential damages for lost revenues, lost profits, or
otherwise, arising from or in connection with these materials.

Khronos® and Vulkan® are registered trademarks, and ANARI™, WebGL™, glTF™, NNEF™, OpenVX™,
SPIR™, SPIR&#8209;V™, SYCL™, OpenVG™ and 3D Commerce™ are trademarks of The Khronos Group Inc.
OpenXR™ is a trademark owned by The Khronos Group Inc. and is registered as a trademark in
China, the European Union, Japan and the United Kingdom. OpenCL™ is a trademark of Apple Inc.
and OpenGL® is a registered trademark and the OpenGL ES™ and OpenGL SC™ logos are trademarks
of Hewlett Packard Enterprise used under license by Khronos. ASTC is a trademark of
ARM Holdings PLC. All other product names, trademarks, and/or company names are used solely
for identification and belong to their respective owners.
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"$schema": "http://json-schema.org/draft-04/schema",
"title": "KHR_collision_shapes glTF Document Extension",
"type": "object",
"description": "Top level collision primitives.",
"allOf": [ { "$ref" : "glTFProperty.schema.json" } ],
"properties": {
"shapes": {
"type": "array",
"description": "An array of shape descriptions.",
"items": {
"type": "object",
"$ref": "glTF.KHR_collision_shapes.shape.schema.json"
},
"minItems": 1
},
"extensions": { },
"extras": { }
},
"required": [
"shapes"
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"$schema": "http://json-schema.org/draft-04/schema",
"title": "KHR_collision_shapes Box Shape",
"type": "object",
"description": "Parameters describing a box shape.",
"allOf": [ { "$ref": "glTFProperty.schema.json" } ],
"properties": {
"size": {
"type": "array",
"description": "The extents of the box in each axis in local space.",
"items": {
"type": "number",
"exclusiveMinimum": 0.0
},
"minItems": 3,
"maxItems": 3,
"default": [ 1, 1, 1 ]
},
"extensions": { },
"extras": { }
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{
"$schema": "http://json-schema.org/draft-04/schema",
"title": "KHR_collision_shapes Capsule Shape",
"type": "object",
"description": "Parameters describing a capsule shape.",
"allOf": [ { "$ref": "glTFProperty.schema.json" } ],
"properties": {
"height": {
"type": "number",
"description": "The distance between the centers of the two capping spheres of capsule.",
"exclusiveMinimum": 0.0,
"default": 0.5
},
"radiusBottom": {
"type": "number",
"description": "The radius of the sphere located at the bottom of the capsule (i.e. the sphere at the half-height along -Y)",
"minimum": 0.0,
"default": 0.25
},
"radiusTop": {
"type": "number",
"description": "The radius of the sphere located at the top of the capsule (i.e. the sphere at the half-height along +Y)",
"minimum": 0.0,
"default": 0.25
},
"extensions": { },
"extras": { }
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{
"$schema": "http://json-schema.org/draft-04/schema",
"title": "KHR_collision_shapes Cylinder Shape",
"type": "object",
"description": "Parameters describing a cylinder shape.",
"allOf": [ { "$ref": "glTFProperty.schema.json" } ],
"properties": {
"height": {
"type": "number",
"description": "The height of the cylinder, centered along the Y axis.",
"exclusiveMinimum": 0.0,
"default": 0.5
},
"radiusBottom": {
"type": "number",
"description": "The radius of the bottom of the cylinder (the disk located along -Y.)",
"minimum": 0.0,
"default": 0.25
},
"radiusTop": {
"type": "number",
"description": "The radius of the top of the cylinder (the disk located along +Y.)",
"minimum": 0.0,
"default": 0.25
},
"extensions": { },
"extras": { }
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"$schema": "http://json-schema.org/draft-04/schema",
"title": "KHR_collision_shapes Mesh Shape",
"type": "object",
"description": "Parameters describing a mesh shape.",
"allOf": [ { "$ref": "glTFProperty.schema.json" } ],
"properties": {
"convexHull": {
"type": "boolean",
"description": "Specifies whether the collision geometry is a surface or a convex hull.",
"gltf_detailedDescription": "Specifies how the collision shape is generated from the referenced mesh. When this value is false, the collision volume of this shape is the surface of the referenced mesh. The primitives in the mesh **MUST** contain triangles. When this value is true, this shape represents the convex hull of the set of primitives in the referenced mesh.",
"default": false
},
"mesh": {
"description": "The index of the mesh from which to build the collision representation.",
"allOf": [{"$ref": "glTFid.schema.json"}]
},
"skin": {
"description": "The index of a skin with which to deform the mesh.",
"allOf": [ { "$ref": "glTFid.schema.json" } ]
},
"weights": {
"type": "array",
"description": "The weights of the instantiated morph target.",
"minItems": 1,
"items": {
"type": "number"
}
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused, why are skins and weights used for collision meshes? Where was the discussion on this? What is the use case?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was discussed in several Khronos meetings on the interactivity spec. These parameters match functionality already in the glTF node. The usecase is to allow users to describe collision shapes which accurately match the render meshes.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

collision shapes which accurately match the render meshes

I could see weights/skins being calculated once at load time, but since "interactivity" is brought up, it sounds like the intent is to permit changing vertices on any given frame. I cannot find a good specification for the intended behavior when vertices of a collision mesh are animated at runtime (e.g. on every frame). I see two big problems:

  1. Firstly, it seems highly unperformant: both the need to regenerate optimized collision geometry (such as a convex hull or deduplicated trimesh vertices); and to run skinning on the CPU (some engines upload skinned mesh geometry to the GPU and do not store a copy on the CPU, so this may involve data transfer from the GPU on each frame). In the case of convex hulls, it might not be feasible to re-build the hull every frame in which it may have been animated.

  2. In a physics sense, it seems to me problematic that the skinned mesh deformation is non-physical: if the change in shape causes a collision, does it exert a force? if so, what force? if not, will it be allowed to intersect with adjacent geometry from one frame to the next, or entirely phase through it?

As a concrete example, imagine a razor-thin mesh floor of an elevator, using a skin joint to move upwards, and a small marble resting on the floor of this elevator. Imagine an implementation performs CPU deformation by calling BakeMesh() every frame, effectively swapping out one mesh for another. If the joint node moves quickly enough, the floor of the elevator may move more in one physics frame than the radius of the marble, causing it to fall. This will be highly dependent on physics framerate, which could differ either due to hardware specs or by user agent.

Ultimately, I believe the likely outcome is the majority of conventional game engines will choose between ignoring support for skins/morphs for physics shapes (which means inconsistent support), or doing something unperformant (for example, calling BakeMesh() every frame). This sort of feature sounds cool, but it's a pie in the sky, and the likelihood of inconsistent support could scare users away from using the feature, which tells me that it belongs in a separate extension.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that mesh objects already have weights defined in the core glTF specification. Those weights may be animated, so this needs to be supported anyway. These properties merely the shapes consistent with how nodes can apply per-instance weights to a mesh, as well as using a skinned mesh.

As with many glTF features, it's possible to describe scenarios which are non-physical or may perform poorly, but at the end of the day, glTF content is made by a human who is ultimately making decisions about what that content is. The fact that a feature may be misused is not a good reason to say "we don't support this feature which is in the core spec."

"extensions": { },
"extras": { }
},
"required": [
"mesh"
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
{
"$schema": "http://json-schema.org/draft-04/schema",
"title": "KHR_collision_shapes Shape Resource",
"type": "object",
"description": "Parameters describing a node's physics collision geometry.",
"allOf": [ { "$ref": "glTFChildOfRootProperty.schema.json" } ],
"properties": {
"type": {
"description": "Specifies the shape type.",
"anyOf": [
{
"enum": [ "sphere" ],
"description": "A sphere with a specified radius, centered at the origin in local space."
},
{
"enum": [ "box" ],
"description": "An axis-aligned box with a size per-axis, centered at the origin in local space"
},
{
"enum": [ "capsule" ],
"description": "A capsule shape, centered at the origin in local space, equivalent to the convex hull of two spheres located along the Y axis (in local space) at a specified distance."
},
{
"enum": [ "cylinder" ],
"description": "A cylinder shape, centered at the origin in local space, equivalent to the convex hull of two circles in the X/Z plane positioned along the Y axis at a specified distance."
},
{
"enum": [ "mesh" ],
"description": "A shape generated from a referenced `mesh` object"
},
{
"type": "string"
}
]
},
lexaknyazev marked this conversation as resolved.
Show resolved Hide resolved
"sphere": {
"type": "object",
"description": "A set of parameter values that are used to define a sphere shape.",
"$ref": "glTF.KHR_collision_shapes.shape.sphere.schema.json"
},
"box": {
"type": "object",
"description": "A set of parameter values that are used to define a box shape.",
"$ref": "glTF.KHR_collision_shapes.shape.box.schema.json"
},
"capsule": {
"type": "object",
"description": "A set of parameter values that are used to define a capsule shape.",
"$ref": "glTF.KHR_collision_shapes.shape.capsule.schema.json"
},
"cylinder": {
"type": "object",
"description": "A set of parameter values that are used to define a cylinder shape.",
"$ref": "glTF.KHR_collision_shapes.shape.cylinder.schema.json"
},
"mesh": {
"type": "object",
"description": "A set of parameter values that are used to define a shape derived from a mesh.",
"$ref": "glTF.KHR_collision_shapes.shape.mesh.schema.json"
},
"name": { },
"extensions": { },
"extras": { }
},
"oneOf": [
{ "required": ["sphere"] },
{ "required": ["box"] },
{ "required": ["capsule"] },
{ "required": ["cylinder"] },
{ "required": ["mesh"] }
],
"required": [
"type"
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"$schema": "http://json-schema.org/draft-04/schema",
"title": "KHR_collision_shapes Sphere Shape",
"type": "object",
"description": "Parameters describing a sphere shape.",
"allOf": [ { "$ref": "glTFProperty.schema.json" } ],
"properties": {
"radius": {
"type": "number",
"description": "The radius of the sphere.",
"exclusiveMinimum": 0.0,
"default": 0.5
},
"extensions": { },
"extras": { }
}
}
Loading