diff --git a/doc/design.md b/doc/design.md index 679d2de..af40243 100644 --- a/doc/design.md +++ b/doc/design.md @@ -1,73 +1,459 @@ -## Graph +# AnimationGraph -### 1. Generalized data connections / Support of math nodes (or non-AnimNodes in general) +## Blend Trees -Use case: +A Blend Tree is a directed acyclic graph. Nodes produce or process "AnimationData" and the connections transport " +AnimationData". "AnimationData" here is anything that can be sampled in an Animation. + +### Example: + +```plantuml +@startuml + +left to right direction + +abstract Output {} +abstract Blend2 { +bool is_synced + +float weight() +} +abstract AnimationA {} +abstract TimeScale {} +abstract AnimationB {} + +AnimationA --> Blend2 +AnimationB --> TimeScale +TimeScale --> Blend2 +Blend2 --> Output + +@enduml +``` + +A Blend Tree always has a designated output node where the time delta is specified as an input and after processing of +the Blend Tree it emits the animation data. + +Some nodes have special names in the Blend Tree: + +* **Root node** The output node is also called the root node of the graph. +* **Leaf nodes** These are the nodes that have no inputs. In the example these are the nodes AnimationA and AnimationB. + +### Animation and Animation Data + +For Godot an Animation has multiple tracks where each Track is of a specific type such as "Position", "Rotation", " +Method Call", "Audio Playback", etc. Each Track is associated with a node path on which the value of a sampled Track +acts. + +Animation Data represents a sampled animation for a given time. For each track in an animation it contains the sampled +values, e.g. a Vector3 for a position, a Quaternion for a rotation, a function name and its parameters, etc. + +## State Machines + +```plantuml +@startuml + +State Idle +State Walk +State Run +State Fall + +[*] -right-> Idle +Idle -right-> Walk +Walk -left-> Idle +Walk -right-> Run +Run -left-> Walk +Walk -up-> Fall +'Idle -right-> Fall +Run -up-> Fall + +@enduml +``` + +## 1. Generalized data connections / Support of math nodes (or non-AnimationNodes in general) + +### Description + +The connections in the current AnimationTree only support animation values. Node inputs are specified as parameters of +the AnimationTree. E.g. it is not possible to do some math operation on the floating point value that ends up as blend +weight input of a Blend2 node. + +We use the term "value data" to distinguish from Animation Data. + +### Use case * Enables animators to add custom math for blend inputs or to adjust other inputs (e.g. LookAt or IK targets). +* Together with "Support of multiple output sockets" this can be used to add input parameters that affect multiple node + inputs. -Effects on the graph topology? +### Effects on the graph topology * Need to generalize Output Sockets to different types instead of only "Animation Data". * How to evaluate? Two types of subgraphs: - a) Instant inputs (needed for blend node inputs) that have to be evaluated before - UpdateConnections. Maybe restrict to data that is not animation data dependent? - b) Processing nodes, e.g. for extracted bones. + * a) Data/value inputs (e.g. for blend weights) that have to be evaluated before UpdateConnections. Maybe restrict + to data that is not animation data dependent? + * b) Processing nodes, e.g. for extracted bones. -### 2. Support of multiple output sockets +## 2. Support of multiple output sockets -Use case: +### Description + +Current AnimationTree nodes have a single designated output socket. A node cannot extract a value that then gets used as +input at a laters tage in the graph. + +**Depends on**: "Generalized data connections". + +### Use case * E.g. extract Bone transform and use in pose modifying nodes. * Chain IK. -Effects on graph topology? +### Effects on graph topology * Increases Node complexity: * AnimOutput * AnimOutput + Data * Data + (Data = bool, float, vec3, quat or ...) +* Can a BlendTree emit values? + * If so: what happens with the output if the BlendTree is used in a State Machine? + * => Initially: State Machines only emit Animation Data. +* Simplest case: + * All value data connections are evaluated always before UpdateConnections. + * BlendTrees (and therefore embedded graphs) cannot emit values. -(Data = bool, float, vec3, quat or ...) - -**Open Issues** +### Open Issues 1. Unclear when this is actually needed. Using more specific nodes that perform the desired logic may be better ( c.f. https://dev.epicgames.com/documentation/en-us/unreal-engine/animation-blueprint-bone-driven-controller-in-unreal-engine). Likely this is not crucial so should be avoided for now. -### 3. Multi-skeleton evaluation +## 3. Multi-skeleton evaluation -Use case: riding on a horse, interaction between two characters. +### Description -### 4. Output re-use +Allow an animation graph to affect multiple skeletons. -Description: +### Use case -Output of a single node may be used as input of two or more other nodes. +Riding on a horse, interaction between two characters. -Use case: +## 4. Output re-use -* (Related to 1.) an input that does some computation gets reused in two separate subtrees. +### Description -### 5. Inputs into Subgraphs +Output of a single node may be used as input of two or more other nodes. One has to consider re-use of Animation Data +and general value data (see "Generalized data connections") separately. -Description: +#### Animation Data Output re-use -An embedded blend tree can receive inputs of the surrounding blend tree. Inputs are animations or - depending on 1. - also general input values. +Animation Data connections are generally more heavy weight than "Generalized data connections". The latter only contain +a small data type such as a Vector3 or Quaternion, whereas Animation Data is a container of those (and possibly more, +e.g. function names and their parameters). So it is desirable to have a minimal number of Animation Data objects +allocated. -Use case: +```plantuml +@startuml + +left to right direction + +abstract Output {} +abstract Blend2 { +bool is_synced + +float weight() +} + +abstract AnimationA {} +abstract TwoBoneIK {} +abstract BoneExtractor {} +abstract AnimationB {} + +AnimationA --> Blend2 +AnimationB --> Blend2 +AnimationB --> BoneExtractor +BoneExtractor --> TwoBoneIK +Blend2 --> TwoBoneIK +TwoBoneIK --> Output + +@enduml +``` + +In this case the Animation Data output of AnimationB is used both in the Blend2 node and the BoneExtractor node. The +extracted bone information is passed into the TwoBoneIK node. For this to work we have to ensure that the output of +AnimationB is freed only when both the Blend2 node and the BoneExtractor have finished their work. + +An alternative layout with the same nodes would not suffer from this: + +```plantuml +@startuml + +left to right direction + +abstract Output {} +abstract Blend2 { +bool is_synced + +float weight() +} + +abstract AnimationA {} +abstract TwoBoneIK {} +abstract BoneExtractor {} +abstract AnimationB {} + +AnimationA --> Blend2 +BoneExtractor --> Blend2 +AnimationB --> BoneExtractor +BoneExtractor --> TwoBoneIK +Blend2 --> TwoBoneIK +TwoBoneIK --> Output + +@enduml +``` + +Here the AnimationData output of BoneExtractor is used by Blend2 and the extracted bone information is still passed to +the TwoBoneIK. + +A more complex case is the following setup where Animation Data is used by two nodes: + +```plantuml +@startuml + +left to right direction + +abstract Output {} +abstract Blend2_UpperBody { +bool is_synced + +float weight() +} + +abstract Blend2_LowerBody { +bool is_synced + +float weight() +} + + +abstract AnimationA {} +abstract AnimationB {} + +AnimationA --> Blend2_UpperBody +AnimationB --> Blend2_UpperBody +AnimationB --> Blend2_LowerBody +Blend2_UpperBody --> Blend2_LowerBody + +Blend2_LowerBody --> Output + +@enduml +``` + +Here the output of AnimationB is used in both Blend2_UpperBody and Blend2_LowerBody. However, if both blends are synced +the Blend2_UpperBody and Blend2_LowerBody nodes may compute different time values as their SyncTracks differ. Generally +a connection is invalid if the Animation Data of a node gets combined with itself via a different path in the tree. + +#### Data value re-use + +```plantuml +@startuml + +left to right direction + +abstract Output {} +abstract Blend2 { +bool is_synced + +float weight() +} + +abstract AnimationA {} +abstract Vector3Input {} +abstract UnpackVector3 { +x +y +z +} + +AnimationA --> Blend2 +Vector3Input --> UnpackVector3 +UnpackVector3 --> Blend2 +Blend2 --> Output + +@enduml +``` + +Here the UnpackVector3 provides access to the individual 3 components of a Vector3 such that e.g. the x value can be +used as an input to a weight node. + +Data value re-use is not a problem as the values would be stored directly in the nodes and are not allocated/deallocated +when a node becomes active/deactivated. + +### Use case + +* Depends on "Generalized data connections" an input that does some computation gets reused in two separate subtrees. + +## 5. Inputs into Subgraphs + +### Description + +An embedded blend tree can receive inputs of the surrounding blend tree. Inputs are animations or - depending on 1. - +also general input values. + +### Use case * Reuse of some blend logic or to move logic of a part of a blend tree into its own node. -Effects on graph topology: +### Effects on graph topology * Great flexibility and possibly reusability. * Improves logical block building. * Probably only of bigger use with 1. -Open issues: +### Open issues * Inputs to embedded state machines? + +## 6. Evaluation API + +### Description + +Evaluation of a node happens in multiple phases: + +1. UpdateConnections(): right to left (i.e. from the root node via depth first to the leave nodes) +2. CalculateSyncTracks(): left to right (leave nodes to root node) +3. UpdateTime(): right to left +4. Evaluate(): left to right + +One question here is how to transport the actual data from one node to another. There are essentially two options: + +#### Data owned by connections + +#### Explicit input node references + +Nodes store references or pointers to all input nodes. + +``` C++ +void Node::evaluate(AnimationData& output) { + AnimationData input_node_0_data; + input_node_0->evaluate(input_node_0_data); + + AnimationData input_node_1_data; + input_node_1->evaluate(input_node_1_data); + + output = lerp(input_node_0_data, input_node_1_data, blend_weight); +} +``` + +* [-] Makes Blend Tree evaluation recursive. + +#### Indirect input node references + +```c++ +// BlendTree.cpp +void BlendTree::initialize_tree() { + for (int i = 0; ci < num_connections; i++) { + const Connection& connection = connections[i]; + connection.target_node->set_input_node(connection.target_socket_name, connection.source_node); + } +} + +void BlendTree::activate_inputs() { + for (int i = 0; i < num_nodes; i++) { + if (nodes[i].is_active()) { + nodes[i].activate_inputs() + } + } +} + +void Blend2Node::activate_inputs() { + if (weight < EPS) { + input_node_0->set_active(false); + } else { + input_node_0->set_active(true); + } + + if (weight > 1.0 - EPS) { + input_node_1->set_active(false); + } else { + input_node_1->set_active(true); + } +} + +void BlendTree::calculate_sync_tracks() { + for (int i = num_nodes; i > 0; i--) { + if (nodes[i]->is_active()) { + nodes[i]->calculate_sync_track(); + } + } +} + +void BlendTree::propagate_time() { + for (int i = 1; i < num_nodes; i++) { + if (nodes[i]->is_active()) { + nodes[i]->update_time(); + } + } +} + +void BlendTree::evaluate(AnimationData& output) { + for (int i = num_nodes; i > 0; i--) { + if (nodes[i]->is_active()) { + nodes[i]->output = AnimationDataPool::allocate(); + nodes[i]->evaluate(); + + // node[i] is done, so we can deallocate the output handles of all input nodes of node[i]. + for (AnimationGraphnNode& input_nodes: input_nodes[i]) { + AnimationDataPool::deallocate(nodes[i].output); + } + } + + nodes[i]->set_active(false); + } + + output = nodes[0].output; + + // free output buffers + for (int i = 1; i < num_nodes; i++) { + AnimationDataPool::deallocate(nodes[i].output); + } +} + +void Blend2Node::evaluate(AnimationData& output) { + output = lerp(input_node_0->get_output(), input_node_1_data->get_output(), blend_weight); +} + +void TimeScaleNode::evaluate(AnimationData& output) { + std::swap(output, input_node_0->output); +} +``` + +```c++ +// Node.cpp +void Node::evaluate(AnimationData& output) { + output = lerp(input_node_0->get_output(), input_node_1_data->get_output(), blend_weight); +} +``` + +#### Data injected by Blend Tree + +Nodes store references or pointers to all input nodes. + +```c++ +void Node::evaluate(const Array& animation_inputs, const Array& data_inputs>, AnimationData& output) { + output = lerp(animation_inputs[0], animation_inputs[1], data_inputs[0]); +} +``` + +* [+] This would allow easy extension of animation nodes via GDScript or GDExtension based nodes. + * [-] Though this could maybe be achieved using a specific customizable node for other approaches. +* [-] Easy to mess up indices. +* [-] Type safety of data_inputs messy. + +## Glossary + +### Animation Data + +Output of an AnimationGraphNode. Contains everything that can be sampled from an animation such as positions, rotations +but also method calls including function name and arguments. diff --git a/doc/design.puml b/doc/design.puml new file mode 100644 index 0000000..4d2094c --- /dev/null +++ b/doc/design.puml @@ -0,0 +1,79 @@ +@startuml + +class Node { +} + +class AnimationGraph { + AnimationGraphNode root_node + NodePath animation_player + + void _notification(int p_what) + static void _bind_methods() +} + +class AnimationGraphNode { + NodeTimeInfo time_info + SyncTrack sync_track + + virtual void initialize(GraphEvaluationContext &context) + virtual void activate_inputs(GraphEvaluationContext &context) + virtual void calculate_sync_track(GraphEvaluationContext &context) + virtual void update_time(double p_delta) + virtual void evaluate(GraphEvaluationContext &context, AnimationData &data) + + virtual int get_num_inputs() { return 0; } + virtual AnimationGraphNode* get_input_node(int input_index) { return nullptr; } +} + +class AnimationGraphBlendTree { + // nodes are ordered by evaluation + Array nodes + Array connections + + void initialize_tree() + void activate_nodes() + void calculate_sync_tracks() + void propagate_time() + void evaluate_nodes() +} + +class AnimationGraphStateMachine { + Array states + Array transitions + + AnimationGraphNode *current_state + AnimationGraphNode *previous_state + + Transition* active_transition + double transition_time + + void initialize_states() + void activate_transitions() + void calculate_sync_tracks() + void calculate_state_times() + void evaluate_nodes_and_output() +} + +class AnimationGraphBlend2Node { + AnimationGraphNode input_0 + AnimationGraphNode input_1 + double blend_weight +} + +class AnimationGraph2BoneIK { + AnimationGraphNode input + double blend_weight + + Vector3 pole_target + StringName bone_name +} + +Node <|-- AnimationGraph +AnimationGraph *-- AnimationGraphNode + +AnimationGraphNode <|-- AnimationGraphBlendTree +AnimationGraphNode <|-- AnimationGraphStateMachine +AnimationGraphNode <|-- AnimationGraphBlend2Node +AnimationGraphNode <|-- AnimationGraph2BoneIK + +@enduml \ No newline at end of file diff --git a/synced_animation_graph.h b/synced_animation_graph.h index d38841f..973e170 100644 --- a/synced_animation_graph.h +++ b/synced_animation_graph.h @@ -79,7 +79,7 @@ struct AnimationData { AnimationData() = default; ~AnimationData() { _clear_values(); - }; + } void set_value(Animation::TypeHash thash, TrackValue *value) { if (!track_values.has(thash)) { @@ -101,7 +101,6 @@ protected: memdelete(K.value); } } - }; struct GraphEvaluationContext { @@ -110,7 +109,13 @@ struct GraphEvaluationContext { Skeleton3D *skeleton_3d = nullptr; }; +struct SyncTrack { + +}; + class SyncedAnimationNode { + friend class SyncedAnimationGraph; + public: struct NodeTimeInfo { double length = 0.0; @@ -118,12 +123,13 @@ public: double delta = 0.0; Animation::LoopMode loop_mode = Animation::LOOP_NONE; + SyncTrack sync_track; }; NodeTimeInfo node_time_info; virtual ~SyncedAnimationNode() = default; virtual void initialize(GraphEvaluationContext &context) {} - virtual void activate_inputs(Vector input_names) {} + virtual void activate_inputs(GraphEvaluationContext &context, Vector input_names) {} virtual void calculate_sync_track() {} virtual void update_time(double p_delta) { node_time_info.position += p_delta; @@ -147,7 +153,15 @@ public: } } } - virtual void evaluate(GraphEvaluationContext &context, AnimationData &output) {} + virtual void evaluate(GraphEvaluationContext &context) {} + + bool is_active() const { return active; } + bool set_input_node(const StringName &socket_name, SyncedAnimationNode *node); + void get_input_names(Array &inputs); + +private: + AnimationData *output = nullptr; + bool active = false; }; class AnimationSamplerNode : public SyncedAnimationNode {