Skip to main content

External Signing Hashing Algorithm

Introduction

This document specifies the encoding algorithm used to produce a deterministic hash of a com.daml.ledger.api.v2.interactive.PreparedTransaction. The resulting hash is signed by the holder of the external party’s private key. The signature authorizes the ledger changes described by the transaction on behalf of the external party. The specification can be implemented in any language, but certain encoding patterns are biased due to Canton being implemented in a JVM-based language and using the Java protobuf library. Those biases are made explicit in the specification. Protobuf serialization is unsuitable for signing cryptographic hashes because it is not canonical. We must define a more precise encoding specification that can be re-implemented deterministically across languages and provide the required cryptographic guarantees. See https://protobuf.dev/programming-guides/serialization-not-canonical/ for more information on the topic.

Versioning

Hashing Scheme Version

The hashing algorithm as a whole is versioned. This enables updates to accommodate changes in the underlying Daml format, or, for instance, to the way the protocol verifies signatures. The implementation must respect the specification of the version it implements.
The hashing algorithm is tied to the protocol version of the synchronizer used to synchronize the transaction. Specifically, each hashing scheme version is supported on one or several protocol versions. Implementations must use a hashing scheme version supported on the synchronizer on which the transaction is submitted.

Transaction Nodes

Transaction nodes are additionally individually versioned with a Daml version (also called LF version). The encoding version is decoupled from the LF version and implementations should only focus on the hashing version. However, new LF versions may introduce new fields in nodes or new node types. For that reason, the protobuf representation of a node is versioned to accommodate those future changes. In practice, every new Daml language version results in a new hashing version.

V3

General approach

The hash of the PreparedTransaction is computed by encoding every protobuf field of the messages to byte arrays, and feeding those encoded values into a SHA-256 hash builder. The rest of this section details how to deterministically encode every proto message into a byte array. Sometimes during the process, partially encoded results are hashed with SHA-256, and the resulting hash value serves as the encoding in messages further up. This is explicit when necessary. Big Endian notation is used for numeric values. Furthermore, protobuf numeric values are encoded according to their Java type representation. Refer to the official protobuf documentation for more information about protobuf to Java type mappings: https://protobuf.dev/programming-guides/proto3/#scalar In particular:
In Java, unsigned 32-bit and 64-bit integers are represented using their signed counterparts, with the top bit simply being stored in the sign bit
Additionally, this is the java library used under the hood in Canton to serialize and deserialize protobuf: https://github.com/protocolbuffers/protobuf/tree/v3.25.5/java

Changes from V2

  • Addition of an max_record_time field in metadata to make maximum record time explicit in the signed metadata.

Changes from V1

  • Addition of an interface_id field in Fetch nodes for support of Daml interfaces.
  • Addition of the hashing scheme version in the final hash to make the hash more robust to cross version collisions.
  • Replace ledger_effective_time in the metadata with min_ledger_effective_time and max_ledger_effective_time.
    • These effectively replace a fixed ledger time with time bounds, allowing Daml Models to make assertions based on time without restricting the signing window as was required with a fixed set ledger time.
V3 introduces support for contract keys. Usage of contract keys in externally signed transactions requires usage of V3. Contract keys will not work on V2. Also note that V3 is only supported on protocol version 35.

Notation and Utility Functions

  • encode: Function that takes a protobuf message or primitive type T and transforms it into an array of bytes: encode: T => byte[]
e.g:
  • to_utf_8: Function converting a Java String to its UTF-8 encoded version: to_utf_8: string => byte[]
e.g:
  • len: Function returning the size of a collection (array, list etc…) as a signed 4 bytes integer: len: Col => Int
e.g:
  • split: Function converting a Java String to a list of String, by splitting the input using the provided delimiter: split: (string, char) => byte[]
e.g:
  • ||: Symbol representing concatenation of byte arrays
e.g:
  • []: Empty byte array. Denotes that the value should not be encoded.
  • from_hex_string: Function that takes a string in the hexadecimal format as input and decodes it as a byte array: from_hex_string: string => byte[]
e.g:
  • int_to_string: Function that takes an int and converts it to a string : int_to_string: int => string
e.g:
  • some: Value wrapped in a defined optional. Should be encoded as a defined optional value: some: T => optional T
e.g:
See encoding of optional values below for details.

Primitive Types

Unless otherwise specified, this is how primitive protobuf types should be encoded.
Not all protobuf types are described here, only the ones necessary to encode a PreparedTransaction message.
Even default values must be included in the encoding. For instance if an int32 field is not set in the serialized protobuf, its default value (0) should be encoded. Similarly, an empty repeated field still results in a 0x00 byte encoding (see the repeated section below for more details)

google.protobuf.Empty

bool

int64 - uint64 - sint64 - sfixed64

e.g:

int32 - uint32 - sint32 - sfixed32

e.g:

bytes / byte[]

e.g

string

e.g

Collections / Wrappers

repeated

repeated protobuf fields represent an ordered collection of values of a specific message of type T. It is critical that the order of values in the list is not modified, both for the encoding process and in the protobuf itself when submitting the transaction for execution. Below is the pseudocode algorithm encoding a protobuf value repeated T list;
This encoding function also applies to lists generated from utility functions (e.g: split).

optional

is_set returns true if the value was set in the protobuf, false otherwise.

map

The ordering of map entries in protobuf serialization is not guaranteed, making it problematic for deterministic encoding. To address this, repeated values are used instead of map throughout the protobuf definitions.

gRPC Ledger API Value

Encoding for the Value message defined in com.daml.ledger.api.v2.value.proto For clarity, all value types are exhaustively listed here. Each value is prefixed by a tag unique to its type, which is explicitly specified for each value below.

Unit

Protobuf Definition

Bool

Protobuf Definition

Int64

Protobuf Definition

Numeric

Protobuf Definition

Timestamp

Protobuf Definition

Date

Protobuf Definition

Party

Protobuf Definition

Text

Protobuf Definition

Contract_id

Protobuf Definition

Optional

Protobuf Definition Note this is conceptually the same as for the primitive optional protobuf modifier, with the addition of the type tag prefix.

List

Protobuf Definition

TextMap

Protobuf Definition TextMap.Entry
Protobuf Definition

Record

Protobuf Definition RecordField
Protobuf Definition

Variant

Protobuf Definition

Enum

Protobuf Definition

GenMap

Protobuf Definition GenMap.Entry
Protobuf Definition

Identifier

Protobuf Definition

Transaction

A transaction is a forest (list of trees). It is represented with a following protobuf message found here. The encoding function for a transaction is
encode_node_ids(node_ids) encodes lists in the same way as described before, except the encoding of a node_id is NOT done by encoding it as a string, but instead uses the following encode(node_id) function:
encode(node_id) effectively finds the corresponding node in the list of nodes and encodes the node. The node_id is an opaque value only used to reference nodes and is itself never encoded. Additionally, each node’s encoding is hashed using the sha_256 hashing algorithm. This is relevant when encoding root nodes here as well as when recursively encoding sub-nodes of Exercise and Rollback nodes as seen below.

Node

Each node’s encoding is prefixed with additional meta-information about the node, this is made explicit in the encoding of each node.
Exercise and Rollback nodes both have a children field that references other nodes by their NodeId. The following find_seed: NodeId => optional bytes function is used in the encoding:
some represents a set optional field, none an empty optional field.

Create

Exercise

For Exercise nodes, the node seed MUST be defined. Therefore it is encoded as a non optional field, as noted via the .get in find_seed(node.node_id).get. If the seed of an exercise node cannot be found in the list of node_seeds, encoding must be stopped and it should be reported as a bug.
The last encoded value of the exercise node is its children field. This recursively traverses the transaction tree.

Fetch

Rollback

Rollback nodes do not have an lf version.

Transaction Hash

Once the transaction is encoded, the hash is obtained by running sha_256 over the encoded byte array, with a hash purpose prefix:

Metadata

The final part of PreparedTransaction is metadata. Note that all fields of the metadata need to be signed. Only some fields contribute to the ledger change triggered by the transaction. The rest of the fields are required by the Canton protocol but either have no impact on the ledger change, or have already been signed indirectly by signing the transaction itself.

ProcessedDisclosedContract

Metadata Hash

Once the metadata is encoded, the hash is obtained by running sha_256 over the encoded byte array, with a hash purpose prefix:

Final Hash

Finally, compute the hash that needs to be signed to commit to the ledger changes.
This resulting hash must be signed with the protocol signing private key(s) used to onboard the external party. Both the signature along with the PreparedTransaction must be sent to the API to submit the transaction to the ledger.

Example

Example V3 implementation in Python
Example V2 implementation in Python
Both versions make use of the following common code: