Nomadic Labs
Nomadic Labs

Delphi: changelog

007 Delphi Changelog

Getting the source code of the proposal

The source code of this proposal is available in this tar archive: https://blog.nomadic-labs.com/files/delphi_007_PsDELPH1.tar.

To compute the hash, you can use the tezos-protocol-compiler as follows. The result should be PsDELPH1Kxsxt8f9eWbxQeRxkjfbxoqM52jvs5Y5fBxWWh4ifpo.

mkdir proposal && cd proposal
curl https://blog.nomadic-labs.com/files/delphi_007_PsDELPH1.tar | tar -x
tezos-protocol-compiler -hash-only delphi_007_PsDELPH1

If you want to directly investigate how Delphi differs from the current protocol, you can use diff against folder proto_006_PsCARTHA of the mainnet source code. All the changes should be the ones detailed in the changes section.

Injecting the proposal

Any active baker can inject the proposal by simply downloading the code, injecting it in a running mainnet node with tezos-admin-client and using the submit command of tezos-client.

$ tezos-admin-client inject protocol delphi_007_PsDELPH1
Injected protocol PsDELPH1Kxsxt8f9eWbxQeRxkjfbxoqM52jvs5Y5fBxWWh4ifpo successfully

$ tezos-client submit proposals for <delegate> PsDELPH1Kxsxt8f9eWbxQeRxkjfbxoqM52jvs5Y5fBxWWh4ifpo

Performance Improvements

Some effort was directed to streamline the Michelson interpreter and improve its performance. This mostly consisted in factoring gas consumption, simplifying logging and removing some calls to Lwt (cf. the dedicated section on that last point).

Patches:

  • 217b4fd025 Proto/Michelson: abstract logging facility
  • 7504b35b94 Proto/Michelson: make lists carry their size and type of elements
  • c744b2e36e Proto/Michelson: hoist gas consumption in Slice_{string,bytes}
  • 9854ddc12a Proto/Michelson: hoist Gas.consume out of interp. pattern matching

Gas Changes

Gas accounting has seen a significant update. Even though gas limits are the same as in Carthage, the amount of computation per unit of gas should see a significant increase in Delphi.

More precisely, the following parts of the gas subsystem have been updated. - Following the optimization work on the interpreter, gas costs for all instructions have been recomputed. - The cost model for IOs has been updated and should account better for performances of modern storage hardware. - The typechecking system has an entirely new cost model. - Finally, the base gas cost of manager operations has been reduced from 10000 to 1000 units of gas.

Overall, users should see significant gas cost reductions accross the board.

Patches:

  • 8747f0a987 Proto/Michelson: Use the right cost function for SET_DELEGATE instr
  • 6825b14cc0 Protocol/Michelson: fix gas consumption in Loop exit case
  • 267ddd55d9 Protocol/gas: fix scaling of gas accounting for number of writes
  • f1397bffb6 Proto/gas: make internal gas visible in interpreter traces
  • 1a874fb990 Proto/gas: rescale gas by 2^7, making internal_gas always 0
  • 866fa99e71 Proto/gas: remove internal_gas
  • 3eab3718a4 Proto/Michelson: properly carbonate extract_big_map_updates and collect_big_maps
  • a5baf0ed59 Proto/Michelson: Remove gas cost of ‘map_to_list’ and ‘set_to_list’ conversions
  • 1ae30c0932 Protocol/Migration: scale gas limit constants in the context by 128
  • 76918bf6fa Proto/Gas: remove intermediary cost record
  • 3e11d74d91 Proto/Gas: factor calls to scaling function
  • 07f89a7a9e Proto/Gas: inline calls to scale
  • c67f0dd61c Proto/Gas: perform rescaling from x128 to x1000
  • 786bdb360f Proto/Gas: arithmetic optimization (remove multiplication by 1)
  • 6c611b948d Protocol/Migration: scale gas limit constants in the context by 1000
  • 5dfb79a814 Revert “Protocol/Migration: scale gas limit constants in the context by 1000”
  • fbccb2390a Revert “Vendors/flextesa: scale gas limit constants in the context by 1000”
  • f3201231f2 Revert “Protocol/Migration: scale gas limit constants in the context by 128”
  • b50d6f870a Revert “Vendors/flextesa: scale gas limit constants in the context by 128”
  • 27bb7aeb2d Proto/gas: set 1 milligas = 1 atomic_step_cost
  • 7e50a74755 Proto/Gas: add new gas cost functions in dedicated module
  • 097e79f626 Proto/Gas: inject new cost functions in Gas.cost, expose in .mli
  • 78a2e8069c Proto/Gas: plug new costs on the interpreter
  • 02b5fc516c Proto/Gas: fix gas for Concat_string & Concat_bytes
  • e00ec5eff8 Proto/Gas: unplug 006 interpreter gas
  • 1838b7da21 Proto/Gas: introduce Storage_costs module
  • a21c1ed287 Proto/Gas: use Storage_costs in carbonated storage functors
  • 3281de90c2 Proto/Gas: introduce Contract.get_balance_carbonated
  • 2e1a9d88e3 Proto/Gas: adding encoding/decoding cost constants
  • 68b4c2c37d Proto/Gas: injecting Gas_limit_repr.cost into Alpha_context.Gas.cost
  • 33a0d8f015 Proto/Gas: expose 007 typechecking costs
  • be9c3159d8 Proto/Gas: Adapt translator to 007 typechecking costs
  • bd205ad65e Proto/Gas: expose 007 unparsing costs
  • eb784b35be Proto/Gas: adapt translator to 007 unparsing costs
  • 9b478326fd Proto/Gas: add cost functions and helpers for strip_locations
  • b574bff205 Proto/Gas: carbonate calls to Micheline.strip_locations
  • 6e6d59f578 Proto/Gas: Unplug 006 translator gas
  • 9ae9b3c771 Proto/Gas: prettify constants
  • 987c4f93a7 Proto/Gas: remove useless let bindings in cost functions
  • 4ffc37e769 Proto/tests: add basic unit test for cost functions
  • 5ef9a9aae1 Proto/Gas: reduce gas cost of manager op

The protocol now internally handles milligas but exposes gas to the user. - Protocol constants such as per-block and per-operation gas limits are still specified in gas. - On the input side of the protocol, the gas_limit field in manager operations is still specified in gas, as before. - On the output side, receipts contain both the consumed_gas (as before) and a new field consumed_milligas. The relationship between this two fields is the following:

$$ consumed\_gas = \lceil consumed\_milligas / 1000 \rceil $$

In other terms, consumed_gas, provided for backwards compatibility, is the rounded-up version of consumed_milligas. Users wanting to have the most accurate description of the gas consumption of their transfers should consider using consumed_milligas.

To make understanding the change easier, here is a small example illustrating a transfer in sandboxed mode to a toy minimal.tz contract:

{ parameter int ;
  storage unit ;
  code { CDR ; NIL operation; PAIR } }

Let us originate this contract.

tezos-client -l originate contract mini transferring 0 from bootstrap1 running minimal.tz --burn-cap 0.07375 --init 'Unit'

During simulation, the preapply RPC is given the following operation:

{ "protocol": "ProtoALphaALphaALphaALphaALphaALphaALphaALphaDdp3zK",
  "branch": "BMSAKmQD2Q2Ghk2jtHMGpx5Zww98oLsFdh4XW1863TizUjyfY8S",
  "contents":
  [ { "kind": "origination",
      "source": ..., "fee": "434", "counter": "1",
      "gas_limit": "1690",
      "storage_limit": "315",
      "balance": "0",
      "script": ... }

The gas_limit, equal to 1690 in this example, is labelled in gas. The outcome of the simulation is the following receipt:

{ "kind": "origination",
            "source": ..., "fee": "434",
            "counter": "1",
            "gas_limit": "1690",
            "storage_limit": "315", "balance": "0",
            "script": ...,
            "metadata":
              { "balance_updates": ...,
                "operation_result":
                  { "status": "applied", "big_map_diff": [],
                    "balance_updates": ...,
                    "originated_contracts": ...,
                    "consumed_gas": "1590",
                    "consumed_milligas": "1589562",
                    "storage_size": "38",
                    "paid_storage_size_diff": "38" } } }

The metadata contains both: - the field consumed_milligas, corresponding to the exact amount of gas consumed, labelled in milligas; - the field consumed_gas, obtained by ceiling the milligas value as described above, labelled in gas.

Note that the tezos-client binary prints gas, not milligas. However it displays three decimals:

This sequence of operations was run:
  Manager signed operations:
    From: tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx
    Fee to the baker: ꜩ0.000434
    Expected counter: ...
    Gas limit: 1690
    Storage limit: 315 bytes
    Balance updates:
      tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx ........... -ꜩ0.000434
      fees(tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx,0) ... +ꜩ0.000434
    Origination:
      From: tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx
      Credit: ꜩ0
      Script:
        { parameter int ; storage unit ; code { CDR ; NIL operation ; PAIR } }
        Initial storage: Unit
        No delegate for this contract
        This origination was successfully applied
        Originated contracts:
          KT1HUgv4V8RnEhzNXUxWSqU2PGZ6MgRh34n3
        Storage size: 38 bytes
        Paid storage size diff: 38 bytes
        Consumed gas: 1589.562
        Balance updates:
          tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx ... -ꜩ0.0095
          tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx ... -ꜩ0.06425

Patches:

  • 7868fa947b Proto/gas: introduce Fixed_point_repr module
  • c436869c49 Proto/tests: unit tests for Fixed_point_repr
  • 5a4f408582 Proto/gas: edit TEZOS_PROTOCOL for Fixed_point, move Gas_limit_repr
  • e939017844 Proto/gas: use fixed-point computations for gas (FIXME)
  • 6031a0dc8c Proto/client: use fixed-point computations for gas
  • 7d9f76418d Proto/delegate: use fixed-point computations for gas
  • 804b42cd86 Proto: restore format of receipts to a retrocompatible schema

Lowered storage costs

In Tezos, storing data in the state leads to a burn of tez proportional to the size of the storage increase. This happens when creating a new account, originationg a new smart contract or making the storage of a smart contract grow above its historical higher size. This is different from fees that are paid to the baker proportionally to the size of operations. Indeed, operations are transient (and can be forgotten by nodes in the lighweight rolling mode), while data in the state is replicated by all nodes and can be so forever.

In Delphi, the amount of tez burned to store data in the ledger’s state is decreased by a factor of 4, going from 1 tez to 0.25 tez for a kilobyte. The price to create a new account is thus lowered from 0.257 tez to 0.06425.

Patches:

  • 8f808e8317 proto/parameters: reduce cost_per_byte to 0.000250tz
  • ffaba729db proto/migration: set cost_per_byte to 0.000250tz

Bug fixes

Compatibility with 32-bit platforms

Carthage has some parts that assume a 64-bit runtime. These modifications restore the compatibility with 32-bit systems, and in general make the code clearer and less dependent on the underlying word size.

Patches:

  • da91297c0d Protocol/Michelson: avoid overflowing [Int32.to_int Int32.max_int]
  • 377af3acb0 Proto/Michelson: Simplify parse_uint30
  • 06c2f6f97b Protocol: Safer Int64.to_int
  • 40f9a2c9a3 Storage: rename Int -> UInt16
  • ceb4ef33ba Storage: rename Int_index -> Int31_index
  • 70c0aa4641 Proto/Gas: Zarith-ify some cost functions

BREAKING CHANGE: Michelson annotations

Field annotations in Michelson types were not properly checked and could contain invalid characters. In particular, it was possible to use a digit as the first meaningful character of a field annotation but only if it appeared in a type; a few contracts on Mainnet contain such numerals as first meaningful character. We have added the missing check and extended the syntax of Michelson annotations to allow digits. At the time of writing, all Mainnet and Carthagenet contracts successfully typecheck.

BREAKING CHANGE: If a smart contract containing a non-numeral invalid character in a field annotation inside a type were to be originated before the activation of Delphi, such a contract would be locked by the activation.

Patches:

  • ec1d992e5c Use plain algebraic types for the Michelson annotations
  • 642bab2f97 Proto/Michelson: ensures all annotations are checked
  • 0f12f628e9 Proto/Michelson: extend the set of allowed Michelson annotations
  • 1b179aeb0b Proto/Michleson: make annotations on Right and Elt consistent

Miscellanous bug fixes

Patches:

  • 8c1dd8e53b Proto/Michelson: fix the arity check for chain_id
  • dde9e19d55 Proto/RPC: return all delegates if no flags were used
  • 28b8181a8c Proto/Michelson: fix registration of error Invalid_syntactic_constant
  • 7f329a1700 Proto/Michelson: add missing cases in typechecking error reporting
  • 083e9c6f9b Proto: Add missing case for keyword namespace encoding
  • b1af688dfd Proto: for transfers with 0 tz, check if target exists
  • 982dd6ec77 Proto: Update max_revelations_per_block to include anon ops
  • 508b3ae5f4 Proto/Michelson: readable unparsing of chain ids
  • b03b8b0883 Proto: Fix double encoding of big_map_diff
  • ae0626d708 Proto: make signature check of operation packs more efficient

General code cleanup and refactoring

Removal of Lwt when possible

A large batch of changes consists in mostly trivial modifications that drop parts of the code that don’t perform IOs out of the IO monad. This makes the code a bit less uniform, but helps with safety and future refactoring, and in some cases increases performance (in particular in the interpreter).

Patches:

  • 9e38b8fece Proto/Michelson: remove some Lwt wrappers
  • eda54e8ba3 Proto/Michelson: avoid Lwt.bind when logging
  • 187d9d7cab Proto: bind -> to
  • b2f8aac20c List.map -> fold_left_s
  • 85592c26e9 Proto: filter_map_s -> filter_s
  • 14eda1e7a9 List.fold_left -> Error_monad.map
  • 43306a4b89 Script: force_decode/bytes, get rid of Lwt.t
  • 7b1199d76d Baking: minimal_(valid_)time, get rid of Lwt
  • c5a0f66bb0 Baking.earlier_predecessor_timestamp: get rid of Lwt
  • e112d592b2 Baking: baking/endorsing_reward, get rid of Lwt
  • 622ac83eaf Baking/tests: baking/endorsing_reward, get rid of Lwt
  • 287985b4c7 Baking.check_fitness_gap: get rid of Lwt
  • a9cdf7e3bc Contract_storage.fresh_contract_from_current_nonce: get rid of Lwt
  • aef7541f69 Delegate_services.required_endorsements/minimal_valid_time: get rid of Lwt
  • 3854c15614 Fees_storage.origination_burn: get rid of Lwt
  • d71fe7c127 Operation.check_signature: get rid of Lwt
  • 419d29b41b Proto: Lwt.return >>=? -> >>?=
  • 36172136a2 Raw_context.add_fees/rewards/deposit: get rid of Lwt
  • f19e52bd86 Script_ir_annot: get rid of fail_unexpected_annot
  • b55bfc4673 Script_ir_translator: remove lots of Lwt in parse_instr
  • 4381c2b266 Script_ir_translator.parse_data: remove lots of Lwt
  • 509decd748 Script_ir_translator.parse_data: traced
  • 8da99eff1a Script_ir_translator: less Lwt
  • 407e460bce Script_ir_translator.collect_big_maps: get rid of Lwt
  • 1c94741b2b Storage: less Lwt
  • 30fad4cb40 Proto: fail_unless/when -> error_unless/when
  • 01aeae86a3 Proto: use predefined ok constants
  • 0b8964ad8f Proto: lift some Lwt.return

Miscellanous code improvements & refactors

A long series of patches consists in minor improvements to the code style and contents of comments or error messages. Part of this work is thanks to the automated transcription of the code of the protocol into Coq (as part of our formal verification effort).

Patches:

  • 88bc601fe9 Proto: Uniform variable names for context + comments clean-up
  • 6d59b3f32a Michelson: exposes parsing without specifying storage
  • bef609828f Proto: expose parse_storage
  • abb3a3a48c Include ty_eq in merge_types
  • b1519b85e2 Express (comparable_)ty_eq in terms of merge_(comparable_)ty
  • d96d1cc2d1 Migration: remove leftovers Alpha_previous and Babylon_005
  • cd4bcec080 Proto/Michelson: remove dead code in parse_instr
  • 74db352afd Remove mutual dependency between numeric types
  • 60b8998900 Proto: remove unused exported vals
  • 83aa3f49a5 Rewrite some when clauses for Coq
  • 4b5307d995 Remove useless recursion
  • f34ec2a28d Proto: Fix formatting
  • 99d7b81533 Proto: Move and export Michelson prim namespace function
  • 29153f27ff Proto/Michelson: simplify the interpreter
  • 36cbee348b Michelson: rename Left/Right as Cons_left/right
  • c6cba0db30 Michelson: simplify GADT matchings in script_ir_translator 1/4
  • e828029fb3 Michelson: simplify GADT matchings in script_ir_translator 2/4
  • 0bff45c7f2 Michelson: simplify GADT matchings in script_ir_translator 3/4
  • edfae2bcf8 Michelson: simplify GADT matchings in script_ir_translator 4/4
  • cff2ab3b69 Proto/Michelson: extract_big_map_updates: aux
  • 425f3eaaa1 Proto/Michelson: compute has_big_map only when needed
  • 3b59480b92 Proto/Michelson: get rid of has_big_map flag and old function
  • dd34270a63 Proto/Michelson: factorize parse_storage_ty
  • 8d17e58cc4 Proto/RPCs: use parse_packable/parameter_ty instead of parse_ty in services
  • 09013da929 Proto/Contracts: big_map_diff/Copy: use inline record
  • 084037dc9c Proto/Michelson: remove dead code
  • 5e83fe9126 Proto/Michelson: simplify Big_map.fresh
  • 0172dec928 Proto/Michelson: expose parse_ty for convenience to external tools
  • 6c8e8a7f46 Proto/Michelson: expose unparse_code in Script_ir_translator
  • d793d30003 Proto_alpha: spell check
  • 5807a28c44 Proto/coq-of-ocaml: Remove a polymorphic variant in raw_context
  • 5da72db33f Proto: update the old operator to regular naming scheme
  • 03ab590364 Protocol/coq-of-ocaml: name the signatures of the protocol
  • a06caece9f Protocol/coq-of-ocaml: avoid a name collision on encoding in the generated Coq
  • a25feb04f5 Use Option.value
  • 12c7cf2f81 Protocol/coq-of-ocaml: rename of_seconds to prevent a collision of name
  • b8dd4fc1f9 Protocol/coq-of-ocaml: renaming of force_decode / force_bytes to prevent collision
  • c42b58c8c3 Protocol/coq-of-ocaml: rename consume and check_enough to prevent collisions
  • c4ac279d65 Protocol/coq-of-ocaml: renaming to compile alpha_context
  • d6ec7dd8c7 Protocol/coq-of-ocaml: lint the interpreter
  • da7d945551 Protocol/coq-of-ocaml: changes to compile storage.ml
  • 75dad446a4 Protocol/coq-of-ocaml: add signature annotations
  • 036c287157 Script_ir_translator.parse_data: rename error
  • b25d542877 Proto: fix typo in docstring
  • fe988f439c Proto: remove dead code

Safety of Michelson

Explicit limitations in the Michelson typechecker and interpreter

Up until now, some features of Michelson were implicitly limited by gas. This series of changes add explicit limits that will help with safety, future refactoring, and debugging (more precise errors).

  • Deep stack instructions DIG, DUG, DROP, and DIP are now bounded to stacks of less than 1024 elements
  • The interpreter (resp. typechecker) now have explicit recursion limits, setting a maximum depths for terms that can be interpreted (resp. typechecked).

Both changes set limits to high enough values, and should thus be invisible to almost all contract authors. Reaching the limits is only likely to happen when debugging erroneous (such as non-terminating) code.

Patches:

  • 4c019e1004 Proto/Michelson: fix error message for DIP with wrong constant
  • 7d0211b648 Michelson: fix number_of_generated_growing_types
  • 6a7bbf17a4 Proto: add max stack depth
  • 040abed403 Proto: normalize stackoverflows in typechecking and unparsing
  • 95c31f963c Michelson: restrict deep stack instructions to 1023

Receive Updates

ATOM

Contacts