Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
419 lines (304 loc) · 15.8 KB

CONTRIBUTING.md

File metadata and controls

419 lines (304 loc) · 15.8 KB

Thank you for taking the time to contribute 🙌

The following is a set of guidelines for contributing to the Consensus component of Cardano. If you have suggestions on how to improve this document, please feel free to propose changes to it in a pull request. Bear in mind that the document should remain simple. See the quick reference section if you are in a hurry.

Documentation

Documentation should be a first-class citizen of the Consensus code base. We are in the process of improving the Consensus documentation, and your improvement proposals are welcome.

We have two types of documentation:

  • Markdown files, which can be found in the docs directory. They contain information that is not strictly related to the code itself, such as getting started guides, references, tutorials, etc.

    They are rendered at the Consensus website.

  • Haddock comments. They contain more low level information about the code and should be used as a reference, although we aim to provide some context navigation (cross referencing modules) on the module descriptions.

When starting to work on Consensus, we recommend to take a look at the following resources:

When adding or improving documentation about the implementation, it is preferable to add haddock comments since they are closer to the code. However not all documentation can be placed inside haddock comments, and in such cases the contributor can update the markdown files in docs.

This repository also contains a technical report that describes the implementation of the Consensus layer. We will not update this report. We keep it here as a historical reference, and we will systematically convert the relevant parts of the report into the two types of documentation mentioned above.

When somebody asks a question about the code, we should try to refer people to the documentation. If no relevant entry exists, we should create it and submit a pull request.

For the time being, all markdown files that contain the Consensus documentation live in this repository.

Setting up the build tools

Using Nix

Consensus can be built using Nix. The installation and configuration instructions are taken from cardano-node, and detailed below. To install nix run:

curl -L https://nixos.org/nix/install > install-nix.sh
chmod +x install-nix.sh
./install-nix.sh

Then enable the following features:

sudo mkdir -p /etc/nix
cat <<EOF | sudo tee /etc/nix/nix.conf
experimental-features = nix-command flakes
allow-import-from-derivation = true
EOF

As an optional step, to improve build speed (highly recommended), you can set up a binary cache maintained by IOG. If you use flakes, you will be asked to accept using it automatically; alternatively, you can add it globally:

sudo mkdir -p /etc/nix
cat <<EOF | sudo tee -a /etc/nix/nix.conf
substituters = https://cache.nixos.org https://cache.iog.io
trusted-public-keys = hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
EOF

Using cabal

An alternative to using nix is to set up the development environment yourself. Follow these instructions to properly configure your system.

Building the project

To build all the packages in this repository run:

cabal build all

in the command line, either inside nix-shell if you use nix, or in a system with cabal installed.

Specific executables can be executed through cabal once built:

cabal run db-analyser

Testing

To test all the packages in this repository run:

cabal test all

in the command line, either inside nix-shell if you use nix, or in a system with cabal installed.

For running specific test-suites (such as consensus-test), we recommend one of the following commands:

cabal run ouroboros-consensus:test:consensus-test -- <args>
cabal test ouroboros-consensus:test:consensus-test --test-show-details=direct

Note the second one cannot be used when we want to provide CLI arguments to the test-suite.

Generating documentation and setting up hoogle

The documentation contains some tikz figures that require some preprocessing for them to be displayed. To do this, use the documentation script:

./scripts/docs/haddocks.sh

If not already in your PATH (eg when in a Nix shell), this will install cabal-docspec from a binary, and then build the haddocks for the project.

Often times, it is useful to have a hoogle server at hand, with the packages and its dependencies. Our suggestion is to install cabal-hoogle from github:

git clone git@github.com:kokobd/cabal-hoogle
cd cabal-hoogle
cabal install exe:cabal-hoogle

and then run cabal-hoogle:

cabal-hoogle generate
cabal-hoogle run -- server --local

This will fire a hoogle server at https://localhost:8080/ with the local packages and their dependencies.

Contributing to the code

The following sections contain some guidelines that should be followed when contributing to the Consensus code base. Please take some time to go through them. We do not expect newcomers to adhere to these guidelines perfectly, and we will guide you through the process when reviewing your pull request.

Quick reference

This section contain guidelines on what to check when making a pull request.

Following our git process

Our git process describes the git practices we encourage when working with the code in this repository.

Updating the documentation

When submitting a pull request, please look update the relevant parts of the documentation (see this section).

Following the style guide

We have a Haskell style guide that should be followed when writing code in Consensus. Our style guide is not set in stone, and improvements are always welcome.

Formatting the code

We use fourmolu for Haskell code formatting. See tools.nix for the fourmolu version used in CI.

Either enable editor integration or call the script used by CI itself:

./scripts/ci/run-fourmolu.sh

When using Nix, you can use the following command, which will build and use the right version of fourmolu.

nix develop -c ./scripts/ci/run-fourmolu.sh

Generating documentation and setting up hoogle

To generate the documentation, use the documentation script:

./scripts/docs/haddocks.sh

Often times, it is useful to have a hoogle server at hand, with the packages and its dependencies. Our suggestion is to use cabal-hoogle which is included in the nix shell:

nix develop
$ cabal-hoogle generate
$ cabal-hoogle run -- server --local

This will fire a hoogle server at https://localhost:8080/ with the local packages and their dependencies.

Making and reviewing changes

If you are working on changing a substantial part of Consensus, it is important that you contact the core developers first to discuss design considerations (See section Contacting the developers). This will help detecting any potential problems with the change in the design phase, preventing misunderstandings and frustrations later on in the process.

We maintain a changelog. If your pull request requires a changelog entry, please follow these instructions. Even if your change doesn't require a changelog fragment, create an empty one as CI will reject your change otherwise. We made this choice to ensure authors of PRs would always take a moment to consider whether a changelog fragment should be added for their PR. For more information see our release process.

When creating a pull-request (PR), it is crucial that the PR:

  • has a clear description,

  • targets the main branch (unless there is a good reason not to),

  • is as small as possible, and

  • is organized in a cohesive sequence of commits, each representing a meaningful, logical and reviewable step.

Updating dependencies (index-state)

Our Haskell packages come from two package repositories:

  • Hackage
  • CHaP (which is essentially another Hackage)

The index-state of each repository is pinned to a particular time in cabal.project. This tells Cabal to treat the repository as if it was the specified time, ensuring reproducibility. If you want to use a package version from repository X which was added after the pinned index state time, you need to bump the index state for X. This is not a big deal, since all it does is change what packages cabal considers to be available when doing solving, but it will change what package versions cabal picks for the plan, and so will likely result in significant recompilation, and potentially some breakage. That typically just means that we need to fix the breakage (increasing the lower-bound on the problematic package if fix is not backward compatible), or delay that work and instead decrease the upper-bound on the problematic package for now.

Note that cabal's own persistent state includes which index states it is aware of, so when you bump the pinned index state you may need to call cabal update in order for cabal to be happy.

The Nix code which builds our packages also needs some information relating to the index-state. This information needs to be new enough to include the index-state specified in cabal.project. The information is represented by Nix flake inputs. You can update these by running:

  • nix flake update hackageNix for Hackage
  • nix flake update CHaP for CHaP

If you fail to do this you may get an error like this from Nix:

error: Unknown index-state 2021-08-08T00:00:00Z, the latest index-state I know about is 2021-08-06T00:00:00Z. You may need to update to a newer hackage.nix.

The index-state of the tools is pinned in ./nix/tools.nix to ensure that an incompatible change in the set of packages available in Hackage doesn't break the shell. From time to time, this index-state should be updated manually.

Updating the dependencies bounds

Sometimes, when creating pull requests to CHaP, it is desirable to loose/tighten certain dependencies bounds via a revision.

  • If you do so for a Consensus package, please first open a PR to Consensus mirroring the change in CHaP; but do not increment the version number of the Consensus package.

  • If your revision is about allowing a new version of package, please update the version bound in a way that keeps the previously allowed versions, unless this is undesirable due to eg a bug in an earlier version.

    For example, if we have cardano-ledger-core ^>= 1.1 and you want to also allow 1.2, use ^>= 1.1 || ^>= 1.2.

Use of source-repository-packages

We can use Cabal's source-repository-package mechanism to pull in un-released package versions. This can be useful when debugging/developing across different repositories. However, we should not release our packages to CHaP while we depend on a source-repository-package since downstream consumers would not be able to build such package.

If we are stuck in a situation where we need a long-running fork of a package, we should release it to CHaP instead (see the CHaP README for more).

In general, we strive to avoid having source-repository-packages on our main branch. However, there are situations where we want to prevent pull requests from piling up while awaiting the release of upstream components1. In these cases, we allow merging pull requests that contain source-repository-packages, provided the referenced commit is on the main branch of the upstream package.

If you do add a temporary source-repository-package stanza, you need to provide a --sha256 comment in cabal.project so that Nix knows the hash of the content. There are two relatively straightforward ways to do this:

  1. The TOFU approach: put in the wrong hash and then Nix will tell you the correct one, which you can copy in.
  2. Calculate the hash with nix-shell -p nix-prefetch-git --run 'nix-prefetch-git <URL> <COMMIT_HASH>'

Inspecting dependencies as used by Nix

When debugging an issue or when not using the shipped Nix shell, it is often desirable to know the exact version/revision of a specific tool or library as used by Nix.

To get the exact libsodium-vrf used by Nix:

 $ nix eval --json .#libsodium-vrf.src.rev
"dbb48cce5429cb6585c9034f002568964f1ce567"

To get the cabal-gild version used by Nix:

 $ nix eval --json .#cabal-gild.version
"1.3.0.1"

In more complex cases, you can start a Nix REPL and go on to explore interactively:

 $ nix repl
nix-repl> :lf .
nix-repl> pkgs = legacyPackages.${__currentSystem}
nix-repl> pkgs.hsPkgs.ouroboros-consensus.components.exes.db-
pkgs.hsPkgs.ouroboros-consensus.components.exes.db-analyser     pkgs.hsPkgs.ouroboros-consensus.components.exes.db-synthesizer

Reporting a bug or requesting a feature

If you happen to encounter what you think is a bug or you wish there was some feature that was added to Consensus, please open an issue in our issue tracker.

Submitting pull requests

We monitorize the repository constantly so we should be aware if you open a pull-request, however if some reasonable time passes and we miss your PR, feel free to ping someone from the team.

Contacting the developers

The core contributors to consensus codebase are:

Code of conduct

See Cardano engineering handbook's code of conduct.

Footnotes

  1. #1376 provides an example of an integration pull request that incorporated changes from several others that were waiting on the release of upstream packages. The resulting pull request was extremely tedious to review, making the process more error-prone than reviewing a shorter one due to its sheer size. ↩