Setup with Imported Dockerized Weaver Components

In this document, we detail the steps using which you can bring up networks using the default configuration settings and by fetching pre-built Weaver interoperation modules, SDK libraries, and relay docker image, drivers docker images from GitHub Package repositories. To customize these settings (e.g., hostnames, ports), refer to the Advanced Configuration page.

Notes
All components are run within Docker containers, except client applications.

Follow the instructions below to build and run components followed by interoperation flows. These instructions have been tested on Ubuntu Linux (bash shell) and Mac OS. In general, they should work on any system and shell as long as the various dependencies have been installed and configured.

Prerequisites¶

Software¶

Before starting, make sure you have the following software installed on your host machine:

Curl: install using package manager, like apt on Debian/Ubuntu Linux
Git: sample instructions
Docker: sample instructions (Latest version)
Docker-Compose: sample instructions (Version 2 or higher)
Golang: sample instructions (Version 1.20 or higher)
Java (JDK and JRE): sample instructions (Version 8)
Node.js and NPM: sample instructions (Version 11 to Version 16 Supported)
Yarn: sample instructions

Credentials¶

Make sure you have an SSH or GPG key registered in https://github.com to allow seamless cloning of repositories (at present, various setup scripts clone repositories using the https:// prefix but this may change to git@ in the future).

Package Access Token:¶

Create a personal access token with read:packages access in GitHub in order to use modules published in GitHub packages. Refer Creating a Personal Access Token for help.

Run docker login ghcr.io, and provide GitHub email id as username and personal access token created above as password. This will allow Docker to fetch images of relay, fabric-driver and corda-driver from hyperledger/cacti.

Getting the Code and Documentation¶

Clone the cacti repository. The code to get a basic test network up and running and test data-sharing interoperation flows lies in the subfolder weaver/tests/network-setups, which should be your starting point, though the setups will rely on other parts of the repository, as you will find out in the instructions given on this page.

Securing Components¶

Notes

The relays and drivers corresponding to the different test networks you will encounter below can be run with or without TLS enabled. But the default files used in the demonstrations assume that either all relays and drivers are TLS-enabled or none are. Therefore, you should determine at the outset whether or not you wish to run the entire set of components in TLS-enabled mode, and select appropriate commands in the provided instructions.

Hyperledger Fabric Components¶

Using the sequence of instructions below, you can start two separate Fabric networks, each with a single channel and application contract (chaincode). You can also start an interoperation contract, a relay and a driver acting on behalf of each network. You can build a Fabric CLI tool with which you can initialize both networks' ledgers with access control policies, foreign networks' security groups (i.e., membership providers' certificate chains), and some sample key-value pairs that can be shared during subsequent interoperation flows.

Fabric Network¶

The code for this lies in the weaver/tests/network-setups folder.

This folder contains code to create and launch networks network1 and network2 of identical specifications:

Network: 1 peer, 1 peer CA, 1 ordering service node, 1 ordering service CA
Single channel named mychannel
One of the following contracts deployed on mychannel, the choice depending on the interoperability mode you wish to test:
- simplestate (Data Sharing): supports simple transactions (Create, Read, Update, Delete) involving storage and lookup of pairs.
- simplestatewithacl (Data Sharing): identical to simplestate but with extra security features to ensure that the Weaver infrastructure cannot be bypassed by a malicious client of the network.
- simpleasset (Asset Exchange): supports creation, modification, transfer, and deletion, as well as locking, unlocking, and claiming, of simple bonds and tokens (examples of non-fungible and fungible assets respectively).
- simpleassetandinterop (Asset Exchange): identical to simpleasset but where the locking, unlocking, and claiming logic is imported as a library in the chaincode rather than available in the common Fabric Interoperation Chaincode (a Weaver component).
- simpleassettransfer (Asset Exchange or Asset Transfer): augmentation of simpleasset with asset pledging, claiming, and reclaiming features for cross-network transfers.

Notes
For new users, we recommend testing the Data Sharing feature first with the `simplestate` contract. To test the other modes, you can simply tear down the Fabric networks and restart them with the appropriate chaincodes installed.

Follow the instructions below to build and launch the networks:

Navigate to the weaver/tests/network-setups/fabric/dev folder.
To spin up both network1 and network2 with the interoperation chaincode and the default simplestate chaincode installed, run:
```
make start-interop
```
To launch the networks with a different application chaincode from the above list, run:
```
make start-interop CHAINCODE_NAME=<chaincode-name>
```
To launch the networks with 2 organizations, each with a peer (this will enable more variation and experimentation, which you can attempt after testing interoperation protocols across basic network configurations), run:
```
make start-interop-local PROFILE="2-nodes"
```

Notes
If you do not wish to test Fabric-Fabric interoperation, you can choose to launch only one of the two networks along with its interoperation chaincode. For `network1`, run `make start-interop-network1`, and for `network2`, run `make start-interop-network2`
If you wish to enable end-to-end confidentiality by default in the interoperation modules that are deployed during network launch, set the environment variable `E2E_CONFIDENTIALITY` to `true` in the command line as follows: `E2E_CONFIDENTIALITY=true make start-interop`

For more information, refer to the associated README.

Troubleshooting Tips:

If you see any errors during the launches, re-check the prerequisites (software installations and credentials). Ensure your network connection is working. As a safe bet, you can retry after cleanup: kill and remove all Docker containers and associated volumes.

Fabric Relay¶

The relay is a module acting on behalf of a network, enabling interoperation flows with other networks by communicating with their relays. The code for this lies in the weaver/core/relay folder.

Navigate to the weaver/core/relay folder and run a relay as follows:

The docker-compose.yaml in this folder is minimally configured with default values. To modify it for use with the Fabric testnets, run:
```
make convert-compose-method2
```
(The .env.n1 and .env.n1.tls files in the docker/testnet-envs directory contain environment variables used by the network1 relay at startup and runtime.)
(The .env.n2 and .env.n2.tls files in the docker/testnet-envs directory contain environment variables used by the network2 relay at startup and runtime.)

To deploy the relay server for network1 without TLS, run:

make start-server COMPOSE_ARG='--env-file docker/testnet-envs/.env.n1'

Instead, to deploy the relay server with TLS, run:

make start-server COMPOSE_ARG='--env-file docker/testnet-envs/.env.n1.tls'

To deploy the relay server for network2 without TLS, run:

make start-server COMPOSE_ARG='--env-file docker/testnet-envs/.env.n2'

Instead, to deploy the relay server with TLS, run:

make start-server COMPOSE_ARG='--env-file docker/testnet-envs/.env.n2.tls'

After launching the relay(s), you can revert the docker-compose.yaml changes by running:
```
make convert-compose-method1
```

For more information, see the relay-docker README.

Fabric Driver¶

A driver is a DLT-specific plugin invoked by the relay while channelling external data queries to the local peer network and collecting a response with proofs. The Fabric driver is built as a Fabric client application on the fabric-network NPM package. The code for this lies in the weaver/core/drivers/fabric-driver folder.

Use the following steps to run Fabric drivers in Docker containers:

Navigate to the weaver/core/drivers/fabric-driver folder.
- The .env.n1 and .env.n1.tls files in the docker-testnet-envs directory contain environment variables used by the network1 driver at startup and runtime. Edit either of these files (depending on whether you wish to start the relay with or without TLS) as follows:
  - Replace <PATH-TO-WEAVER> with the absolute path of the weaver folder within your Cacti repository clone.
- Repeat the above step for .env.n2 or .env.n2.tls in docker-testnet-envs directory, which contain environment variables for the network2 driver.
- To deploy the Fabric driver for network1 without TLS, run:
```
make deploy COMPOSE_ARG='--env-file docker-testnet-envs/.env.n1' NETWORK_NAME=$(grep NETWORK_NAME docker-testnet-envs/.env.n1 | cut -d '=' -f 2)
```
  Instead, to deploy the driver with TLS, run:
```
make deploy COMPOSE_ARG='--env-file docker-testnet-envs/.env.n1.tls' NETWORK_NAME=$(grep NETWORK_NAME docker-testnet-envs/.env.n1.tls | cut -d '=' -f 2)
```
- To deploy the Fabric driver for network2 without TLS, run:
```
make deploy COMPOSE_ARG='--env-file docker-testnet-envs/.env.n2' NETWORK_NAME=$(grep NETWORK_NAME docker-testnet-envs/.env.n2 | cut -d '=' -f 2)
```
  Instead, to deploy the driver with TLS, run:
```
make deploy COMPOSE_ARG='--env-file docker-testnet-envs/.env.n2.tls' NETWORK_NAME=$(grep NETWORK_NAME docker-testnet-envs/.env.n2.tls | cut -d '=' -f 2)
```

Fabric IIN Agent¶

IIN Agent is a client of a member of a DLT network or security domain with special permissions to update security domain identities and configurations on the ledger via the network's interoperation module. The code for this lies in the weaver/core/identity-management/iin-agent folder. Navigate to the weaver/core/identity-management/iin-agent folder.

Deployment¶

Use the following steps to run Fabric IIN Agents in Docker containers:

The .env.n1.org1 and .env.n1.org1.tls files in the docker-testnet/envs directory contain environment variables used by the iin-agent of org1 of network1 at startup and runtime. Edit either of these files (depending on whether you wish to start the relay with or without TLS) as follows:
- Replace <PATH-TO-WEAVER> with the absolute path of the weaver folder within your Cacti repository clone.
- If Fabric network was started with 1 org, and IIN Agents are to be started with TLS enabled, update the DNS_CONFIG_PATH variable as:
```
DNS_CONFIG_PATH=./docker-testnet/configs/dnsconfig-tls.json
```
- If Fabric network was started with 2 orgs, and IIN Agents are to be started without TLS, update the DNS_CONFIG_PATH variable as
```
DNS_CONFIG_PATH=./docker-testnet/configs/dnsconfig-2-nodes.json
```
- If Fabric network was started with 2 orgs and IIN Agents are to be started with TLS enabled, update the DNS_CONFIG_PATH variable as:
```
DNS_CONFIG_PATH=./docker-testnet/configs/dnsconfig-tls-2-nodes.json
```
Repeat the above steps for all other environment variable files (depending upon whether tls is enabled) in docker-testnet/envs directory.

To deploy the Fabric IIN Agent for org1 of network1 without TLS, run:

make deploy COMPOSE_ARG='--env-file docker-testnet/envs/.env.n1.org1' DLT_SPECIFIC_DIR=$(grep DLT_SPECIFIC_DIR docker-testnet/envs/.env.n1.org1 | cut -d '=' -f 2)

Instead, to deploy the IIN Agent with TLS, run:

make deploy COMPOSE_ARG='--env-file docker-testnet/envs/.env.n1.org1.tls' DLT_SPECIFIC_DIR=$(grep DLT_SPECIFIC_DIR docker-testnet/envs/.env.n1.org1.tls | cut -d '=' -f 2)

To deploy the Fabric IIN Agent for org2 of network1 without TLS (only required if Fabric network was started with 2 orgs), run:

make deploy COMPOSE_ARG='--env-file docker-testnet/envs/.env.n1.org2' DLT_SPECIFIC_DIR=$(grep DLT_SPECIFIC_DIR docker-testnet/envs/.env.n1.org2 | cut -d '=' -f 2)

Instead, to deploy the IIN Agent with TLS, run:

make deploy COMPOSE_ARG='--env-file docker-testnet/envs/.env.n1.org2.tls' DLT_SPECIFIC_DIR=$(grep DLT_SPECIFIC_DIR docker-testnet/envs/.env.n1.org2.tls | cut -d '=' -f 2)

To deploy the Fabric IIN Agent for org1 of network2 without TLS, run:

make deploy COMPOSE_ARG='--env-file docker-testnet/envs/.env.n2.org1' DLT_SPECIFIC_DIR=$(grep DLT_SPECIFIC_DIR docker-testnet/envs/.env.n2.org1 | cut -d '=' -f 2)

Instead, to deploy the IIN Agent with TLS, run:

make deploy COMPOSE_ARG='--env-file docker-testnet/envs/.env.n2.org1.tls' DLT_SPECIFIC_DIR=$(grep DLT_SPECIFIC_DIR docker-testnet/envs/.env.n2.org1.tls | cut -d '=' -f 2)

To deploy the Fabric IIN Agent for org2 of network2 without TLS (only required if Fabric network was started with 2 orgs), run:

make deploy COMPOSE_ARG='--env-file docker-testnet/envs/.env.n2.org2' DLT_SPECIFIC_DIR=$(grep DLT_SPECIFIC_DIR docker-testnet/envs/.env.n2.org2 | cut -d '=' -f 2)

Instead, to deploy the IIN Agent with TLS, run:

make deploy COMPOSE_ARG='--env-file docker-testnet/envs/.env.n2.org2.tls' DLT_SPECIFIC_DIR=$(grep DLT_SPECIFIC_DIR docker-testnet/envs/.env.n2.org2.tls | cut -d '=' -f 2)

Fabric Client (Application)¶

The CLI is used to interact with a Fabric network, configure it and run chaincode transactions to record data on the channel ledger or query data. It is also used to interact with remote networks through the relay in order to trigger an interoperation flow for data request and acceptance.

The fabric-cli Node.js source code is located in the weaver/samples/fabric/fabric-cli folder and the Golang source code in the weaver/samples/fabric/go-cli folder.

Prerequisites¶

If you are using a Linux system, make sure that lib64 is installed.

Notes
For the Node.js version of the `fabric-cli`, the setup and running instructions below were tested with all Node.js versions from v11.14.0 to v14.17.3.

Installation¶

You can install fabric-cli as follows (for both the Node.js and Golang versions):

Navigate to the weaver/samples/fabric/fabric-cli folder (for the Node.js version) or the weaver/samples/fabric/go-cli folder (for the Golang version).
Create .npmrc from template .npmrc.template, by replacing <personal-access-token> with yours created above..
Run the following to install dependencies (for the Node.js version) or the executable (for the Golang version):
```
make build
```
Use the fabric-cli executable in the bin folder for subsequent actions.

Corda Components¶

Using the sequence of instructions below, you can start a Corda network and run an application CorDapp on it. You can also run an interoperation CorDapp, a relay and a driver acting on behalf of the network. You can initialize the network's vault with access control policies, foreign networks' security groups (i.e., membership providers' certificate chains), and some sample state values that can be shared during subsequent interoperation flows.

Corda Network¶

The Corda networks' code lies in the weaver/tests/network-setups/corda folder. You can launch two separate Corda networks, namely Corda_Network and Corda_Network2. Each network runs the weaver/samples/corda/corda-simple-application CorDapp by default, which maintains a state named SimpleState containing a set of key-value pairs (of strings).

The following steps will, in addition to launching the network, build the CorDapp and a Corda client in weaver/samples/corda/corda-simple-application/client.

Running with Interoperation CorDapp from GitHub Packages¶

Follow the instructions below to build and launch the network:

Navigate to the weaver/tests/network-setups/corda folder.
Create copy of github.properties.template as github.properties.
Replace <GITHUB email> with your GitHub email, and <GITHUB Personal Access Token> with the access token created above.
To spin up the Corda networks with the Interoperation CorDapps:
- Each consisting of 1 node and a notary (for data-transfer), run:
```
make start
```
- Each consisting of 2 nodes and a notary (for asset-exchange/transfer), run:
```
make start PROFILE="2-nodes"
```
- Each consisting of 3 nodes and a notary (for asset-exchange/transfer), run:
```
make start PROFILE="3-nodes"
```

You should see the following message in the terminal:

Waiting for network node services to start

The Corda nodes and notary may take a while (several minutes on memory-constrained systems) to start. If they start up successfully, you should something like the following for each network, though the number of node entries will depend on the profile you used to start the network with (replace <network-name> with Corda_Network or Corda_Network2):

PartyA node services started for network <network-name>
PartyB node services started for network <network-name>
PartyC node services started for network <network-name>
Notary node services started for network <network-name>

Corda Relay¶

Navigate to the weaver/core/relay folder and run a relay for Corda_Network and/or Corda_Network2 in Docker container as follows:

The docker-compose.yaml in this folder is minimally configured with default values. To modify it for use with the Fabric testnets, run:
```
make convert-compose-method2
```
(The .env.corda and .env.corda.tls files in the docker/testnet-envs directory contain environment variables used by the Corda_Network relay at startup and runtime.)
(The .env.corda2 and .env.corda2.tls files in the docker/testnet-envs directory contain environment variables used by the Corda_Network2 relay at startup and runtime.)

To deploy the relay server for Corda_Network without TLS, run:

make start-server COMPOSE_ARG='--env-file docker/testnet-envs/.env.corda'

Instead, to deploy the relay server with TLS, run:

make start-server COMPOSE_ARG='--env-file docker/testnet-envs/.env.corda.tls'

To deploy the relay server for Corda_Network2 without TLS, run:

make start-server COMPOSE_ARG='--env-file docker/testnet-envs/.env.corda2'

Instead, to deploy the relay server with TLS, run:

make start-server COMPOSE_ARG='--env-file docker/testnet-envs/.env.corda2.tls'

After launching the relay(s), you can revert the docker-compose.yaml changes by running:
```
make convert-compose-method1
```

Corda Driver¶

Use the following steps to run Corda drivers in Docker containers:

Navigate to the weaver/core/drivers/corda-driver folder.
(The .env.corda and .env.corda.tls files in the docker-testnet-envs contain environment variables used by the Corda_Network driver at startup and runtime.)
(The .env.corda2 and .env.corda2.tls files in the docker-testnet-envs contain environment variables used by the Corda_Network2 driver at startup and runtime.)
To deploy the Corda driver for Corda_Network without TLS, run:
```
make deploy COMPOSE_ARG='--env-file docker-testnet-envs/.env.corda'
```
Instead, to deploy the driver with TLS, run:
```
make deploy COMPOSE_ARG='--env-file docker-testnet-envs/.env.corda.tls'
```
If the driver starts successfully, it should log the following message when you run docker logs corda-driver-Corda_Network:
```
Corda driver gRPC server started. Listening on port 9099
```
To deploy the Corda driver for Corda_Network2 without TLS, run:
```
make deploy COMPOSE_ARG='--env-file docker-testnet-envs/.env.corda2'
```
Instead, to deploy the driver with TLS, run:
```
make deploy COMPOSE_ARG='--env-file docker-testnet-envs/.env.corda2.tls'
```
If the driver starts successfully, it should log the following message when you run docker logs corda-driver-Corda_Network2:
```
Corda driver gRPC server started. Listening on port 9098
```

Tear Down the Setup¶

Bring down the various components as follows (Navigate to the weaver folder of your clone of the Cacti repository):

Relay¶

To bring down the relays (for all 3 networks), run:

cd core/relay
make convert-compose-method2
make stop COMPOSE_ARG='--env-file docker/testnet-envs/.env.n1'
make stop COMPOSE_ARG='--env-file docker/testnet-envs/.env.n2'
make stop COMPOSE_ARG='--env-file docker/testnet-envs/.env.corda'
make stop COMPOSE_ARG='--env-file docker/testnet-envs/.env.corda2'
make convert-compose-method1
cd -

Fabric Driver¶

To bring down the fabric drivers (for both networks), run:

cd core/drivers/fabric-driver
make stop COMPOSE_ARG='--env-file docker-testnet-envs/.env.n1'
make stop COMPOSE_ARG='--env-file docker-testnet-envs/.env.n2'
cd -

Corda Driver¶

To bring down the corda driver, run:

cd core/drivers/corda-driver
make stop COMPOSE_ARG='--env-file docker-testnet-envs/.env.corda'
make stop COMPOSE_ARG='--env-file docker-testnet-envs/.env.corda2'
cd -

Corda Network¶

To bring down the Corda network:

cd tests/network-setups/corda
make clean
cd -

Fabric Network¶

To bring down both of the Fabric networks along with weaver components:

cd tests/network-setups/fabric/dev
make clean
cd -