Skip to main content

IPFS

IPFS is a decentralized protocol and peer-to-peer network that stores and shares hypermedia using content-addressing. It ensures global, permanent access to files by assigning unique cryptographic hashes as addresses, employs a distributed network for resilience, tracks file versions, organizes data with a Merkle DAG, supports offline sharing, and utilizes caching for efficient content retrieval. IPFS is widely used for decentralized file storage, web hosting, DApps, and ensuring data integrity in various domains, including blockchain and cryptocurrency.

Integrating IPFS with a Substrate-based Blockchain​

Integrating IPFS with a Substrate-based blockchain allows for decentralized storage solutions in blockchain applications. This guide outlines the process to connect to IPFS, store data, generate a unique storage key, and interact with the blockchain to store and retrieve data CIDs.

Connect to IPFS​

We will be using Helia to connect to a node. Helia gives developers the ability to run IPFS-compatible functionality without requiring a full IPFS daemon. It is a newer, modular, implementation for building IPFS-like networks. A main benefit of Helia is that it can operate in browser environments, enabling IPFS-based applications to run entirely in the browser without requiring an HTTP API connection to a remote IPFS node.

First we will need to install the libraries that Helia depends on. You can do so using npm with the following commands:

  • npm install helia - Used to import the Helia package which is used in creating a new instance of a Helia node.

  • npm install @helia/unixfs - Installs the unixfs package from Helia that is used to create a filesystem.

Import the two packages previously downloaded, connect to the node, and create a filesystem for that particular node.

import { createHelia } from 'helia';
import { unixfs } from '@helia/unixfs';

// create node
const helia = await createHelia();

// create filesystem
const fs = unixfs(helia);

Add data to Node​

Next we provide a code snippet to show how to add data to the newly created Helia node.

// convert string to Uint8Array
const encoder = new TextEncoder();

const bytes = encoder.encode('Hello peaq data');

// adds bytes to node and receives a CID back (content identifier)
const cid = await fs.addBytes(bytes);

console.log('CID of the data added:', cid.toString());

The code snippet above encodes the string data into bytes and adds it to the filesystem (fs) initialized in the previous example. Take special note of the cid that is returned back after adding your data. That value will be needed when reading back from the node.

Retrieve data from Node​

The next lines of code provides an example on how to read data back from the node using the cid that was returned previously.

// decoder converts Uint8Array to strings
const decoder = new TextDecoder()
let stored_data = ''

for await (const data of fs.cat(cid)) {
    stored_data += decoder.decode(data, {
    stream: true
  })
}

console.log('Read file contents from data store:', stored_data);

helia.stop();

Initialize Blockchain API Connection​

Next we will need to install the libraries that the Substrate-based blockchain depends on. You can do so using npm with the following commands:

  • npm install @polkadot/api - Used to import the websocket provider and api promise to communicate to the Substrate-based blockchain. Imports Keyring object to sign the write transaction to peaq storage.

  • npm install @peaq-network/types - Used to see the options that peaq offers in the types package.

In this example we will use the testnet url, agung, to connect to the Substrate blockchain using Polkadot.js API.

import { ApiPromise, WsProvider, Keyring } from '@polkadot/api';
import { defaultOptions } from '@peaq-network/types';

const provider = new WsProvider('wss://wsspc1-qa.agung.peaq.network');
const api = await ApiPromise.create({ provider, ...defaultOptions });

Add CID to peaqStorage​

Store the previously created CID with an item_type name for the data to be added in peaq Storage.

  const item_type = "user-data";
  const keyring = new Keyring({ type: 'sr25519' });

  // Add Alice to our keyring with a hard-derivation path.
  // Can add agung funded wallet here with your substrate wallet's mnemonic phrase (recommended).
  const UserPair = keyring.addFromUri('//Alice');

  await api.tx.peaqStorage.addItem(item_type, cid).signAndSend(UserPair);

Create a Unique Storage Key and Retrieve Data​

Import the following libraries that the Substrate-based blockchain uses to decode and generate a key for storage.

  • npm install @polkadot/util-crypto - Used to import the decodeAddress and blake2AsHex packages to successfully generate a storage key used in retrieving data from peaq Storage.

  • npm install @polkadot/util - Used to import polkadot-provided conversion mechanisms to read data from peaq Storage.

The following code utilizes the user's address and data type to generate a unique key for identifying stored data on the blockchain. Then it fetches the corresponding data from IPFS.

import { decodeAddress, blake2AsHex } from '@polkadot/util-crypto';
import { u8aToU8a, u8aToHex, u8aConcat, hexToString } from "@polkadot/util";


// generate storage key
const storageKeyByteArray = [];
const decodedAddress = decodeAddress(UserPair.address, false, 42);
storageKeyByteArray.push(decodedAddress);

const hashItemType = u8aToU8a(cid);
storageKeyByteArray.push(hashItemType);

const key = u8aConcat(...storageKeyByteArray);

const storageKey = blake2AsHex(key, 256);

const val = await api.query.peaqStorage.itemStore(storageKey);

// convert u8a to hex to obtain data from peaq storage
var retrieved_cid = hexToString(u8aToHex(val));

const decoder = new TextDecoder()
let stored_data = ''

for await (const data of fs.cat(retrieved_cid)) {
    stored_data += decoder.decode(data, {
    stream: true
    })
}

console.log('Read file contents from data store:', stored_data);

Putting it all together​

The code snippet provides an overview of how to use this functionality with everything put together:

import { createHelia } from 'helia';
import { unixfs } from '@helia/unixfs';
import { ApiPromise, WsProvider, Keyring } from '@polkadot/api';
import { defaultOptions } from '@peaq-network/types';
import { decodeAddress, blake2AsHex } from '@polkadot/util-crypto';
import { u8aToU8a, u8aToHex, u8aConcat, hexToString} from "@polkadot/util";

const provider = new WsProvider('wss://wsspc1-qa.agung.peaq.network');
const api = await ApiPromise.create({ provider, ...defaultOptions });

// create node
const helia = await createHelia();

// create filesystem
const fs = unixfs(helia);

async function createAddHelia() {
    // convert string to Uint8Array
    const encoder = new TextEncoder();

    const bytes = encoder.encode('Hello peaq data');

    // adds bytes to node and receives a CID back (content identifier)
    const cid = await fs.addBytes(bytes);

    console.log('CID of the data added:', cid.toString());

    helia.stop();

    return cid;
}

async function peaqStorageAdd(item_type, cid){
    const keyring = new Keyring({ type: 'sr25519' });
  
    // Add Alice to our keyring with a hard-derivation path.
    // Can add agung funded wallet here with your substrate wallet's mnemonic phrase (recommended).
    const UserPair = keyring.addFromUri('//Alice');
  
    await api.tx.peaqStorage.addItem(item_type, cid).signAndSend(UserPair);

    return UserPair;
}

async function peaqStorageRetrieve(UserPair, item_type){
    // generate storage key
    const storageKeyByteArray = [];
    const decodedAddress = decodeAddress(UserPair.address, false, 42);
    storageKeyByteArray.push(decodedAddress);

    const hashItemType = u8aToU8a(item_type);
    storageKeyByteArray.push(hashItemType);

    const key = u8aConcat(...storageKeyByteArray);

    const storageKey = blake2AsHex(key, 256);

    const val = await api.query.peaqStorage.itemStore(storageKey);

    // convert u8a to hex to obtain data from peaq storage
    var retrieved_cid = hexToString(u8aToHex(val));

    return retrieved_cid
}

async function readHelia(returned_cid){
    // decoder converts Uint8Array to strings
    const decoder = new TextDecoder()
    let stored_data = ''

    for await (const data of fs.cat(returned_cid)) {
        stored_data += decoder.decode(data, {
        stream: true
        })
    }

    console.log('Read file contents from data IPFS store:\n', stored_data);
}

async function main() {
    const cid = await createAddHelia();
    const item_type = 'user-data-1';
    const UserPair = await peaqStorageAdd(item_type, cid);

    await sleep(25000); // 25 second delay to guarantee it has been added
    
    const returned_cid = await peaqStorageRetrieve(UserPair, item_type); // may have to wait until block has been appended before reading
    await readHelia(returned_cid);

    // disconnect to terminate the process
    await api.disconnect();
}

function sleep(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
}

main();



This guide provides a foundational approach for leveraging IPFS with Substrate-based blockchains, emphasizing decentralized storage solutions and data integrity within blockchain applications. For any addition questions please see the Helia Documentation.

Previous
Storage
Next
Mongo-DB