SubQuery

Overview
What is SubQuery?
SubQuery’s goal is to become an Omni indexer for both EVM/substrate-native smart contract infrastructure in the Polkadot ecosystem. Connecting directly to substrate with WebSocket, developers can get an insight into their smart contracts from Subquery’s indexing mechanism by detecting smart contract events in a more native way without running separate RPC nodes for HTTP connection. The insights are often used for recent trades in DEXes, yield reserves tracking in money markets, NFT transfers, and many more. In this tutorial, we will look at how to set up a substrate native indexer for frontier EVM tracking ERC20 token transfers.
SubQuery has an advantage over Graph in that it requires javascript or typescript while each subgraph requires AssemblyScript, which is a more focused native-friendly language and may have unexpected behavior. It also tracks EVM calls so that developers can still make insight out of calls without adding event code. For deployment workflow, since Subgraph is made for the public, it requires developers to add Subgraphs one by one, but Subquery just needs one command to build a dedicated indexer.
Prerequisite
Before you setup SubQuery for your platform, you need:
​Docker : Containerization platform for software solutions
​Docker-compose : Used to automate interactions between docker containers
​GraphQL: Simple knowledge on how to propose entity and query it is required
Getting started
First of all, clone the boilerplate for setting up the indexer:
1
git clone <https://github.com/AstarNetwork/astar-evm-example.git>
2
cd astar-evm-example
3
yarn
Copied!
1. Setting up typedef for connecting to parachain
To connect to parachain with websocket, type definitions, typedef in short, is used to encode data for communication. chainTypes.ts manages the manifest of the types of data that will be shared between the parachain and the indexer. As Astar.js goes through an upgrade of type definitions, chainTypes.ts has to be updated with the latest typedefs to be fully able to connect to parachain.
1
import type { OverrideBundleDefinition } from "@polkadot/types/types";
2
​
3
const definitions: OverrideBundleDefinition = {
4
    types: [
5
        {
6
            // on all versions
7
            minmax: [0, undefined],
8
            types: {
9
                Keys: "AccountId",
10
                Address: "MultiAddress",
11
                LookupSource: "MultiAddress",
12
                AmountOf: "Amount",
13
                Amount: "i128",
14
                SmartContract: {
15
                    _enum: {
16
                        Evm: "H160",
17
                        Wasm: "AccountId",
18
                    },
19
                },
20
                EraStakingPoints: {
21
                    total: "Balance",
22
                    stakers: "BTreeMap<AccountId, Balance>",
23
                    formerStakedEra: "EraIndex",
24
                    claimedRewards: "Balance",
25
                },
26
                EraRewardAndStake: {
27
                    rewards: "Balance",
28
                    staked: "Balance",
29
                },
30
                EraIndex: "u32",
31
            },
32
        },
33
    ],
34
};
35
​
36
export default { typesBundle: definitions };
Copied!
2. Setup entity for storing event topics
SubQuery indexer filters event topics from the connected parachain and then stores them in its database for search. A topic is a unit of event data on a Solidity smart contract that is used for giving updates to its state. Entity declares the shape of data which event data is stored in the indexer database through the handler. To declare an entity to store event topics, you can edit schema.graphql in the root directory with GraphQL syntax.
1
type Transfer @entity {
2
  id: ID! # Tx hash
3
​
4
  from: String!
5
  to: String!
6
  contractAddress: String!
7
  amount: BigInt!
8
  blockNumber: BigInt!
9
}
Copied!
After declaring entity, you can run command yarn codegen to generate model types for handling events in handler code. Then, src directory will have types ready for the handler to add indexing logic. types is the directory which declares data entities as models for handler to manage.
1
src/
2
├── chaintypes.ts
3
├── index.ts
4
├── mappings
5
│   └── mappingHandlers.ts
6
**└── types
7
    ├── index.ts
8
    └── models
9
        ├── Transfer.ts
10
        └── index.ts**
Copied!
3. Setup handler for indexing
Now that storage is declared, we can use handler to declare how to add data on each event emission. mappings directory stores handlers mapping solidity event topic to SubQuery data entity. In this tutorial, we will see how erc20 transfer event is handled.
1
// import model from types
2
import { Transfer } from "../types"; 
3
// contract processor library
4
import {
5
  FrontierEvmEvent,
6
} from "@subql/contract-processors/dist/frontierEvm";
7
// uint256 in js
8
import { BigNumber } from "@ethersproject/bignumber";
9
​
10
// event data declaration 
11
//[ /*topic types in order address as string */ ] & {
12
// /*
13
//   mapping of event topic args and types
14
// */
15
type TransferEventArgs = [string, string, BigNumber] & {
16
  from: string;
17
  to: string;
18
  value: BigNumber;
19
};
20
​
21
// When event occurs
22
export async function handleERC20Transfer(
23
  event: FrontierEvmEvent<TransferEventArgs>
24
): Promise<void> {
25
  logger.warn("Calling handleERC20Transfer");
26
  // fill entity with event data
27
  const transfer = Transfer.create({
28
    amount: event.args.value.toBigInt(),
29
    from: event.args.from,
30
    to: event.args.to,
31
    contractAddress: event.address,
32
    blockNumber: BigInt(event.blockNumber),
33
    id: event.transactionHash,
34
  });
35
	// save it to indexer database
36
  await transfer.save();
37
}
Copied!
SubQuery indexer can also track calls for solidity precompiles which event is hard to track.
1
import { Transfer } from "../types";
2
import {
3
  FrontierEvmCall
4
} from "@subql/contract-processors/dist/frontierEvm";
5
import { BigNumber } from "@ethersproject/bignumber";
6
​
7
type TransferCallArgs = [string, BigNumber] & {
8
  _to: string;
9
  _value: BigNumber;
10
};
11
​
12
export async function handleERC20TransferCall(
13
  call: FrontierEvmCall<TransferCallArgs>
14
): Promise<void> {
15
  logger.warn("Calling handleERC20TransferCall");
16
​
17
  const transfer = Transfer.create({
18
    amount: call.args._value.toBigInt(),
19
    from: call.from,
20
    to: call.args._to,
21
    contractAddress: call.to,
22
    id: call.hash,
23
    blockNumber: undefined,
24
  });
25
​
26
  await transfer.save();
27
}
Copied!
Once the handler is built, run yarn build to compile codes into a deployable format. Confirm dist directory is formed in the root directory.
1
astar-evm-example
2
├── LICENSE
3
├── README.md
4
**├── dist**
5
├── docker-compose.yml
6
├── erc20.abi.json
7
├── node_modules
8
├── package.json
9
├── project.yaml
10
├── schema.graphql
11
├── src
12
├── tsconfig.json
13
└── yarn.lock
Copied!
4. Deploy indexer
SubQuery indexer is a multi-container solution that runs with 3 different containers.
postgres : Database which stores all data from indexer
subquery-node : Event subscriber which detects EVM call/event from connected blockchain then writes in the database
graphql-engine : graphql engine which indexes stored data in the database
docker-compose.yml already handles specification of how they are connected, but project.yaml has to be edited to give information to subquery-node on what to track in its container.
Here is the each necessary information in project.yaml which needs to be adjusted.
1
# metadata
2
specVersion: 0.2.0 
3
name: astar-evm 
4
version: 0.0.1
5
description: This SubQuery project can be use as a starting point for Astar network
6
repository: <https://github.com/subquery/astar-subql-starters>
7
# schema directory
8
schema:
9
  file: ./schema.graphql
10
# network 
11
network:
12
  # wss endpoint
13
  endpoint: wss://astar.api.onfinality.io/public-wss 
14
  # genesis hash of connecting blockchahin
15
  genesisHash: '0x9eb76c5184c4ab8679d2d5d819fdf90b9c001403e9e17da2e14b6d8aec4029c6'
16
  # chain types directory
17
  chaintypes:
18
    file: ./dist/chaintypes.js
19
dataSources:
20
  # one kind per one contract to track
21
  - kind: substrate/FrontierEvm
22
    startBlock: 436282 # block to start tracking events
23
​
24
    assets:
25
    # Smart contract ABIs
26
      erc20: # declare ABI to refer within the file
27
        file: './erc20.abi.json' # abi file directory
28
    # Processor library for websocket data
29
    processor:
30
      file: './node_modules/@subql/contract-processors/dist/frontierEVM.js'
31
      options:
32
        abi: erc20
33
        address: '0x3d4dcfd2b483549527f7611ccfecb40b47d0c17b'
34
    # mapping for contract event
35
    mapping:
36
      file: ./dist/index.js
37
      handlers:
38
        - handler: handleFrontierEvmEvent
39
          kind: substrate/FrontierEvmEvent
40
          filter:
41
            ## Topics that follow Ethereum JSON-RPC log filters
42
            ## <https://docs.ethers.io/v5/concepts/events/>
43
            ## With a couple of added benefits:
44
            ##  - Values don't need to be 0 padded
45
            ##  - Event fragments can be provided and automatically converted to their id
46
            topics:
47
              ## Example valid values:
48
              # - '0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef'
49
              # - Transfer(address,address,u256)
50
              # - Transfer(address from,address to,uint256 value)
51
​
52
              ## Example of OR filter, will capture Transfer or Approval events
53
              # - - 'Transfer(address indexed from,address indexed to,uint256 value)'
54
              #   - 'Approval(address indexed owner, address indexed spender, uint256 value)'
55
​
56
              - Transfer(address indexed from,address indexed to,uint256 value)
57
        - handler: handleFrontierEvmCall
58
          kind: substrate/FrontierEvmCall
59
          filter:
60
            ## The function can either be the method fragment or signature
61
            # function: '0x095ea7b3'
62
            # function: '0x7ff36ab500000000000000000000000000000000000000000000000000000000'
63
            # function: approve(address,uint256)
64
            function: approve(address to,uint256 value)
65
            ## The transaction sender
66
            from: '0x6bd193ee6d2104f14f94e2ca6efefae561a4334b'
Copied!
After editing project.yaml , execute docker-compose to pull the latest containers and deploy
1
docker-compose pull && docker-compose up
Copied!
5. Exploring collected data
After you run docker-compose , SubQuery supports graphiQL playground to explore indexed data. You can explore the data by sending graphQL queries at http://localhost:3000
Glossary
Topic
A unit of event subscription data on a solidity smart contract is used for giving updates to its state.
Entity
A distinctive data structure that is used to store event topics in the database of SubQuery indexer
GraphQL
An indexing tool for aggregated data
SubSquid
Covalent
Last modified 1mo ago
Copy link
Contents
1. Setting up typedef for connecting to parachain
2. Setup entity for storing event topics
3. Setup handler for indexing
4. Deploy indexer
5. Exploring collected data
Glossary