plugins: add plugin system #30824

MariusVanDerWijden · 2024-11-28T09:03:31Z

This PR outlines an idea for a new plugin system that allows 3rd parties to build plugins that hook into go-ethereum on a node level, compared to the tracing hooks that allow to hooking into the transaction processing level. The work is inspired by Peters work on ExEx style plugins.

A plugin must implement the following methods:

type Plugin interface {
	// Setup is called on startup when the Plugin is initialized.
	Setup(chain Chain, hooks SetHooks)
	// Close is called when the geth node is torn down.
	Close()
	// NewHead is called when a new head is set.
	NewHead()
}

Which allows the node to trigger an update on the plugin whenever the state of the canonical chain changes. This way the plugin can react to new heads and reorgs. It also allows the plugin to install and un-install other hooks (right now only tracing.Hooks are supported, but this could be extended in the future to allow for other hooks).

The plugin needs to store the Chain object that is passed on startup and can use it to get information about the current state of the chain.
The chain interface provides the following functionality which allows the plugin to read the chain:

// The Chain interface allows for interacting with the chain from a plugin.
type Chain interface {
	// Head returns the number of the current head and finalized block.
	Head() (uint64, uint64)
	// Header returns a header in the canonical chain.
	Header(number uint64) *types.Header
	// Block returns a block in the canonical chain.
	Block(number uint64) *types.Block
	// Receipts returns the receipts of a block in the canonical chain.
	Receipts(number uint64) types.Receipts
	// State returns the state at a certain root.
	State(root common.Hash) State
}

The most important function is the State(root common.Hash) State which provides access to a state with a certain root. This object should not be long lived, as storing it will interfere with garbage collection. The State object provides access to data concerning a single state:

type State interface {
	// Account retrieves an account from the state.
	Account(addr common.Address) Account
	// AccountIterator creates an iterator to iterate over accounts.
	AccountIterator(seek common.Hash) snapshot.AccountIterator
	// NewAccount interprets an rlp slim account as an Account.
	NewAccount(addr common.Address, account []byte) Account
}

With this object, the plugin can iterate over all accounts and retrieve specific accounts that its interested in. Again the accounts should not be long-lived as they hold references to the state. The account object provides access to all data belonging to a specific account:

type Account interface {
	// Balance returns the balance of an account.
	Balance() *uint256.Int
	// Nonce returns the nonce of an account.
	Nonce() uint64
	// Code returns the code of an account.
	Code() []byte
	// Storage returns a storage slot.
	Storage(slot common.Hash) common.Hash
	// StorageIterator creates an iterator over the storage slots of an account.
	StorageIterator(seek common.Hash) snapshot.StorageIterator
}

holiman · 2024-11-28T12:29:07Z

We discussed this a bit on the triage today, and I'd like to write it down too.

It's easy to design an API which is fully user-centric, is very surface-level user-friendly, but in actuality very user-unfriendly.

Let's say we put the user fully in the driver-seat, giving active access to stae:

- GetState(root) -> State
  - State.IterateAccounts() -> Account
    ->Account.IterateStorage() ...

I would call this an API where the user is active.

A developer using this API can just "go wild" and query the state. However, there is one obvious and one less obvious problem.

First of all, geth typically only has access to the last couple of hundred states. So a user trying to iterate all accounts at block 1000 will simply fail.

Secondly, geth progresses. If a user obtains a recent state, and starts iterating, then after a while, that state will become 'stale' due to chain progression. So the iterators will stop working. The user is once again let down by the API, and his usecase simply fails.

To solve the latter problem, the API needs to provide stability. Which means that geth needs to pause chain progression while the state is being used by the API.

A very elegant way to achieve this, is to hook into block progression. In this case, the user is passive, asking the node to inform him when a block is being imported.

OnBlock(block, context)

Once the OnBlock callback is invoked, the user can now take an active role. The block progression cannot continue until the callback ends. The context can give access to the current state.

An API like the latter does not make false promises about data delivery.

You can only access current state, but you can access it as long as you want.
If you want to access older state, you simply have to resync block-by-block.

In both these cases, the usecase can be achieved. There is no "it failed because you do not have an archive node, tough luck".

Similarly, if we want to give access to e.g. register custom RPC handlers, we could define even higher level callbacks, where users can register handlers, open db connections and later tear them down and close files.

OnStartup( ctx ) // When geth starts up
OnShutdown( ctx )

MariusVanDerWijden added 2 commits November 28, 2024 09:36

plugins: start work on plugin system

176eafd

plugins: add SetHooks struct

b93908e

MariusVanDerWijden changed the title ~~plugins: added plugin system~~ plugins: add plugin system Nov 28, 2024

plugins/internal: better show which interface the plugin needs

73618f1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

plugins: add plugin system #30824

plugins: add plugin system #30824

MariusVanDerWijden commented Nov 28, 2024

holiman commented Nov 28, 2024

plugins: add plugin system #30824

Are you sure you want to change the base?

plugins: add plugin system #30824

Conversation

MariusVanDerWijden commented Nov 28, 2024

holiman commented Nov 28, 2024