SABlockFL: a blockchain-based smart agent system architecture and its application in federated learning

Abstract

Purpose

The purpose of this work is to bridge FL and blockchain technology through designing a blockchain-based smart agent system architecture and applying in FL. and blockchain technology through designing a blockchain-based smart agent system architecture and applying in FL. FL is an emerging collaborative machine learning technique that trains a model across multiple devices or servers holding private data samples without exchanging their data. The locally trained results are aggregated by a centralized server in a privacy-preserving way. However, there is an assumption where the centralized server is trustworthy, which is impractical. Fortunately, blockchain technology has opened a new era of data exchange among trustless strangers because of its decentralized architecture and cryptography-supported techniques.

Design/methodology/approach

In this study, the author proposes a novel design of a smart agent inspired by the smart contract concept. Specifically, based on the proposed smart agent, a fully decentralized, privacy-preserving and fair deep learning blockchain-FL framework is designed, where the agent network is consistent with the blockchain network and each smart agent is a participant in the FL task. During the whole training process, both the data and the model are not at the risk of leakage.

Findings

A demonstration of the proposed architecture is designed to train a neural network. Finally, the implementation of the proposed architecture is conducted in the Ethereum development, showing the effectiveness and applicability of the design.

Originality/value

The author aims to investigate the feasibility and practicality of linking the three areas together, namely, multi-agent system, FL and blockchain. A blockchain-FL framework, which is based on a smart agent system, has been proposed. The author has made several contributions to the state-of-the-art. First of all, a concrete design of a smart agent model is proposed, inspired by the smart contract concept in blockchain. The smart agent is autonomous and is able to disseminate, verify the information and execute the supported protocols. Based on the proposed smart agent model, a new architecture composed by these agents is formed, which is a blockchain network. Then, a fully decentralized, privacy-preserving and smart agent blockchain-FL framework has been proposed, where a smart agent acts as both a peer in a blockchain network and a participant in a FL task at the same time. Finally, a demonstration to train an artificial neural network is implemented to prove the effectiveness of the proposed framework.

Citation

Zhang, Z., Yang, T. and Liu, Y. (2020), "SABlockFL: a blockchain-based smart agent system architecture and its application in federated learning", International Journal of Crowd Science, Vol. 4 No. 2, pp. 133-147. https://doi.org/10.1108/IJCS-12-2019-0037

1. Introduction

With the advance of internet technology and artificial intelligence, more and more people and devices are connected and interacted in a distributed manner (Vlassis, 2007). Technologies such as parallel computing and distributed processing (including distributed artificial intelligence) have emerged and keep growing rapidly over the past 20 years (Bond and Gasser, 2014; Yang et al., 2019a, 2019b).

Federated learning (FL) (Yang et al., 2019a, 2019b), also known as distributed learning or collaborative learning, was proposed by Google in 2016 (McMahan et al., 2016). The main goal of FL is to build a machine-learning model via data sets that are distributed across multiple data owners (Li et al., 2019). Actually, it is the combination of distributed computation and deep learning. In common FL scenarios, there are many participants in possession of local data samples and a central server, called parameter server, which maintains the machine learning model. The training process can be briefly described as follows.

At the beginning stage, the data samples are distributed among participants, however, which are not necessarily independent and identically distributed. Then, each participant downloads the snapshot of the machine learning model maintained by the parameter server, then it trains this model in use of local data samples to generate parameter gradients. Then, each participant uploads its local gradients to the central server. Finally, upon receiving the gradients from all participants, the central server aggregates those gradients and updates the global model. This process is repeated until the loss function convergences.

The agent concept was firstly proposed by Minsky (1987). Minsky believed that certain individuals in society can seek solutions to problems after negotiation. These individuals are called agents. Agents should be socially interactive and intelligent. As then, the concept of agent has been introduced into the field of computer science and has quickly become a research hot topic. In addition to intelligence, an agent should have many other attractive features such as autonomy, reactivity, structural distribution, sociability and coordination of cooperation (Wooldridge and Jennings, 1994; Dragoni et al., 2011). Each agent has limited capability in processing information, but through cooperating and coordinating with each other the multi-agent system could solve problems, which are nearly impossible to handle by a single individual. A multi-agent system can be regarded as a society composed by multiple agents, which is mostly organized in a distributed manner. The existing research studies mainly focus on the negotiation behavior happening among a group of autonomous agents, for example, how the distributed agents negotiate their goals; how the goal can be achieved through planning.

Meanwhile, with the fast development of blockchain technology in the past decade (Crosby et al., 2016), smart contract technology has also received much attention in both academics and industry (Delmolino et al., 2016; Liu et al., 2019). People have developed many distributed applications (DAPP) based on blockchain smart contract platforms. A smart contract program is more than a computer program that can be executed automatically but can also be a participant in the system. A smart contract can send out, receive and store information and values, and can also respond to the received information according to the predefined logic. More importantly, such a smart contract is trustworthy and reliable, as the contract is non-repudiation and transparent once it is published in a block (Kolvart et al., 2016; Kim et al., 2019).

In this work, we aim to investigate the feasibility and practicality of linking the three areas together, namely, multi-agent system, FL and blockchain. A blockchain-FL framework, which is based on a smart agent system, has been proposed in Section 2. We have made several contributions to the state-of-the-art. First of all, a concrete design of a smart agent model is proposed in Section 3, inspired by the smart contract concept in blockchain. The smart agent is autonomous and is able to disseminate, verify the information and execute the supported protocols. Based on the proposed smart agent model, a new architecture composed by these agents is formed, which is a blockchain network. Then, a fully decentralized, privacy-preserving and smart agent blockchain-FL framework (SABlockFL) has been proposed in Section 4, where a smart agent acts as both a peer in a blockchain network and a participant in a FL task at the same time. Finally, a demonstration to train an artificial neural network is implemented to prove the effectiveness of our proposed framework in Section 5.

2. Related work

2.1 Federated learning

To achieve higher accuracy, deep learning models often need large amount of training data, which can be solved by using distributed learning (Chilimbi et al., 2014). Among these existing studies, FL is the widely adopted method in recent years, and it is also an effective solution for high scale secure multi-party machine learning scenarios where clients train a shared model under the organization of a secure central server.

However, the classical FL framework presents some date privacy vulnerabilities, even though the training dataset is divided and stored separately (Accountability Act, 1996; Ben-Or and Hassidim, 2005). For instance, an adversary poses as an honest client and tries to steal sensitive training data through careful observation and isolation of victims’ model updates (Song et al., 2017; Melis et al., 2018). On the other hand, the authors in Hitaj et al. (2017) pointed out that a curious parameter server can learn private data through generative adversarial network learning. Therefore, has tried to solve it through replacing the parameter server with a blockchain network, which is called blockchain-federate learning (BlockFL) (Shayan et al., 2018). Similar systems are already emerging in the internet of things (IoT) field (Lu et al., 2019; Zhao et al., 2019).

2.2 Multi-agent system

Multi-agent systems can be classified into two categories according to their architecture, namely, internal architecture and overall agent structure (Balaji and Srinivasan, 2010). The internal architecture can also be further divided into homogeneous and heterogeneous structures. The overall agent structure can also be divided into hierarchical, holographic and alliance structure.

There are many examples in using multi-agent systems to construct novel applications. For example, an agent-based multi-natural language anti-spam (MNLAS) model is proposed (Mohammed et al., 2018). The MNLAS model considers visual and text information in emails during the spam filtering process. The model is composed by five types of agents, namely, the formal abbreviation agent, the feature extraction agent, the feature selection agent, the instance representation agent and the mail classification agent, and these agents complete different tasks collaboratively.

Multi-agent systems have also been widely applied in the development of distributed systems, aiming to achieve high system reliability, good availability, openness, resource sharing and re-usability. A distributed object-based agent framework for building enterprise DAPP is proposed by Zhou et al. (2000). A proxy communication language KQML is introduced in this framework, which is feasible for fast and easy development of a distributed system. This work also provides a general distributed architecture for multi-agent systems.

2.3 Smart contract and agent

The smart contract concept enables a blockchain (Nakamoto et al., 2008; Wood et al., 2014) system programmable, which provides the possibility of application with rich functionalities. A smart contract provides a trustless trust in bilateral or multi-party transactions without third-party intervention. Yuanyu Zhang studied a key access control problem on the IoT (Zhang et al., 2018). A framework based on smart contract is proposed, which consists of multiple access control contracts, a judging contract and a registration contract to implement distributed, trusted access control in IoT systems.

The combination of smart contracts and agents can achieve high security in providing an intelligent service. Aleksandr builds a communication system between agents in a peer to peer network to enable multi-agents and smart contracts to work together (Kapitonov et al., 2017). Based on this communication method, an architecture of an autonomous business activity protocol is proposed, toward to a self-managed unmanned aerial vehicles (UAVs) system where the UAVs transportation and airspace route management are conducted in an automatic manner. Different from the above work, our study aims to design a smart agent-based on smart contract spirit.

3. Smart agent and smart agent system

The conventional distributed computing technology is based on the classic client/server computing model, which decomposes a distributed application into two parts, namely, the client side and the server side. The server side can only provide services after receiving requests from the client side. These technologies are formed in the context of applications that share distributed resources and bear many limitations, especially in DAPP. In this section, we are to propose a new design of smart agent based on the smart contract concept.

For a smart agent, it should own the basic characteristics of an intelligent agent such as autonomy. Moreover, a smart agent should also have the features of a smart contract, for example, automatic execution once certain conditions are satisfied. In our design, when the condition that supports the occurrence of a certain activity is valid, the smart agent autonomously selects the corresponding business based on its own state. Moreover, the smart agents support the distributed operation mode, providing convenience for the development of DAPP.

3.1 Design of a smart agent

By comparing the characteristics of agents and smart contracts, we find that both of them are autonomous and can complete the established functions without the participation of third parties. At the same time, both agents and smart contracts have their own distinguishing characteristics. Our design will combine agents and smart contract through implementing the functions of smart contract in agents, so as to autonomously disseminate, verify the information and execute supported protocols.

The main components and their relationship are showed in Figure 1. In Figure 1, a smart agent is composed by six basic functional elements, namely, perception tool (PT), logical distribution (LD), business process (BP) and contract agreement (CA), as well as address management (AM) and data access tool (DAT). Each element is specified as follows:

• Perception tool (PT): The PT is the administrator of the system interacting with the external environment. In a realistic system, the number of instances/entities can increase or decrease according to the actual situation. The main duty of a PT includes: collecting system environment changes, making corresponding judgments for different changes and providing basic data for business processing.

• Logical distribution (LD): It is the work manager of a smart agent and there is only one running instance in a running agent. It is mainly in charge of distributing the tasks to different business processing instances according to the sensed data and collected requirements, performing the corresponding business processes and receiving the BP results.

• Business process (BP): It is the basic executive unit of a smart agent. To provide a certain service, an agent can instantiate multiple BPs simultaneously. The main functions include: receiving the message/data sent by its LD module, completing system-defined actions according to different input requirements and returning processing results to its LD module.

• Contract agreement (CA): It defines the communication protocol between different agents, and only one instance is allowed in a running agent. As a contract maker for multi-party communication within a distributed system, its main task is to develop communication rules for distributed parties and make the communication process standardized.

• Address management (AM) and data access tool (DAT): To enable the communication between distributed agents and exchange information with the blockchain ledger, a smart agent also requires AM and (DAT).

Based on the above design of a smart agent, the main features of a smart agent are summarized as follows.

• Autonomy: A smart agent has its own computing resources in BPs and behavior control mechanisms in PT and LD, which can determine and control its own behavior according to its internal state and perceived environmental information without direct external manipulation.

• Interactivity: A smart agent can implement flexible and diverse interactions with other smart agents (including people) in its address according to CA supported communication protocols. Thus, smart agents can effectively work together by interacting with each other.

• Reactivity: A smart agent can sense the environment (possibly the physical world, the user who manipulates the graphical interface or other agents) by PT and respond to the relevant events in a run-time manner via BPs.

• Initiative: An agent is able to behave properly so as to achieve the pre-defined goals in LD, presenting the goal-oriented initiative behavior.

• Collaboration: The collaboration of multiple agents can solve complex problems that are difficult or impossible to solve by a single entity. A smart agent achieves this feature through LD and CA models.

• Distribution: Information about the entire agent system, including data and control, is logically or physically distributed, without centralized control or global data storage. Each agent can solve problems in parallel, which improves the efficiency of a system composed of a set of smart agents.

With the above features of smart agents, an intelligent system can be realized. Next, we are to introduce a general architecture for smart agent systems.

3.2 Architecture of smart agent systems

When using smart agents to form a decentralized system, the business requirements in the system need to be designed into different BPs and communication contracts should be designed in CA according to the corresponding business. At the same time, the system environment that needs to be perceived is designed in PT, and the corresponding processing method is set in LD.

By deploying the smart agent in each node of the system network, an agent-to-agent distributed and the decentralized system is constructed. The network topology diagram is shown in Figure 2. It is noted that the blockchain network is also composed by the smart agents and each agent saves a replica of the complete blockchain ledger.

When the system network is built, the entire system can spontaneously handle the operations involved in the system at runtime.

4. Smart agent blockchain-based federated learning

In this section, a demonstration based on the above smart agent system of training FL neural network is implemented. The goal of the system is to effectively train a neural network and protect the local data in a proper manner. Intermediate gradients are stored in the blockchain, making our systems decentralized, reliable and privacy-preserving.

The neural network training system consists of smart agents where each agent is composed by six main components, namely, environmental perception (PT), local server (LD), model trainer (BP), partner control (CA), key management (AM) and chain pipe (DAT). These components correspond to a smart agent design in Section 3.

4.1 The module design

A user i is trying to join in our system, and it cannot do anything until it passes the authentication that it owns the private key Key_i in the blockchain network. After registration, a unique identity Id_i generated by Key_i is assigned to the user. From then on, once the user wants to interact with our system, its private key is the only valid credential. Next, we are going to introduce the detailed design of the main components and process procedures.

4.1.1 Environment perception.

This module is equivalent to PT in our proposed framework, and it provides the smart agent with communication interfaces with external environments. External environments will send many instructions to the agent, including receiving hyperparameters and begin the training, sending final model and destroy the model. PT is responsible for listening the information from external environments and forwarding it to LD.

4.1.2 Local server.

This module is equivalent to LD in our proposed framework. In a traditional FL task, we need a central server to aggregate the gradients from all participants in an iteration. However, in our design, the central server is substituted with a blockchain network. Besides, the LD is also in charge of gradient generation and model update procedure.

4.1.3 Model trainer.

This module is equivalent to BP in our proposed framework. As a basic executive unit of the smart agent, it implements gradients generation and model update, which will be specifically described in the next section.

4.1.4 Partner control.

This module is equivalent to CA in our proposed framework. Its main function is to communicate with other smart agents via generating the authentication token Aum of the intermediate gradients for the authenticated users set. It is well known that the machine learning model has commercial value. So if we want to put it on the market, the authenticated users set should have good a scalability to add or remove.

4.1.5 Key management.

This module is equivalent to AM in our proposed framework. It has two functions. On one hand, it is in charge of AM where each smart agent has a pair of keys, i.e. public key Id and privacy key Key. On the other hand, a user who wants to join the FL is also required to pass the authentication to ensure the user has a valid private key in the blockchain network.

4.1.6 Chain pipe.

This module is equivalent to a DAT, and it provides smart agents with two interfaces to interact with the blockchain, namely, writing and reading data from the blockchain.

4.2 The main procedure design

In this section, gradient generation protocol, block generation protocol and model update protocol are specifically designed.

The FL under study is operated by a set of data owners, we might as well call them smart agents here. Considering there is a set of smart agents A = {1, 2, 3,…,N_A} with |A| = N_A, the i-th smart agent owns a set of data samples S_i with |S_i| = N_i. All data samples S = ∪i=1NASi with |S| = N_S. The k-th data sample s_k ∈ S is given as s_k = {x_k, y_k} for a d-dimensional column vector x_k ∈ ℝ^d and a scalar value y_k ∈ ℝ. Global model has a loss function and there are many kinds of loss functions such as cross entropy. Without loss of generality, the mean square error algorithm is chosen as the loss function:

(1) l(w)=1NS(∑i=1NA∑sk∈Si12(xkTw−yk)2)

The training of a federated model consists of multiple steps called iteration. In i-th iteration, smart agents use their local training data and their last snapshots of the global model state w_t to compute the gradient of the loss function ∇l(w_i,x_i,y_i) to upload it to the central server, then the server uses the gradients to update the global model. The global model are updated at each iteration i as follows:

(2) wt+1=wt−ηt(λwt+1Nt∑(xi,yi)∈At∇l(wt,xi,yi))

where η_t indicates a degrading learning rate and λ is a regularization parameter that prevents over-fitting.

The method of stochastic gradient descent (SGD) is a general learning algorithm that can be used to train various models, including deep learning, This is the reason of choosing SGD based FL in our study. If the SGD has run for a fixed number of iterations or the magnitude of the gradient has fallen below a threshold, the model training then finishes and the global model is considered as the optimal resulting model w*.

In traditional FL, the global update is computed by the server, and in our proposed framework the server entity is replaced with a blockchain network.

4.2.1 Gradient generation procedure.

After a BP receives the iteration number i from LD, as mentioned above, it fetches the local data and its last snapshot of the global model state and computes the gradient of loss function ∇l(w_i,x_i,y_i). Then, ∑(xi, yi)∈At∇l(wt,xi,yi) will be returned to associated LD and the CA will generate tokens Aum(ID, i) and distribute them to the authenticated users set. The time sequence of the gradient generation protocol is shown in Figure 3.

4.2.2 Block generation procedure.

In our proposed SABlockFL framework, the blocks and their verification are designed so as to exchange parameters updates truthfully.

At first, the LD will use tokens issued by its and other smart agents’ CA to get gradients, which will be packaged into a block. The LD has a function of mining a new block in this procedure. Each block consist of its head and body, and the body stores the gradients, as well as its digital signature. The header contains the output value of the POW, block generation rate and a pointer to the previous block.

The process of mining a block is similar to that in the BitCoin (Nakamoto et al., 2008). The miner who is also the LD randomly generates a hash value by changing its input number, i.e. nonce, until hash value is smaller than a given difficulty value. Once the miner M succeeds in finding a suitable nonce, the miner then generate a new block and propagate it to all the other miners, i.e. other LD. Those miners who receive the generated block are verified and valid blocks are to be added to their local ledgers via DAT. The time sequence of the block generation is shown in Figure 4.

4.2.3 Model update procedure.

As soon as the block is added to the ledger, the BP will be awakened by a signal. It will read the block from its ledger and compute equation (2) to update its local model. The time sequence of this procedure is shown in Figure 5.

The procedure will not stop until the SGD has run for a fixed number of iterations or the magnitude of the gradient has fallen below a threshold.

4.3 Implementation of smart agent blockchain-based federated learning system

We use JADE 4.5.0, which compiles with FIPA specification to simulate a multi-agent system. In the developed system, each node is equipped with six functions in the six modules correspondingly. The agent information running on each node is shown as in Figure 6.

We implement the designed framework in an Ethereum development platform to present its effectiveness. Based on the implement SABlockFL system, we are going to demonstrate how an artificial neural network is trained, which will be used to solve the classical XOR problem (Miller et al., 1989).

In the PT, we implement the interface to get hyperparameters from the environment. The LD has three behavours, namely, getting intermediate gradients from BP, packing gradients into a candidate block and mining a new block. The AM has two main functions, namely, generating identity authentication and verifying identity authentication.

We use Ganache [1], which is a private Ethereum blockchain client for developers, to simulate a local blockchain network with 10 nodes, where the rate of block generation is 1 block per second. Their accounts and private keys are as follow (Figure 7).

The parameter settings are summarized in Table 1, which are created by a FL initiator to PT and will be recorded in the genesis block.

After the system parameters are passed to LD by the PT, the smart agent proceed to pass an authentication process. Now, that our smart agents network and the blockchain network are overlapped, and each smart agent has no access to the blockchain network until it owns a private key of these nodes and passes the authentication executed by AM as shown in Figure 8.

Next, let us begin to train our neural network. The dataset is distributed among nodes. When the smart agent has been proved to be a legitimate user, its LD instantiates a BP to compute the intermediate gradient i.e. ∇l(w_i,x_i,y_i) via local data and sends it as a transaction to Ganache. For instance, the 4th node gets an intermediate gradient and sends it out, then receives a transaction receipt. This transaction process is shown in Figure 9.

At the same time, the global model can only be accessed by those agents with legitimate private keys as justified by the training of the model. The process of accessing the global model is shown in Figure 10.

Finally, let us take a look at the effectiveness of our system. After five thousand iterations, the loss function is gradually close to zero as presented in Figure 11, which means the global model has converged.

At the same time, the accuracy of the model is also satisfactory, which can be seen in Figure 12. Although the XOR problem is simple, however, it proves that our model is useful and feasible. We can observe that multiple smart agents interact well with each other to achieve the designed functionalities, through following the proposed architecture. Because of the combination of smart agent, blockchain technology and FL, the deployed system has achieved many attractive features, for example, intelligence, well flexibility and high security.

5. Conclusion and future work

Inspired by the concept of smart contract, this work proposes a smart agent model. A smart agent can actively handle users’ adaptive needs and void suffering from the over monitor/control of the third party. Based on this smart agent system, a SABlockFL framework has been proposed, which is not under the vulnerability of the central server’s information leakage attack. Finally, we demonstrate the application of the proposed architecture in training a machine learning task in solving a neural network for classic XOR problem. The performance of our designed framework shows that the proposed smart agent architecture is practical and effective in developing secure distributed application.

The application of blockchain technology has brought another advantage that we can send rewards to the participants and design an incentive mechanism (Kang et al., 2019) to encourage them to contribute quality data, which will be the focus of our future work. Furthermore, we also plan to evaluate the proposed framework in training extensive machine learning tasks such as MINIST and CIFAR. We also plan to use some secure multiple party computation technologies such as homomorphic encryption techniques.

Figures

Figure 1.

Typical architecture of a smart agent

Figure 2.

Network topology of a smart agent system

Figure 3.

Time sequence of gradient generation procedure

Figure 4.

Time sequence of block generation procedure

Figure 5.

Time sequence of model update procedure

Figure 6.

Smart agent in a node

Figure 7.

Nodes in local Ethereum stimulator ganache

Figure 8.

Smart agent receives parameters

Figure 9.

Fourth smart agent sends a transaction and receives receipt

Figure 10.

Global model accessed by the 4th agent

Figure 11.

Convergence of the loss function with iterations

Figure 12.

Accuracy of global model with iterations

Table 1.

Parameter settings in SABlockFL

Parameter	Value
# of neurons in input layer	2
# of neurons in hidden layer	2
# of neurons in output layer	2
η	0.05
λ	0.1
Iteration	5,000

Note

1. www.trufflesuite.com