doc: fix dchat tutorial chapter2

doc/src/learn/dchat/creating-dchatd/accept-addr.md

@@ -0,0 +1,39 @@
+# Accept addr
+
+To deploy the `RequestHandler` and start receiving `JSON-RPC` requests,
+we'll need to configure a `JSON-RPC` accept address.
+
+We'll add a `rpc_listen` address to our `Args` struct. It will look
+like this:
+
+```rust
+{{#include ../../../../../example/dchat/dchatd/src/main.rs:args}}
+```
+
+This encodes a default `rpc_listen` address on the port `51054`. To be
+able to modify the default, we can also add `rpc_listen` to the default
+config at `../dchatd_config.toml` as follows:
+
+```toml
+# dchat toml
+
+## RPC listen address. 
+rpc_listen =["tcp://127.0.0.1:51054"]
+
+[net]
+## P2P accept addresses Required for inbound nodes.
+inbound=["tcp://127.0.0.1:51554"]
+
+## P2P external addresses. Required for inbound nodes.
+external_addr=["tcp://127.0.0.1:51554"]
+
+## Seed nodes to connect to. Required for inbound and outbound nodes.
+seeds=["tcp://127.0.0.1:50515"]
+
+## Outbound connect slots. Required for outbound nodes.
+outbound_connections = 5
+```
+
+Regenerate the config by deleting the previous one, rebuilding and
+rerunning `dchatd`. Now the `rpc_listen` address can be modified any
+time by editing the config file.

doc/src/learn/dchat/creating-dchatd/message.md

@@ -0,0 +1,24 @@
+# Creating a Message type
+
+We'll start by creating a custom `Message` type called `DchatMsg`. This is the
+data structure that we'll use to send messages between `dchat` instances.
+
+Messages on the p2p network must implement the `Message` trait. `Message` is a
+generic type that standardizes all messages on DarkFi's p2p network.
+
+We define a custom type called `DchatMsg` that implements the
+`Message` trait. We also add `darkfi::util::SerialEncodable` and
+`darkfi::util::SerialDecodable` macros to our struct definition so our
+messages can be parsed by the network.
+
+`Message` requires that we implement a method called `name`, which
+returns a `str` of the struct's name.
+
+For the purposes of our chat program, we will also define a buffer where
+we can write messages upon receiving them on the p2p network. We'll wrap
+this in a `Mutex` to ensure thread safety and an `Arc` pointer so we can
+pass it around.
+
+```rust
+{{#include ../../../../../example/dchat/dchatd/src/dchatmsg.rs:msg}}
+```

doc/src/learn/dchat/creating-dchatd/part-2.md

@@ -0,0 +1,13 @@
+# Part 2: Creating dchat
+
+Now that we've deployed a local version of the p2p network, we can start
+creating a custom protocol and message types that dchat will use to
+send and receive messages across the network.
+
+This section will cover:
+
+* The `Message` type
+* `Protocols` and the `ProtocolRegistry`
+* The `MessageSubsystem`
+* `MessageSubscription`
+* `Channel`

doc/src/learn/dchat/creating-dchatd/pong.md

@@ -0,0 +1,20 @@
+# Methods
+
+We're ready to deploy our `JsonRpcInterface`. But right now now it just
+returns `JsonError::MethodNotFound`. So before testing out the JSON-RPC,
+let's implement some methods.
+
+We'll start with a simple `pong` method that replies to `ping`.
+
+```rust
+{{#include ../../../../../example/dchat/src/rpc.rs:pong}}
+```
+
+And add it to `handle_request()`:
+
+```rust
+        match req.method.as_str() {
+            Some("ping") => self.pong(req.id, req.params).await,
+            Some(_) | None => JsonError::new(ErrorCode::MethodNotFound, None, req.id).into(),
+            }
+```

doc/src/learn/dchat/creating-dchatd/protocol-dchat.md

@@ -0,0 +1,58 @@
+# ProtocolDchat
+
+Let's start tying these concepts together. We'll define a struct called
+`ProtocolDchat` that contains a `MessageSubscription` to `DchatMsg` and a
+pointer to the `ProtocolJobsManager`. We'll also include the `DchatMsgsBuffer`
+in the struct as it will come in handy later on.
+
+```rust
+{{#include ../../../../../example/dchat/dchatd/src/protocol_dchat.rs:protocol_dchat}}
+```
+
+Next we'll implement the trait `ProtocolBase`. `ProtocolBase` requires
+two functions, `start` and `name`. In `start` we will start up the
+`ProtocolJobsManager`. `name` will return a `str` of the protocol name.
+
+```rust
+#[async_trait]
+impl net::ProtocolBase for ProtocolDchat {
+    async fn start(self: Arc<Self>, executor: Arc<Executor<'_>>) -> Result<()> {
+        self.jobsman.clone().start(executor.clone());
+        Ok(())
+    }
+
+    fn name(&self) -> &'static str {
+        "ProtocolDchat"
+    }
+}
+```
+
+Once that's done, we'll need to create a `ProtocolDchat` constructor
+that we will pass to the `ProtocolRegistry` to register our protocol.
+We'll invoke the `MessageSubsystem` and add `DchatMsg` as to the list
+of dispatchers. Next, we'll create a `MessageSubscription` to `DchatMsg`
+using the method `subscribe_msg`.
+
+We'll also initialize the `ProtocolJobsManager` and finally return a
+pointer to the protocol.
+
+```rust
+{{#include ../../../../../example/dchat/dchatd/src/protocol_dchat.rs:constructor}}
+```
+
+We're nearly there. But right now the protocol doesn't actually do
+anything. Let's write a method called `handle_receive_msg` which receives
+a message on our `MessageSubscription` and adds it to `DchatMsgsBuffer`.
+ 
+Put this inside the `ProtocolDchat` implementation:
+
+```rust
+{{#include ../../../../../example/dchat/dchatd/src/protocol_dchat.rs:receive}}
+```
+
+As a final step, let's add that task to the `ProtocolJobManager` that is invoked
+in `start`:
+
+```rust
+{{#include ../../../../../example/dchat/dchatd/src/protocol_dchat.rs:start}}
+```

doc/src/learn/dchat/creating-dchatd/protocols.md

@@ -0,0 +1,64 @@
+# Understanding Protocols
+
+We now need to implement a custom protocol which defines how our chat
+program interacts with the p2p network.
+
+We've already interacted with several protocols already. Protocols
+are automatically activated when nodes connect to eachother on the
+p2p network. Here are examples of two protocols that every node runs
+continuously in the background:
+
+* [ProtocolPing](https://github.com/darkrenaissance/darkfi/blob/master/src/net/protocol/protocol_ping.rs):
+sends `ping`, receives `pong`
+* [ProtocolAddress](https://github.com/darkrenaissance/darkfi/blob/master/src/net/protocol/protocol_address.rs):
+receives a `get_address` message, sends an `address` message
+
+Under the hood, these protocols have a few similarities:
+
+* They create a subscription to a message type, such as `ping` and `pong`.
+* They implement [ProtocolBase](https://github.com/darkrenaissance/darkfi/blob/master/src/net/protocol/protocol_base.rs),
+DarkFi's generic protocol trait.
+* They run asynchronously using the
+[ProtocolJobsManager](https://github.com/darkrenaissance/darkfi/blob/master/src/net/protocol/protocol_jobs_manager.rs).
+* They hold a pointer to [Channel](https://github.com/darkrenaissance/darkfi/blob/master/src/net/channel.rs) which
+invokes the [MessageSubsystem](https://github.com/darkrenaissance/darkfi/blob/master/src/net/message_subscriber.rs#L170).
+
+This introduces several generic interfaces that we must use to build
+our custom protocol. In particular:
+
+**The Message Subsystem**
+
+`MessageSubsystem` is a generic publish/subscribe class that contains
+a list of `Message` dispatchers. A new dispatcher is created for every
+`Message` type. These `Message` specific dispatchers maintain a list of
+susbscribers that are subscribed to a particular `Message`.
+
+**Message Subscription**
+
+A subscription to a specific `Message` type. Handles receiving messages
+on a subscription.
+
+**Channel**
+
+`Channel` is an async connection for communication between nodes. It is
+also a powerful interface that exposes methods to the `MessageSubsystem`
+and implements `MessageSubscription`.
+
+**The Protocol Registry**
+
+`ProtocolRegistry` is a registry of all protocols. We use it through the
+method `register` which passes a protocol constructor and a session
+`bitflag`. The `bitflag` specifies which sessions the protocol is created
+for. The `ProtocolRegistry` then spawns new protocols for different channels
+depending on the session.
+
+**ProtocolJobsManager**
+
+An asynchronous job manager that spawns and stops tasks. Its main
+purpose is so a protocol can cleanly close all started jobs, through
+the function `close_all_tasks`.  This way if the connection between
+nodes is dropped and the channel closes, all protocols are also shutdown.
+
+**ProtocolBase**
+
+A generic protocol trait that all protocols must implement.

doc/src/learn/dchat/creating-dchatd/register-protocol.md

@@ -0,0 +1,75 @@
+# Registering a protocol
+
+We've now successfully created a custom protocol. The next step is the
+register the protocol with the `ProtocolRegistry`.
+
+We'll define a new function inside the `Dchat` implementation called
+`register_protocol()`. It will invoke the `ProtocolRegistry` using the
+handle to the p2p network contained in the `Dchat` struct. It will then
+call `register()` on the registry and pass the `ProtocolDchat` constructor.
+
+```rust
+{{#include ../../../../../example/dchat/dchatd/src/main.rs:register_protocol}}
+```
+
+`register` takes a closure with two arguments, `channel` and `p2p`. We
+use `move` to capture these values. We then create an async closure
+that captures these values and the value `msgs` and use them to call
+`ProtocolDchat::init` in the async block.
+
+The code would be expressed more simply as:
+
+```rust
+registry.register(!net::SESSION_SEED, async move |channel, _p2p| {
+        ProtocolDchat::init(channel, msgs).await
+    })
+    .await;
+```
+
+However we cannot do this due to limitation with async closures. So
+instead we wrap the `async move` in a `move` in order to capture the
+variables needed by `ProtocolDchat::init`.
+
+Notice the use of a `bitflag`. We use `!SESSION_SEED` to specify that
+this protocol should be performed by all sessions aside from the
+seed session.
+
+Also notice that `register_protocol` requires a `DchatMsgsBuffer` that we
+send to the `ProtocolDchat` constructor. We'll create the `DchatMsgsBuffer`
+in `main` and pass it to `Dchat::new`. Let's add `DchatMsgsBuffer` to the
+`Dchat` struct definition first.
+
+```rust
+struct Dchat {
+    p2p: net::P2pPtr,
+    recv_msgs: DchatMsgsBuffer,
+}
+```
+
+And initialize it:
+
+```rust
+async_daemonize!(realmain);
+async fn realmain(args: Args, ex: Arc<smol::Executor<'static>>) -> Result<()> {
+    //...
+
+    let msgs: DchatMsgsBuffer = Arc::new(Mutex::new(vec![DchatMsg { msg: String::new() }]));
+    let mut dchat = Dchat::new(p2p.clone(), msgs);
+
+    //...
+}
+```
+
+Now try running `dchatd` with `lilith` and seeing what debug output you
+get. Keep an eye out for the following:
+
+```
+[DEBUG] (1) net: Channel::subscribe_msg() [START, command="DchatMsg", address=tcp://127.0.0.1:50105]
+[DEBUG] (1) net: Channel::subscribe_msg() [END, command="DchatMsg", address=tcp://127.0.0.1:50105]
+[DEBUG] (1) net: Attached ProtocolDchat
+```
+
+If you see that, we have successfully:
+
+* Implemented a custom `Message` and created a `MessageSubscription`.
+* Implemented a custom `Protocol` and registered it with the `ProtocolRegistry`.

doc/src/learn/dchat/creating-dchatd/rpc-requests.md

@@ -0,0 +1,41 @@
+# Handling RPC requests
+
+Let's connect `dchatd` up to `JSON-RPC` using DarkFi's [rpc
+module](https://github.com/darkrenaissance/darkfi/tree/master/src/rpc).
+
+We'll need to implement a trait called `RequestHandler` for our `Dchat`
+struct. `RequestHandler` is an async trait implementing a handler for
+incoming JSON-RPC requests. It exposes us to several methods automatically
+(including a `pong` response) but it also requires that we implement
+two methods: `handle_request` and `connections_mut`.
+
+Let's start with `handle_request`. `handle_request` is simply a
+handle for processing incoming JSON-RPC requests that takes a
+`JsonRequest` and returns a `JsonResult`. `JsonRequest` is a
+`JSON-RPC` request object, and `JsonResult` is an enum that wraps
+around a given `JSON-RPC` object type. These types are defined inside
+[jsonrpc.rs](https://github.com/darkrenaissance/darkfi/blob/master/src/rpc/jsonrpc.rs).
+
+We'll use `handle_request` to run a match statement on
+`JsonRequest.method`.
+
+Running a match on `method` will allow us to branch out to functions
+that respond to methods received over `JSON-RPC`.  We haven't implemented
+any methods yet, so for now let's just return a `JsonError`.
+
+```rust
+#[async_trait]
+impl RequestHandler for Dchat {
+    async fn handle_request(&self, req: JsonRequest) -> JsonResult {
+        if req.params.as_array().is_none() {
+            return JsonError::new(ErrorCode::InvalidRequest, None, req.id).into()
+        }
+
+        debug!(target: "RPC", "--> {}", serde_json::to_string(&req).unwrap());
+
+        match req.method.as_str() {
+            Some(_) | None => JsonError::new(ErrorCode::MethodNotFound, None, req.id).into(),
+        }
+    }
+}
+```

doc/src/learn/dchat/creating-dchatd/sending-messages.md

@@ -0,0 +1,24 @@
+# Sending messages
+
+The core of our application has been built. All that's left is to make a
+python command-line tool that takes user input and sends it to `dchatd`
+over `JSON-RPC`. 
+
+We'll implement a `JSON-RPC` method called `send` that takes some user
+data. When `dchatd` receives `send` it will create a `DchatMsg` and send
+it over the network using `p2p.broadcast`.
+
+This is `p2p.broadcast`:
+
+```rust
+/// Broadcasts a message concurrently across all active channels.
+pub async fn broadcast<M: Message>(&self, message: &M) {
+    self.broadcast_with_exclude(message, &[]).await
+}
+```
+
+`broadcast` takes a generic `Message` type and sends it across all the
+channels that our node has access to.
+
+All that's left to do is to create a python command-line tool with
+`JSON-RPC` integration.

doc/src/learn/dchat/creating-dchatd/stoppable-task.md

@@ -0,0 +1,111 @@
+# StoppableTask
+
+Implementing a `JSON-RPC` `RequestHandler` also requires that we implement
+a method called `connections_mut`. This introduces us to an important
+`darkfi` type called `StoppableTask`.
+
+`StoppableTask` is a async task that can be prematurely (and safely)
+stopped at any time. We've already encountered this method when we
+discussed `p2p.stop`, which triggers `StoppableTask` to cleanly shutdown
+any inbound, outbound or manual sessions which are running.
+
+This is the basic usage of `StoppableTask`:
+
+```rust
+    let task = StoppableTask::new();
+    task.clone().start(
+        my_method(),
+        |result| self_.handle_stop(result),
+        Error::MyStopError,
+        executor,
+    );
+```
+
+Then at any time we can call `task.stop` to close the task.
+
+To make use of this, we will need to import `StoppableTask` to `dchatd`
+and add it to the `Dchat` struct definition. We'll wrap it in a `Mutex`
+to ensure thread safety.
+
+```rust
+//...
+
+use darkfi::system::{StoppableTask, StoppableTaskPrc};
+
+//...
+
+struct Dchat {
+    p2p: net::P2pPtr,
+    recv_msgs: DchatMsgsBuffer,
+    pub rpc_connections: Mutex<HashSet<StoppableTaskPtr>>,
+}
+
+impl Dchat {
+    fn new(
+        p2p: net::P2pPtr,
+        recv_msgs: DchatMsgsBuffer,
+        rpc_connections: Mutex<HashSet<StoppableTaskPtr>>,
+    ) -> Self {
+        Self { p2p, recv_msgs, rpc_connections }
+    }
+}
+
+```
+
+We'll then add the required trait method `connections_mut` to the `Dchat`
+`RequestHandler` implementation that unlocks the `Mutex`, returning a
+`HashSet` of `StoppableTaskPtr`.
+
+```rust
+    async fn connections_mut(&self) -> MutexGuard<'_, HashSet<StoppableTaskPtr>> {
+        self.rpc_connections.lock().await
+    }
+```
+
+Next, we invoke `JSON-RPC` in the main function of `dchatd`, wielding
+`StoppableTask` to start a `JSON-RPC` server and wait for a stop signal as follows:
+
+```rust
+    info!("Starting JSON-RPC server on port {}", args.rpc_listen);
+    let msgs: DchatMsgsBuffer = Arc::new(Mutex::new(vec![DchatMsg { msg: String::new() }]));
+    let rpc_connections = Mutex::new(HashSet::new());
+    let dchat = Arc::new(Dchat::new(p2p.clone(), msgs.clone(), rpc_connections));
+    let _ex = ex.clone();
+
+    let rpc_task = StoppableTask::new();
+    rpc_task.clone().start(
+        listen_and_serve(args.rpc_listen, dchat.clone(), None, ex.clone()),
+        |res| async move {
+            match res {
+                Ok(()) | Err(Error::RpcServerStopped) => dchat.stop_connections().await,
+                Err(e) => error!("Failed stopping JSON-RPC server: {}", e),
+            }
+        },
+        Error::RpcServerStopped,
+        ex.clone(),
+    );
+
+    //...
+
+    info!("Stopping JSON-RPC server");
+    rpc_task.stop().await;
+```
+
+The method `stop_connections` is implemented by `RequestHandler`
+trait. Behind the scenes it calls the `connections_mut` method we
+implemented above, loops through the `StoppableTaskPtr`'s it returns
+and calls `stop` on them, safely closing each `JSON-RPC` connection.
+
+Notice that when we start the `StoppableTask` using
+`rpc.task.clone().start`, we also pass a method called `listen_and_serve`.
+`listen_and_serve` is a method defined in DarkFi's [rpc
+module](https://github.com/darkrenaissance/darkfi/tree/master/src/rpc/server.rs).
+It starts a JSON-RPC server that is bound to the provided accept address
+and uses our previously implemented `RequestHandler` to handle incoming
+requests.
+
+The async block uses the `move` keyword to takes ownership of
+the `accept_addr` and `RequestHandler` values and pass them into
+`listen_and_serve`.
+
+We have enabled JSON-RPC.