dchat: remove deleted files and add new ones

doc/src/learn/dchat/creating-dchat-cli/part-3.md

@@ -0,0 +1,3 @@
+# Part 3: Creating dchat-cli
+
+TODO

doc/src/learn/dchat/creating-dchat/message.md

@@ -1,24 +0,0 @@
-# Creating a Message type
-
-We'll start by creating a custom `Message` type called `DchatMsg`. This is the
-data structure that we'll use to send messages between `dchat` instances.
-
-Messages on the p2p network must implement the `Message` trait. `Message` is a
-generic type that standardizes all messages on DarkFi's p2p network.
-
-We define a custom type called `DchatMsg` that implements the
-`Message` trait. We also add `darkfi::util::SerialEncodable` and
-`darkfi::util::SerialDecodable` macros to our struct definition so our
-messages can be parsed by the network.
-
-`Message` requires that we implement a method called `name()`, which
-returns a str of the struct's name.
-
-For the purposes of our chat program, we will also define a buffer where
-we can write messages upon receiving them on the p2p network. We'll wrap
-this in a `Mutex` to ensure thread safety and an `Arc` pointer so we can
-pass it around.
-
-```rust
-{{#include ../../../../../example/dchat/src/dchatmsg.rs:msg}}
-```

doc/src/learn/dchat/creating-dchat/part-2.md

@@ -1,13 +0,0 @@
-# Part 2: Creating dchat
-
-Now that we've deployed a local version of the p2p network, we can start
-creating a custom protocol and message types that dchat will use to
-send and receive messages across the network.
-
-This section will cover:
-
-* The `Message` type
-* `Protocols` and the `ProtocolRegistry`
-* The `MessageSubsystem`
-* `MessageSubscription`
-* `Channel`

doc/src/learn/dchat/creating-dchat/protocol-dchat.md

@@ -1,59 +0,0 @@
-# ProtocolDchat
-
-Let's start tying these concepts together. We'll define a struct called
-`ProtocolDchat` that contains a `MessageSubscription` to `DchatMsg` and a
-pointer to the `ProtocolJobsManager`. We'll also include the `DchatMsgsBuffer`
-in the struct as it will come in handy later on.
-
-```rust
-{{#include ../../../../../example/dchat/src/protocol_dchat.rs:protocol_dchat}}
-```
-
-Next we'll implement the trait `ProtocolBase`. `ProtocolBase` requires
-two functions, `start()` and `name()`. In `start()` we will start up the
-`ProtocolJobsManager`. `name()` will return a `str` of the protocol name.
-
-```rust
-#[async_trait]
-impl net::ProtocolBase for ProtocolDchat {
-    async fn start(self: Arc<Self>, executor: Arc<Executor<'_>>) -> Result<()> {
-        self.jobsman.clone().start(executor.clone());
-        Ok(())
-    }
-
-    fn name(&self) -> &'static str {
-        "ProtocolDchat"
-    }
-}
-```
-
-Once that's done, we'll need to create a `ProtocolDchat` constructor
-that we will pass to the `ProtocolRegistry` to register our protocol.
-We'll invoke the `MessageSubsystem` and add `DchatMsg` as to the list
-of dispatchers. Next, we'll create a `MessageSubscription` to `DchatMsg`
-using the method `subscribe_msg()`.
-
-We'll also initialize the `ProtocolJobsManager` and finally return a
-pointer to the protocol.
-
-```rust
-{{#include ../../../../../example/dchat/src/protocol_dchat.rs:constructor}}
-}
-```
-
-We're nearly there. But right now the protocol doesn't actually do
-anything. Let's write a method called `handle_receive_msg()` which receives
-a message on our `MessageSubscription` and adds it to `DchatMsgsBuffer`.
- 
-Put this inside the `ProtocolDchat` implementation:
-
-```rust
-{{#include ../../../../../example/dchat/src/protocol_dchat.rs:receive}}
-```
-
-As a final step, let's add that task to the `ProtocolJobManager` that is invoked
-in `start()`:
-
-```rust
-{{#include ../../../../../example/dchat/src/protocol_dchat.rs:start}}
-```

doc/src/learn/dchat/creating-dchat/protocols.md

@@ -1,64 +0,0 @@
-# Understanding Protocols
-
-We now need to implement a custom protocol which defines how our chat
-program interacts with the p2p network.
-
-We've already interacted with several protocols already. Protocols
-are automatically activated when nodes connect to eachother on the
-p2p network. Here are examples of two protocols that every node runs
-continuously in the background:
-
-* [ProtocolPing](https://github.com/darkrenaissance/darkfi/blob/master/src/net/protocol/protocol_ping.rs):
-sends `ping`, receives `pong`
-* [ProtocolAddress](https://github.com/darkrenaissance/darkfi/blob/master/src/net/protocol/protocol_address.rs):
-receives a `get_address` message, sends an `address` message
-
-Under the hood, these protocols have a few similarities:
-
-* They create a subscription to a message type, such as `ping` and `pong`.
-* They implement [ProtocolBase](https://github.com/darkrenaissance/darkfi/blob/master/src/net/protocol/protocol_base.rs),
-DarkFi's generic protocol trait.
-* They run asynchronously using the
-[ProtocolJobsManager](https://github.com/darkrenaissance/darkfi/blob/master/src/net/protocol/protocol_jobs_manager.rs).
-* They hold a pointer to [Channel](https://github.com/darkrenaissance/darkfi/blob/master/src/net/channel.rs) which
-invokes the [MessageSubsystem](https://github.com/darkrenaissance/darkfi/blob/master/src/net/message_subscriber.rs#L170).
-
-This introduces several generic interfaces that we must use to build
-our custom protocol. In particular:
-
-**The Message Subsystem**
-
-`MessageSubsystem` is a generic publish/subscribe class that contains
-a list of `Message` dispatchers. A new dispatcher is created for every
-`Message` type. These `Message` specific dispatchers maintain a list of
-susbscribers that are subscribed to a particular `Message`.
-
-**Message Subscription**
-
-A subscription to a specific `Message` type. Handles receiving messages
-on a subscription.
-
-**Channel**
-
-`Channel` is an async connection for communication between nodes. It is
-also a powerful interface that exposes methods to the `MessageSubsystem`
-and implements `MessageSubscription`.
-
-**The Protocol Registry**
-
-`ProtocolRegistry` is a registry of all protocols. We use it through the
-method `register()` which passes a protocol constructor and a session
-`bitflag`. The `bitflag` specifies which sessions the protocol is created
-for. The `ProtocolRegistry` then spawns new protocols for different channels
-depending on the session.
-
-**ProtocolJobsManager**
-
-An asynchronous job manager that spawns and stops tasks. Its main
-purpose is so a protocol can cleanly close all started jobs, through
-the function `close_all_tasks()`.  This way if the connection between
-nodes is dropped and the channel closes, all protocols are also shutdown.
-
-**ProtocolBase**
-
-A generic protocol trait that all protocols must implement.

doc/src/learn/dchat/creating-dchat/register-protocol.md

@@ -1,101 +0,0 @@
-# Registering a protocol
-
-We've now successfully created a custom protocol. The next step is the
-register the protocol with the `ProtocolRegistry`.
-
-We'll define a new function inside the `Dchat` implementation called
-`register_protocol()`. It will invoke the `ProtocolRegistry` using the
-handle to the p2p network contained in the `Dchat` struct. It will then
-call `register()` on the registry and pass the `ProtocolDchat` constructor.
-
-```rust
-{{#include ../../../../../example/dchat/src/main.rs:register_protocol}}
-```
-
-There's a lot going on here. `register()` takes a closure with two
-arguments, `channel` and `p2p`. We use `move` to capture these values. We
-then create an async closure that captures these values and the value
-`msgs` and use them to call `ProtocolDchat::init()` in the async block.
-
-The code would be expressed more simply as:
-
-```rust
-registry.register(!net::SESSION_SEED, async move |channel, _p2p| {
-        ProtocolDchat::init(channel, msgs).await
-    })
-    .await;
-```
-
-However we cannot do this due to limitation with async closures. So
-instead we wrap the `async move` in a `move` in order to capture the
-variables needed by `ProtocolDchat::init()`.
-
-Notice the use of a `bitflag`. We use `!SESSION_SEED` to specify that
-this protocol should be performed by all sessions aside from the
-seed session.
-
-Also notice that `register_protocol()` requires a `DchatMsgsBuffer` that we
-send to the `ProtocolDchat` constructor. We'll create the `DchatMsgsBuffer`
-in `main()` and pass it to `Dchat::new()`. Let's add `DchatMsgsBuffer` to the
-`Dchat` struct definition first.
-
-```rust
-{{#include ../../../../../example/dchat/src/main.rs:dchat}}
-```
-
-And initialize it:
-
-```rust
-#[async_std::main]
-async fn main() -> Result<()> {
-    //...
-
-    let msgs: DchatMsgsBuffer = Arc::new(Mutex::new(vec![DchatMsg { msg: String::new() }]));
-
-    let mut dchat = Dchat::new(p2p.clone(), msgs);
-
-    //...
-    let (_, result) = Parallel::new()
-        .each(0..nthreads, |_| smol::future::block_on(ex2.run(shutdown.recv())))
-        .finish(|| {
-            smol::future::block_on(async move {
-                dchat.start(ex3).await?;
-                drop(signal);
-                Ok(())
-            })
-        });
-
-    result
-}
-```
-
-Finally, call `register_protocol()` in `dchat::start()`:
-
-```rust
-    async fn start(&mut self, ex: Arc<Executor<'_>>) -> Result<()> {
-        let ex2 = ex.clone();
-
-        self.register_protocol(self.recv_msgs.clone()).await?;
-        self.p2p.clone().start(ex.clone()).await?;
-        ex2.spawn(self.p2p.clone().run(ex.clone())).detach();
-
-        self.p2p.stop().await;
-
-        Ok(())
-    }
-
-```
-Now try running Alice and Bob and seeing what debug output you get. Keep
-an eye out for the following:
-
-```
-[DEBUG] (1) net: Channel::subscribe_msg() [START, command="DchatMsg", address=tcp://127.0.0.1:55555]
-[DEBUG] (1) net: Channel::subscribe_msg() [END, command="DchatMsg", address=tcp://127.0.0.1:55555]
-[DEBUG] (1) net: Attached ProtocolDchat
-```
-
-If you see that, we have successfully:
-
-* Implemented a custom `Message` and created a `MessageSubscription`.
-* Implemented a custom `Protocol` and registered it with the `ProtocolRegistry`.
-

doc/src/learn/dchat/creating-dchat/sending-messages.md

@@ -1,29 +0,0 @@
-# Sending messages
-
-The core of our application has been built. All that's left is to add a UI
-that takes user input, creates a `DchatMsg` and sends it over the network.
-
-Let's start by creating a `send()` function inside `Dchat`. This will
-introduce us to a new p2p method that is essential to our chat app:
-`p2p.broadcast()`.
-
-```
-{{#include ../../../../../example/dchat/src/main.rs:send}}
-```
-
-We pass a `String` called msg that will be taken from user input. We use
-this input to initialize a message of the type `DchatMsg` that the network
-can now support. Finally, we pass the message into `p2p.broadcast()`.
-  
-Here's what happens under the hood:
-
-```rust
-{{#include ../../../../../src/net/p2p.rs:broadcast}}
-```
-
-This is pretty straightforward: `broadcast()` takes a generic `Message` type
-and sends it across all the channels that our node has access to.
-
-All that's left to do is to create a UI.
-
-

doc/src/learn/dchat/creating-dchat/ui.md

@@ -1,42 +0,0 @@
-# Slap on a UI
-
-We'll create a new method called `menu()` inside the `Dchat`
-implementation. It implements a highly simple UI that allows a user to
-send messages and see received messages inside the inbox. Our inbox
-simply displays the messages that `ProtocolDchat` has saved in the
-`DchatMsgBuffer`.
-
-Here's what it should look like:
-
-```rust
-{{#include ../../../../../example/dchat/src/main.rs:menu}}
-```
-
-We'll call `menu()` inside of `dchat::start()` along with our other methods, like so:
-
-```rust
-    async fn start(&mut self, ex: Arc<Executor<'_>>) -> Result<()> {
-        let ex2 = ex.clone();
-
-        self.register_protocol(self.recv_msgs.clone()).await?;
-        self.p2p.clone().start(ex.clone()).await?;
-        self.p2p.clone().run(ex.clone()).await?;
-
-        self.menu().await?;
-
-        self.p2p.stop().await;
-        Ok(())
-    }
-```
-
-But wait- if you try running this code, you'll notice that the menu never
-gets displayed. That's because we call `.await` on the previous function
-call, `p2p.run()`. `p2p.run()` is a loop that runs until we exit the program,
-so in order for it to not block other threads from executing we'll need
-to detach it in the background.
-
-The complete implementaion looks like this:
-
-```rust
-{{#include ../../../../../example/dchat/src/main.rs:start}}
-```

doc/src/learn/dchat/creating-dchat/using-dchat.md

@@ -1,38 +0,0 @@
-# Using dchat
-
-We are finally ready to test our program. Spin up 5 different terminals.
-
-In terminal 1, run `lilith`.
-
-```
-./lilith
-```
-
-In terminal 2, run Alice.
-
-```
-cargo run a 
-```
-
-In terminal 3, run Bob.
-
-```
-cargo run b
-```
-
-In terminal 4, display Alice's debug output.
-
-```
-multitail -c /tmp/alice.log
-```
-
-In terminal 5, display Bob's debug output.
-
-```
-multitail -c /tmp/bob.log
-```
-
-Now use the UI to send messages between Alice and Bob. We have
-successfully implemented a p2p chat program.
-
-

doc/src/learn/dchat/creating-dchatd/rpc-methods.md

@@ -0,0 +1,3 @@
+# Adding methods
+
+TODO

doc/src/learn/dchat/deployment/start-stop.md

@@ -0,0 +1,121 @@
+# Start-Stop
+
+Now that we have initialized the network settings we can create an
+instance of the p2p network.
+
+We will next create a `Dchat` struct that will store all the data required
+by dchat. For now, it will just hold a pointer to the p2p network.
+
+```rust
+struct Dchat {
+    p2p: net::P2pPtr,
+}
+
+impl Dchat {
+    fn new(
+        p2p: net::P2pPtr,
+    ) -> Self {
+        Self { p2p }
+    }
+}
+```
+
+Let's build out our `realmain` function as follows:
+
+```rust
+use log::info;
+
+async_daemonize!(realmain);
+async fn realmain(args: Args, ex: Arc<smol::Executor<'static>>) -> Result<()> {
+    let p2p = net::P2p::new(args.net.into(), ex.clone()).await;
+    info!("Starting P2P network");
+    p2p.clone().start().await?;
+
+    let (signals_handler, signals_task) = SignalHandler::new(ex)?;
+    signals_handler.wait_termination(signals_task).await?;
+    info!("Caught termination signal, cleaning up and exiting...");
+
+    info!("Stopping P2P network");
+    p2p.stop().await;
+
+    Ok(())
+}
+```
+
+Here, we instantiate the p2p network using `P2p::new`. We then start it
+by calling `start`, and handle shutdown signals to safely shutdown the
+network using `P2p::stop`.
+
+Let's take a quick look at the underlying p2p methods we're using here.
+
+## Start
+
+This is [start](https://github.com/darkrenaissance/darkfi/blob/master/src/net/p2p.rs#L126):
+
+```rust
+/// Starts inbound, outbound, and manual sessions.
+pub async fn start(self: Arc<Self>) -> Result<()> {
+    debug!(target: "net::p2p::start()", "P2P::start() [BEGIN]");
+    info!(target: "net::p2p::start()", "[P2P] Starting P2P subsystem");
+
+    // First attempt any set manual connections
+    for peer in &self.settings.peers {
+        self.session_manual().connect(peer.clone()).await;
+    }
+
+    // Start the inbound session
+    if let Err(err) = self.session_inbound().start().await {
+        error!(target: "net::p2p::start()", "Failed to start inbound session!: {}", err);
+        self.session_manual().stop().await;
+        return Err(err)
+    }
+
+    // Start the outbound session
+    self.session_outbound().start().await;
+
+    info!(target: "net::p2p::start()", "[P2P] P2P subsystem started");
+    Ok(())
+}
+```
+
+`start` attempts to start an `Inbound`, `Manual` or `Outbound` session,
+which will succeed or fail depending on how your TOML is configured. For
+example, if you are an outbound node, `session_inbound.start()` will
+return with the following message:
+
+```rust
+info!(target: "net", "Not configured for accepting incoming connections.");
+```
+
+The function calls in `start` trigger the following processes:
+
+`session_manual.connect`: tries to connect to any `peer` addresses we
+have specified in the TOML, using a `Connector`.
+
+`session_inbound.start`: starts an `Acceptor` on the inbound address
+specified in the TOML, then creates and registers a `Channel` on that
+address.
+
+`session_outbound.start`: tries to establish a connection using a
+`Connector` to the number of slots we have specified in the TOML field
+`outbound_connections`. For every `Slot`, `run` tries to find a valid
+address we can connect to through `PeerDiscovery`, which loops through
+all connected channels and sends out a `GetAddr` message. If we don't
+have any connected channels, `run` performs a `SeedSync`.
+
+## Stop
+
+This is [stop](https://github.com/darkrenaissance/darkfi/blob/master/src/net/p2p.rs#L164).
+
+```rust
+/// Stop the running P2P subsystem
+pub async fn stop(&self) {
+    // Stop the sessions
+    self.session_manual().stop().await;
+    self.session_inbound().stop().await;
+    self.session_outbound().stop().await;
+}
+```
+
+`stop` transmits a shutdown signal to all channels subscribed to the
+stop signal and safely shuts down the network.

doc/src/learn/dchat/network-tools/attaching-dnet.md

@@ -0,0 +1,3 @@
+# Attaching dnet
+
+TODO