net: create `darklist` for unknown transports + share darklist (not greylist)

In monero, nodes broadcast addrs from their whitelist. Receiving nodes
save the information on their greylist.

This is to ensure that honest nodes only broadcast active (i.e. whitelist)
nodes to the network. Dishonest nodes can send garbage info through
the hostlist, and therefore all information received from other nodes
is considered hostile and placed in the greylist, until we independently
verify it is accessible via the refinery.

Previously, darkfi deviated from this design as follows:

* Since peers on the greylist that do not match our transports never
  enter the refinery, we assume that the greylist consists of
  unsupported transports.
* We broadcast the greylist in ProtocolAddr, in an attempt to
  ensure that all transports are propagated.

Rather than simply assuming the greylist contains unsupported
transports, it is better to assume the greylist is hostile (since it
comes from other nodes).

We create a `darklist` specifically for storing unknown/ unsupported
transports. When we receive information from other peer, unsupported
addrs are added to our `darklist`, which is then broadcast to other
peers in ProtocolAddr. This fulfils to requirement (of broadcasting all
transports) without also involving honest peers in the propagating of
hostile info.

Specifically:

* Hostile peers can still broadcast garbage info in their gold, white
  and dark lists.

* Since info from other nodes is potentially hostile, honest peers save
  this info on their greylist and do not broadcast it to other peers
  unless a) it passes the refinery b) we connect to in outbound session
  c) we do not support this transport.

* There is a potential attack in which an attacker could fill their
  darklist with garbage e.g. Nym addresses, and honest nodes that do not
  support Nym will continue sharing these addresses via the dark list.
  The hostile peers will continue to be shared until a Nym-supporting
  node receives them and they pass via the refinery.

* Note that this attack is less severe, since providing the nodes stay
  on the Dark list they are ignored by the refinery and outbound connect
  loop and do not eat up resources of the node. The only time it will
  potentially cause pressure on a node if is the e.g. Nym node receives
  a list of hostile fake Nym addresses and they enter its greylist,
  causing it to refine many garbage addresses and potentially slowing
  its ability to make outbound connections. The latter can be prevented
  by increasing the settings `anchor_connect_count` and
  `white_connection_percent` (meaning outbound connections will not
  select from the greylist, or select less).

* Since there exists a potential attack vector of garbage entries in the
  Dark list, we limit the Dark list size to 1000 peers.

* This also means that supporting all transports is the best setup for a
  since it increases the security of the network (wrt the dark list).

bin/lilith/src/main.rs

@@ -186,7 +186,7 @@ impl Lilith {
 
             if !ping_node(url.clone(), p2p.clone()).await {
                 debug!(target: "lilith", "Host {} is not responsive. Downgrading from whitelist", url);
-                hosts.move_host(url, *last_seen, HostColor::Grey, false, None).await;
+                hosts.move_host(url, *last_seen, HostColor::Grey, false, None).await?;
 
                 continue
             }

src/net/channel.rs

@@ -320,7 +320,7 @@ impl Channel {
     pub async fn ban(&self, peer: &Url) {
         debug!(target: "net::channel::ban()", "START {:?}", self);
         let last_seen = UNIX_EPOCH.elapsed().unwrap().as_secs();
-        self.p2p().hosts().move_host(peer, last_seen, HostColor::Black, false, None).await;
+        self.p2p().hosts().move_host(peer, last_seen, HostColor::Black, false, None).await.unwrap();
 
         self.stop().await;
         debug!(target: "net::channel::ban()", "STOP {:?}", self);

src/net/hosts/refinery.rs

@@ -161,7 +161,7 @@ impl GreylistRefinery {
                     let last_seen = UNIX_EPOCH.elapsed().unwrap().as_secs();
 
                     // Add to the whitelist and remove from the greylist.
-                    hosts.move_host(url, last_seen, HostColor::White, false, None).await;
+                    hosts.move_host(url, last_seen, HostColor::White, false, None).await.unwrap();
                 }
                 None => {
                     debug!(target: "net::refinery", "No matching greylist entries found. Cannot proceed with refinery");

src/net/hosts/store.rs

@@ -38,6 +38,7 @@ use crate::{
 pub const LOCAL_HOST_STRS: [&str; 2] = ["localhost", "localhost.localdomain"];
 const WHITELIST_MAX_LEN: usize = 5000;
 const GREYLIST_MAX_LEN: usize = 2000;
+const DARKLIST_MAX_LEN: usize = 1000;
 
 /// Atomic pointer to hosts object
 pub type HostsPtr = Arc<Hosts>;
@@ -234,6 +235,10 @@ pub enum HostColor {
     /// Hostile peers that can neither be connected to nor establish
     /// connections to us for the duration of the program.
     Black = 3,
+    /// Peers that do not match our accepted transports. We are blind
+    /// to these nodes (we do not use them) but we send them around
+    /// the network anyway to ensure all transports are propagated.
+    Dark = 4,
 }
 
 impl TryFrom<usize> for HostColor {
@@ -245,6 +250,7 @@ impl TryFrom<usize> for HostColor {
             1 => Ok(HostColor::White),
             2 => Ok(HostColor::Gold),
             3 => Ok(HostColor::Black),
+            4 => Ok(HostColor::Dark),
             _ => Err(Error::InvalidHostColor),
         }
     }
@@ -255,12 +261,13 @@ impl TryFrom<usize> for HostColor {
 // TODO: Verify the performance overhead of using vectors for hostlists.
 // TODO: Check whether anchorlist (Gold) has a max size in Monero.
 pub struct HostContainer {
-    pub hostlists: [RwLock<Vec<(Url, u64)>>; 4],
+    pub hostlists: [RwLock<Vec<(Url, u64)>>; 5],
 }
 
 impl HostContainer {
     fn new() -> Self {
-        let hostlists: [RwLock<Vec<(Url, u64)>>; 4] = [
+        let hostlists: [RwLock<Vec<(Url, u64)>>; 5] = [
+            RwLock::new(Vec::new()),
             RwLock::new(Vec::new()),
             RwLock::new(Vec::new()),
             RwLock::new(Vec::new()),
@@ -276,7 +283,6 @@ impl HostContainer {
         HostColor::try_from(color).unwrap());
 
         let mut list = self.hostlists[color].write().await;
-
         list.push((addr, last_seen));
 
         if color == 0 && list.len() == GREYLIST_MAX_LEN {
@@ -295,6 +301,14 @@ impl HostContainer {
             );
         }
 
+        if color == 4 && list.len() == DARKLIST_MAX_LEN {
+            let last_entry = list.pop().unwrap();
+            debug!(
+                target: "net::hosts::store()",
+                "Darklist reached max size. Removed {:?}", last_entry,
+            );
+        }
+
         // Sort the list by last_seen.
         list.sort_by_key(|entry| entry.1);
         list.reverse();
@@ -682,14 +696,17 @@ impl HostContainer {
             };
 
             match data[0] {
-                "greylist" => {
-                    self.store(HostColor::Grey as usize, url, last_seen).await;
+                "gold" => {
+                    self.store(HostColor::Gold as usize, url, last_seen).await;
                 }
-                "whitelist" => {
+                "white" => {
                     self.store(HostColor::White as usize, url, last_seen).await;
                 }
-                "anchorlist" => {
-                    self.store(HostColor::Gold as usize, url, last_seen).await;
+                "grey" => {
+                    self.store(HostColor::Grey as usize, url, last_seen).await;
+                }
+                "dark" => {
+                    self.store(HostColor::Dark as usize, url, last_seen).await;
                 }
                 _ => {
                     debug!(target: "net::hosts::load_hosts()", "Malformed list name...");
@@ -707,9 +724,10 @@ impl HostContainer {
         let mut tsv = String::new();
         let mut hostlist: HashMap<String, Vec<(Url, u64)>> = HashMap::new();
 
-        hostlist.insert("gold".to_string(), self.fetch_all(HostColor::Gold).await);
-        hostlist.insert("white".to_string(), self.fetch_all(HostColor::White).await);
+        hostlist.insert("dark".to_string(), self.fetch_all(HostColor::Dark).await);
         hostlist.insert("grey".to_string(), self.fetch_all(HostColor::Grey).await);
+        hostlist.insert("white".to_string(), self.fetch_all(HostColor::White).await);
+        hostlist.insert("gold".to_string(), self.fetch_all(HostColor::Gold).await);
 
         for (name, list) in hostlist {
             for (url, last_seen) in list {
@@ -1001,17 +1019,6 @@ impl Hosts {
                 continue
             }
 
-            // Reject this peer if it's already stored on the Gold or White list.
-            // If it exists on the Grey list, we will simply update its last_seen
-            // field.
-            if self.container.contains(HostColor::Gold as usize, addr_).await ||
-                self.container.contains(HostColor::White as usize, addr_).await
-            {
-                debug!(target: "net::hosts::filter_addresses()",
-                    "We already have {} in the hostlist. Skipping", addr_);
-                continue
-            }
-
             let host_str = addr_.host_str().unwrap();
 
             if !localnet {
@@ -1064,6 +1071,30 @@ impl Hosts {
                 _ => continue,
             }
 
+            // Store this peer on Dark list if we do not support this transport.
+            // We will personally ignore this peer but still send it to others in
+            // Protocol Addr to ensure all transports get propagated.
+            if !settings.allowed_transports.contains(&addr_.scheme().to_string()) {
+                self.container.store_or_update(HostColor::Dark, addr_.clone(), *last_seen).await;
+
+                debug!(target: "net::hosts::filter_addresses()",
+                    "Added unsupported peer {} to Dark list", addr_);
+                continue
+            }
+
+            // Reject this peer if it's already stored on the Gold or White list.
+            // If it exists on the Grey list, we will simply update its last_seen
+            // field.
+            //
+            // We do this last since it is the most expensive operation.
+            if self.container.contains(HostColor::Gold as usize, addr_).await ||
+                self.container.contains(HostColor::White as usize, addr_).await
+            {
+                debug!(target: "net::hosts::filter_addresses()",
+                    "We already have {} in the hostlist. Skipping", addr_);
+                continue
+            }
+
             ret.push((addr_.clone(), *last_seen));
         }
 
@@ -1084,7 +1115,7 @@ impl Hosts {
         destination: HostColor,
         suspend: bool,
         channel: Option<ChannelPtr>,
-    ) {
+    ) -> Result<()> {
         debug!(target: "net::hosts::move_host()", "Trying to move addr={} node={} destination={:?}",
         addr, self.settings.node_id, destination);
 
@@ -1137,7 +1168,7 @@ impl Hosts {
                     // We mark this peer as Suspend which means we do not try to connect to it until it
                     // has passed through the refinery. This should never panic.
                     self.try_register(addr.clone(), HostState::Suspend).await.unwrap();
-                    return
+                    return Ok(());
                 }
             }
 
@@ -1213,7 +1244,7 @@ impl Hosts {
                 self.try_register(addr.clone(), HostState::Connected(channel.unwrap()))
                     .await
                     .unwrap();
-                return
+                return Ok(());
             }
 
             // Move to black. Remove from all other lists.
@@ -1224,7 +1255,7 @@ impl Hosts {
                     // Localhost connections should never enter the blacklist
                     // This however allows any Tor and Nym connections.
                     if self.is_local_host(addr.clone()).await {
-                        return
+                        return Ok(());
                     }
 
                     // Remove from the grey list if it exists.
@@ -1265,11 +1296,15 @@ impl Hosts {
                     drop(black);
                 }
             }
+
+            HostColor::Dark => return Err(Error::InvalidHostColor),
         }
 
         // Remove this entry from HostRegistry to avoid this host getting
         // stuck in the Moving state.
         self.unregister(addr).await;
+
+        Ok(())
     }
 }
 

src/net/protocol/protocol_address.rs

@@ -152,9 +152,9 @@ impl ProtocolAddress {
                 continue
             }
 
-            // First we grab address with the requested transports from the anchorlist
+            // First we grab address with the requested transports from the gold list
             debug!(target: "net::protocol_address::handle_receive_get_addrs()",
-            "Fetching anchorlist entries with schemes");
+            "Fetching gold entries with schemes");
             let mut addrs = self
                 .hosts
                 .container
@@ -182,8 +182,26 @@ impl ProtocolAddress {
 
             // Next we grab addresses without the requested transports
             // to fill a 2 * max length vector.
+
+            // Then we grab address without the requested transports from the gold list
+            debug!(target: "net::protocol_address::handle_receive_get_addrs()",
+            "Fetching gold entries without schemes");
+            let remain = 2 * get_addrs_msg.max - addrs.len() as u32;
+            addrs.append(
+                &mut self
+                    .hosts
+                    .container
+                    .fetch_n_random_excluding_schemes(
+                        HostColor::Gold,
+                        &get_addrs_msg.transports,
+                        remain,
+                    )
+                    .await,
+            );
+
+            // Then we grab address without the requested transports from the white list
             debug!(target: "net::protocol_address::handle_receive_get_addrs()",
-            "Fetching whitelist entries without schemes");
+            "Fetching white entries without schemes");
             let remain = 2 * get_addrs_msg.max - addrs.len() as u32;
             addrs.append(
                 &mut self
@@ -197,19 +215,17 @@ impl ProtocolAddress {
                     .await,
             );
 
-            // If there's still space available, take from the
-            // greylist. Schemes are not taken into account.
-            //
-            /* NOTE: We share peers from our greylist because our
-            greylist is likely to contain peers that do not match our
-            transports or the requested transports. We want to ensure
+            // If there's still space available, take from the Dark list.
+
+            /* NOTE: We share peers from our Dark list because to ensure
             that non-compatiable transports are shared with other nodes
             so that they propagate on the network even if they're not
             popular transports. */
+
             debug!(target: "net::protocol_address::handle_receive_get_addrs()",
-            "Fetching greylist entries");
+            "Fetching dark entries");
             let remain = 2 * get_addrs_msg.max - addrs.len() as u32;
-            addrs.append(&mut self.hosts.container.fetch_n_random(HostColor::Grey, remain).await);
+            addrs.append(&mut self.hosts.container.fetch_n_random(HostColor::Dark, remain).await);
 
             debug!(
                 target: "net::protocol_address::handle_receive_get_addrs()",

src/net/session/manual_session.rs

@@ -146,7 +146,7 @@ impl ManualSession {
                                     false,
                                     Some(channel.clone()),
                                 )
-                                .await;
+                                .await?;
 
                             // Wait for channel to close
                             stop_sub.receive().await;
@@ -155,7 +155,7 @@ impl ManualSession {
                             self.p2p()
                                 .hosts()
                                 .move_host(&addr, last_seen, HostColor::Grey, false, None)
-                                .await;
+                                .await?;
 
                             info!(
                                 target: "net::manual_session",

src/net/session/outbound_session.rs

@@ -431,13 +431,16 @@ impl Slot {
             self.channel_id.store(channel.info.id, Ordering::Relaxed);
 
             // Add this connection to the anchorlist
-            hosts.move_host(&addr, last_seen, HostColor::Gold, false, Some(channel.clone())).await;
+            hosts
+                .move_host(&addr, last_seen, HostColor::Gold, false, Some(channel.clone()))
+                .await
+                .unwrap();
 
             // Wait for channel to close
             stop_sub.receive().await;
 
             // Channel has disconnected. Downgrade this host to greylist.
-            hosts.move_host(&addr, last_seen, HostColor::Grey, false, None).await;
+            hosts.move_host(&addr, last_seen, HostColor::Grey, false, None).await.unwrap();
 
             self.channel_id.store(0, Ordering::Relaxed);
         }
@@ -466,7 +469,7 @@ impl Slot {
 
                 // At this point we failed to connect. We'll downgrade this peer and
                 // mark its state as Suspend, which sends it to the Refinery for processing.
-                self.p2p().hosts().move_host(&addr, last_seen, HostColor::Grey, true, None).await;
+                self.p2p().hosts().move_host(&addr, last_seen, HostColor::Grey, true, None).await?;
 
                 // Notify that channel processing failed
                 self.p2p().hosts().channel_subscriber.notify(Err(Error::ConnectFailed)).await;