Stream Multiplexing (stream muxing) is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported ‘over the wires’, then it is demulitiplexed (demuxed) so it can be output and used by separate applications.
Multiplexing is by no means unique to libp2p. Most communication networks involve some kind of multiplexing, as the transport medium is generally scarce and needs to be shared by many participants.
This is done to share a single TCP connection using unique port numbers to distinguish streams, between the multiple proceeses (such as kademlia and gossipsub) used by applications (such as ipfs) to make connection and transmission more efficient. With muxing, libp2p applications may have many separate streams of communication between peers, as well as have multiple concurrent streams open at the same time with a peer.
Stream multiplexing allows us to initialize and use the same transport connection across the lifetime of our interaction with a peer. With muxing, we also only need to deal with NAT traversal once to be able to open as many streams as we need, since they will all share the same underlying transport connection.
libp2p provides a common interface for stream multiplexers with several implementations available. Applications can enable support for multiple multiplexers, which will allow you to fall back to a widely-supported multiplexer if a preferred choice is not supported by a remote peer.
libp2p’s multiplexing happens at the “application layer”, meaning it’s not provided by the operating system’s network stack. However, developers writing libp2p applications rarely need to interact with stream multiplexers directly, except during initial configuration to control which modules are enabled.
libp2p maintains some state about known peers and existing connections in a component known as the switch (or “swarm”, depending on the implementation). The switch provides a dialing and listening interface that abstracts the details of which stream multiplexer is used for a given connection.
When configuring libp2p, applications enable stream muxing modules, which the switch will use when dialing peers and listening for connections. If the remote peers support any of the same stream muxing implementations, the switch will select and use it when establishing the connection. If you dial a peer that the switch already has an open connection to, the new stream will automatically be multiplexed over the existing connection.
Reaching agreement on which stream multiplexer to use happens early in the connection establishment process. Peers use protocol negotiation to agree on a commonly supported multiplexer, which upgrades a “raw” transport connection into a muxed connection capable of opening new streams.
The stream multiplexing interface defines how a stream muxing module can be applied to a connection and what operations are supported by a multiplexed connection.
There are several stream multiplexing modules available in libp2p. Please note that not all stream muxers are supported by every libp2p language implementation.
mplex is a protocol developed for libp2p. The spec defines a simple protocol for multiplexing that is widely supported across libp2p language implementations:
yamux offers more sophisticated flow control than mplex, and can scale to thousands of multiplexed streams over a single connection.
yamux is currently supported in go and rust:
quic is currently supported in go via go-libp2p-quic-transport.
SPDY is a Google-developed protocol that was the precursor to HTTP/2. SPDY implements a stream multiplexer, which is supported by some libp2p implementations: