Add text about ZeroMQ

1 files changed, 105 insertions, 1 deletions

diff --git a/interfaces.tex b/interfaces.tex
index d71a2a9..a008819 100644
--- a/interfaces.tex
+++ b/interfaces.tex
@@ -1,5 +1,5 @@
 \section{Interfacing the Tools}
-\subsection{Files and Pipes}
+\subsection{Files}
 The first versions of these tools used files and pipes to exchange data. For
 offline generation of a multiplex or a modulated I/Q, it is possible to
 generate all files separately, one after the other.
@@ -103,4 +103,108 @@ Congratulations! You have just created your first DAB multiplex! With the config
 adding more programmes is easy. More information is available in the \filename{doc/example.mux}
 
 \subsection{Over the Network}
+In a real-time scenario, where the audio sources produce data continuously and the tools have to
+run at the native rate, it is not possible to use files anymore to interconnect the tools. For this
+usage, a network interconnection is available between the tools.
+
+This network connection is based on ZeroMQ, a library that permits the creation of a socket
+connection with automatic connection management (connection, disconnection, error handling).
+ZeroMQ uses a TCP/IP connection, and can therefore be used over any kind of IP networks.
+
+\subsubsection{Between Encoder and Multiplexer}
+
+Between fdk-aac-dabplus and ODR-DabMux, the ZeroMQ connection transmits AAC superframes, with
+additional metadata that contains the audio level indication for monitoring purposes. The
+multiplexer cannot easily derive the audio level from the AAC bitstream without decoding it, so it
+makes more sense to calculate this in the encoder.
+
+The toolame-dab encoder also can send MPEG frames over ZeroMQ, but is not yet able to calculate and
+transmit audio level metadata yet.
+
+\sidenote{Add configuration example for encoders}
+
+On the multiplexer, the subchannel must be configured for ZeroMQ as follows:
+\begin{lstlisting}
+sub-fb {
+    type dabplus
+    bitrate 80
+    id 24
+    protection 3
+
+    inputfile "tcp://*:9001"
+    zmq-buffer 40
+    zmq-prebuffering 20
+}
+\end{lstlisting}
+
+The ZeroMQ input supports several options in addition to the ones of a subchannel that uses a file
+input. The options are:
+
+\begin{itemize}
+    \item \texttt{inputfile}: This defines the interface and port on which to listen for incoming
+        data. It must be of the form \texttt{tcp://*:<port>}. Support for the \texttt{pgm://}
+        protocol is experimental, please see the \texttt{zmq\_bind} manpage for more information
+        about the protocols.
+    \item \texttt{zmq-buffer}: The ZeroMQ input handles an internal buffer for incoming data. The
+        maximum buffer size is given by this option, the units are AAC frames ($24$\ms). Therefore,
+        with a value of $40$, you will have a buffer of $40 * 24 = 960$\ms. The multiplexer will
+        never buffer more than this value, and will discard data one AAC superframe
+        ($5$ frames $= 100$\ms) when the buffer is full.
+    \item \texttt{zmq-prebuffering}: When the buffer is empty, the multiplexer waits until this
+        amount of AAC frames are available in the buffer before it starts to consume data.
+\end{itemize}
+
+The goal of having a buffer in the input of the multiplexer is to be able to absorb network latency
+jitter: Because IP does not guarantee anything about the latency, some packets will reach the
+encoder faster than others. The buffer can then be used to avoid disruptions in these cases, and its
+size should be adapted to the network connection. This has to be done in an empirical way, and is a
+trade-off between absolute delay and robustness.
+
+If the encoder is running remotely on a machine, encoding from a sound card, it will encode at the
+rate defined by the sound card clock. This clock will, if no special precautions are taken, be
+slightly off frequency. The multiplexer however runs on a machine where the system time is
+synchronised over NTP, and will not show any drift or offset. Two situations can occur:
+
+Either the sound card clock is a bit slow, in which case the ZeroMQ buffer in the multiplexer will
+fill up to the amount given by \texttt{zmq-prebuffering}, and then start streaming data. Because the
+multiplexer will be a bit faster than the encoder, the amount of buffered data will slowly decrease,
+until the buffer is empty. Then the multiplexer will enter prebuffering, and wait again until the
+buffer is full enough. This will create an audible interruption, whose length corresponds to the
+prebuffering.
+
+Or the sound card clock is a bit slow, and the buffer will be filled up faster than data is consumed
+by the multiplexer. At some point, the buffer will hit the maximum size, and one superframe will be
+discarded. This also creates an audible glitch.
+
+Consumer grade sound cards have clocks of varying quality. While these glitches would only occur
+sporadically for some, bad sound cards can provoke such behaviour in intervals that are not
+acceptable, e.g. more than once per hour.
+
+Both situations are suboptimal, because they lead to audio glitches, and also degrade the ability to
+compensate for network latency changes. It is preferable to use the drift compensation feature
+available in fdk-aac-dabplus, which insures that the encoder outputs the AAC bitstream at the
+nominal rate, aligned to the NTP-synchronised system time, and not to the sound card clock. The
+sound card clock error is compensated for inside the encoder.
+
+
+
 \subsubsection{Authentication Support}
+In order to be able to use the Internet as contribution network, some form of protection has to be
+put in place to make sure the audio data cannot be altered by third parties. Usually, some form of
+VPN is set up for this case.
+
+Alternatively, the encryption mechanism ZeroMQ offers can also be used. To do this, it is necessary
+to set up keys and to distribute them to the encoder and the multiplexer.
+
+\begin{lstlisting}
+    encryption 1
+    secret-key "keys/mux.sec"
+    public-key "keys/mux.pub"
+    encoder-key "keys/encoder1.pub"
+\end{lstlisting}
+
+\sidenote{Add configuration example}
+
+\subsubsection{Between Multiplexer and Modulator}
+
+\subsection{Pipes}