algcodes.xml

<?xml version="1.0"?>
<!-- This file is part of the book                    -->
<!--                                                  -->
<!-- Abstract Algebra: Theory and Applications        -->
<!--                                                  -->
<!-- Text: Copyright (C) 1997-2025  Thomas W. Judson  -->
<!-- Sage: Copyright (C) 2010-2025  Robert A. Beezer  -->
<!-- See the file COPYING for copying conditions.     -->
<!-- This file is part of the book                    -->
<!--                                                  -->
<!-- See the file COPYING for copying conditions.     -->
<chapter xmlns:xi="http://www.w3.org/2001/XInclude" xml:id="algcodes">
  <title>Algebraic Coding Theory</title>
  <introduction>
    <p>
      Coding theory is an application of algebra that has become increasingly important over the last several decades.
      When we transmit data,
      we are concerned about sending a message over a channel that could be affected by <q>noise.</q>
      We wish to be able to encode and decode the information in a manner that will allow the detection,
      and possibly the correction, of errors caused by noise.
      This situation arises in many areas of communications,
      including radio, telephone,
      television, computer communications,
      and digital media technology.
      Probability, combinatorics, group theory, linear algebra,
      and polynomial rings over finite fields all play important roles in coding theory.
    </p>
  </introduction>

  <section xml:id="algcodes-section-error-detecting-correcting-codes">
    <title>Error-Detecting and Correcting Codes</title>
    <introduction>
      <p>
        Let us examine a simple model of a communications system for transmitting and receiving coded messages (<xref ref="algcodes-figure-encoding"/>).
      </p>

      <figure xml:id="algcodes-figure-encoding">
        
        <caption>Encoding and decoding messages</caption>
            <!-- Replaced figure with tikz figure - TWJ 5/10/2010 -->
        <image width="60%" xml:id="algcodes-encode-decode">
        	<description>A vertical flow chart where an m-digit message is passed to an encoder which is represented by a rectangle to become an n-digit codeword which is passed to a transmitter that is also represented by a rectangle.  The outcome of the transmitter may have noise and is passed to the receiver which is prepresented by a rectangle.  The receiver passes an n-digit received word to a decoder that is represented by a rectangle.  The decoder produces an m-digit received word or an error.</description>
            <latex-image>
                <xi:include href="tikz/algcodes-encode-decode.tex" parse="text"/>
            </latex-image>
        </image>

      </figure>

      <p>
        Uncoded messages may be composed of letters or characters,
        but typically they consist of binary <m>m</m>-tuples.
        These messages are encoded into codewords,
        consisting of binary <m>n</m>-tuples,
        by a device called an <term>encoder</term>.
        The message is transmitted and then decoded.
        We will consider the occurrence of errors during transmission.
        An <term>error</term> occurs if there is a change in one or more bits in the codeword.
        A <term>decoding scheme</term> is a method that either converts an arbitrarily received <m>n</m>-tuple into a meaningful decoded message or gives an error message for that <m>n</m>-tuple.
        If the received message is a codeword
        (one of the special <m>n</m>-tuples allowed to be transmitted),
        then the decoded message must be the unique message that was encoded into the codeword.
        For received non-codewords,
        the decoding scheme will give an error indication,
        or, if we are more clever,
        will actually try to correct the error and reconstruct the original message.
        Our goal is to transmit error-free messages as cheaply and quickly as possible.
      </p>

      <example xml:id="algcodes-example-repeat">
        <p>
          One possible coding scheme would be to send a message several times and to compare the received copies with one another.
          Suppose that the message to be encoded is a binary <m>n</m>-tuple <m>(x_{1}, x_{2}, \ldots,
          x_{n})</m>.
          The message is encoded into a binary <m>3n</m>-tuple by simply repeating the message three times:
          <me>
            (x_{1}, x_{2}, \ldots, x_{n}) \mapsto (x_{1}, x_{2}, \ldots, x_{n}, x_{1}, x_{2}, \ldots, x_{n}, x_{1}, x_{2}, \ldots, x_{n})
          </me>.
          To decode the message,
          we choose as the <m>i</m>th digit the one that appears in the <m>i</m>th place in at least two of the three transmissions.
          For example, if the original message is <m>(0110)</m>,
          then the transmitted message will be <m>(0110\;  0110\;  0110)</m>.
          If there is a transmission error in the fifth digit,
          then the received codeword will be <m>(0110\;  1110\;  0110)</m>,
          which will be correctly decoded as <m>(0110)</m>.<fn>
          We will adopt the convention that bits are numbered left to right in binary <m>n</m>-tuples.
          </fn> This triple-repetition method will automatically detect and correct all single errors,
          but it is slow and inefficient:
          to send a message consisting of <m>n</m> bits,
          <m>2n</m> extra bits are required,
          and we can only detect and correct single errors.
          We will see that it is possible to find an encoding scheme that will encode a message of <m>n</m> bits into <m>m</m> bits with <m>m</m> much smaller than <m>3n</m>.
        </p>
      </example>

      <example xml:id="algcodes-example-even-parity">
        <p>
          <term>Even parity</term>, a commonly used coding scheme,
          is much more efficient than the simple repetition scheme.
          The <acro>ASCII</acro> (American Standard Code for Information Interchange) coding system uses binary <m>8</m>-tuples,
          yielding <m>2^{8} = 256</m> possible <m>8</m>-tuples.
          However, only seven bits are needed since there are only
          <m>2^7 = 128</m> <acro>ASCII</acro> characters.
          What can or should be done with the extra bit?
          Using the full eight bits, we can detect single transmission errors.
          For example, the <acro>ASCII</acro> codes for A, B, and C are
          <md>
            <mrow>\text{A} &amp; = 65_{10} = 01000001_{2},</mrow>
            <mrow>\text{B} &amp; = 66_{10} = 01000010_{2},</mrow>
            <mrow>\text{C} &amp; = 67_{10} = 01000011_{2}</mrow>
          </md>.
          Notice that the leftmost bit is always set to 0;
          that is, the <m>128</m> <acro>ASCII</acro> characters have codes
          <md>
            <mrow>00000000_{2} &amp; = 0_{10},</mrow>
            <mrow>&amp; \vdots</mrow>
            <mrow>01111111_{2} &amp; = 127_{10}</mrow>
          </md>.
          The bit can be used for error checking on the other seven bits.
          It is set to either <m>0</m> or <m>1</m> so that the total number of <m>1</m> bits in the representation of a character is even.
          Using even parity, the codes for A, B, and C now become
          <md>
            <mrow>\text{A} &amp; = 01000001_{2},</mrow>
            <mrow>\text{B} &amp; = 01000010_{2},</mrow>
            <mrow>\text{C} &amp; = 11000011_{2}</mrow>
          </md>.
          Suppose an A is sent and a transmission error in the sixth bit is caused by noise over the communication channel so that  <m>(0100\; 0101)</m> is received.
          We know an error has occurred since the received word has an odd number of <m>1</m>s,
          and we can now request that the codeword be transmitted again.
          When used for error checking,
          the leftmost bit is called a <term>parity check bit</term>.
        </p>

        <p>
          By far the most common error-detecting codes used in computers are based on the addition of a parity bit.
          Typically, a computer stores information in <m>m</m>-tuples called <term>words</term>.
          Common word lengths are <m>8</m>,
          <m>16</m>, and <m>32</m> bits.
          One bit in the word is set aside as the parity check bit,
          and is not used to store information.
          This bit is set to either <m>0</m> or <m>1</m>,
          depending on the number of <m>1</m>s in the word.
        </p>

        <p>
          Adding a parity check bit allows the detection of all single errors because changing a single bit either increases or decreases the number of <m>1</m>s by one,
          and in either case the parity has been changed from even to odd,
          so the new word is not a codeword.
          (We could also construct an error detection scheme based on <term>odd parity</term>;
          that is, we could set the parity check bit so that a codeword always has an odd number of <m>1</m>s.)
        </p>
      </example>

      <p>
        The even parity system is easy to implement, but has two drawbacks.
        First, multiple errors are not detectable.
        Suppose an A is sent and the first and seventh bits are changed from <m>0</m> to <m>1</m>.
        The received word is a codeword,
        but will be decoded into a C instead of an A. Second,
        we do not have the ability to correct errors.
        If the 8-tuple <m>(1001\; 1000)</m> is received,
        we know that an error has occurred,
        but we have no idea which bit has been changed.
        We will now investigate a coding scheme that will not only allow us to detect transmission errors but will actually correct the errors.
      </p>

      <example xml:id="algcodes-example-nearest">
        <p>
          Suppose that our original message is either a <m>0</m> or a <m>1</m>,
          and that <m>0</m> encodes to <m>(000)</m> and <m>1</m> encodes to <m>(111)</m>.
          If only a single error occurs during transmission,
          we can detect and correct the error.
          For example, if a <m>(101)</m> is received,
          then the second bit must have been changed from a <m>1</m> to a <m>0</m>.
          The originally transmitted codeword must have been <m>(111)</m>.
          This method will detect and correct all single errors.
        </p>

        <table xml:id="algcodes-table-repetition-code">
          <title>A repetition code</title>
          <tabular halign="center" top="medium">
            <col/>
            <col/>
            <col/>
            <col/>
            <col/>
            <col/>
            <col/>
            <col/>
            <col/>
            <row>
              <cell>Transmitted</cell>
              <cell colspan="8">Received Word</cell>
            </row>
            <row bottom="medium">
              <cell>Codeword</cell>
              <cell><m>000</m></cell>
              <cell><m>001</m></cell>
              <cell><m>010</m></cell>
              <cell><m>011</m></cell>
              <cell><m>100</m></cell>
              <cell><m>101</m></cell>
              <cell><m>110</m></cell>
              <cell><m>111</m></cell>
            </row>
            <row>
              <cell><m>000</m></cell>
              <cell><m>0</m></cell>
              <cell><m>1</m></cell>
              <cell><m>1</m></cell>
              <cell><m>2</m></cell>
              <cell><m>1</m></cell>
              <cell><m>2</m></cell>
              <cell><m>2</m></cell>
              <cell><m>3</m></cell>
            </row>
            <row bottom="medium">
              <cell><m>111</m></cell>
              <cell><m>3</m></cell>
              <cell><m>2</m></cell>
              <cell><m>2</m></cell>
              <cell><m>1</m></cell>
              <cell><m>2</m></cell>
              <cell><m>1</m></cell>
              <cell><m>1</m></cell>
              <cell><m>0</m></cell>
            </row>
          </tabular>

        </table>

        <p>
          In <xref ref="algcodes-table-repetition-code"/>,
          we present all possible words that might be received for the transmitted codewords <m>(000)</m> and <m>(111)</m>.
          <xref ref="algcodes-table-repetition-code"/> also shows the number of bits by which each received <m>3</m>-tuple differs from each original codeword.
        </p>
      </example>
    </introduction>

    <subsection xml:id="algcodes-subsection-max-likelihood">
      <title>Maximum-Likelihood Decoding</title>
            <!-- Label repaired.  Suggested by R. Beezer. -->
            <!-- TWJ - 12/19/2011 -->
      <p>
        The coding scheme presented in <xref ref="algcodes-example-nearest"/> is not a complete solution to the problem because it does not account for the possibility of multiple errors.
        For example,
        either a (000) or a (111) could be sent and a (001) received.
        We have no means of deciding from the received word whether there was a single error in the third bit or two errors,
        one in the first bit and one in the second.
        No matter what coding scheme is used,
        an incorrect message could be received.
        We could transmit a (000),
        have errors in all three bits,
        and receive the codeword (111).
        It is important to make explicit assumptions about the likelihood and distribution of transmission errors so that,
        in a particular application,
        it will be known whether a given error detection scheme is appropriate.
        We will assume that transmission errors are rare, and,
        that when they do occur, they occur independently in each bit;
        that is, if <m>p</m> is the probability of an error in one bit and <m>q</m> is the probability of an error in a different bit,
        then the probability of errors occurring in both of these bits at the same time is <m>pq</m>.
        We will also assume that a received <m>n</m>-tuple is decoded into a codeword that is closest to it;
        that is, we assume that the receiver uses
        <idx><h>Maximum-likelihood decoding</h></idx><term>maximum-likelihood decoding</term>.<fn>
        This section requires a knowledge of probability,
        but can be skipped without loss of continuity.
        </fn></p>

      <figure xml:id="algcodes-figure-channel">
        
        <caption>Binary symmetric channel</caption>
            <!-- Replaced figure with tikz figure - TWJ 5/10/2010 -->
        <image width="40%" xml:id="algcodes-binary-channel">
        	<description>A schematic diagram where a zero has probabilty p that it will be decoded as a zero and probability q that it will be decoded as a one, which are represented by arrows from 0 to 0 and 1 to 1, respectively.  A one has probabilty p that it will be decoded as a one and probability q that it will be decoded as a zero, which are represented by arrows from 1 to 1 and 0 to 1, respectively.</description>
            <latex-image>
                <xi:include href="tikz/algcodes-binary-channel.tex" parse="text"/>
            </latex-image>
        </image>

      </figure>

      <p>
        A <term>binary symmetric channel</term><idx><h>Binary symmetric channel</h></idx> is a model that consists of a transmitter capable of sending a binary signal,
        either a <m>0</m> or a <m>1</m>, together with a receiver.
        Let <m>p</m> be the probability that the signal is correctly received.
        Then <m>q = 1 - p</m> is the probability of an incorrect reception.
        If a <m>1</m> is sent,
        then the probability that a <m>1</m> is received is <m>p</m> and the probability that a <m>0</m> is received is <m>q</m> (<xref ref="algcodes-figure-channel"/>).
        The probability that no errors occur during the transmission of a binary codeword of length <m>n</m> is <m>p^{n}</m>.
        For example,
        if <m>p=0.999</m> and a message consisting of 10,000 bits is sent,
        then the probability of a perfect transmission is
        <me>
          (0.999)^{10,000} \approx 0.00005
        </me>.
      </p>

      <theorem>
        <statement>
          <p>
            If a binary <m>n</m>-tuple
            <m>(x_{1}, \ldots,
            x_{n})</m> is transmitted across a binary symmetric channel with probability <m>p</m> that no error will occur in each coordinate,
            then the probability that there are errors in exactly <m>k</m> coordinates is
            <me>
              \binom{n}{k} q^kp^{n - k}
            </me>.
          </p>
        </statement>

        <proof>
          <p>
            Fix <m>k</m> different coordinates.
            We first compute the probability that an error has occurred in this fixed set of coordinates.
            The probability of an error occurring in a particular one of these <m>k</m> coordinates is <m>q</m>;
            the probability that an error will not occur in any of the remaining <m>n-k</m> coordinates is <m>p</m>.
            The probability of each of these <m>n</m> independent events is <m>q^{k}p^{n-k}</m>.
            The number of possible error patterns with exactly <m>k</m> errors occurring is equal to
            <me>
              \binom{n}{k}  = \frac{n!}{k!(n - k)!}
            </me>,
            the number of combinations of <m>n</m> things taken <m>k</m> at a time.
            Each of these error patterns has probability <m>q^{k}p^{n-k}</m> of occurring;
            hence, the probability of all of these error patterns is
            <me>
              \binom{n}{k}  q^{k}p^{n - k}
            </me>.
          </p>
        </proof>
      </theorem>

      <example xml:id="algcodes-example-probability">
        <p>
          Suppose that <m>p = 0.995</m> and a <m>500</m>-bit message is sent.
          The probability that the message was sent error-free is
          <me>
            p^{n} = (0.995)^{500} \approx 0.082
          </me>.
          The probability of exactly one error occurring is
          <me>
            \binom{n}{1}  qp^{n - 1}= 500(0.005)(0.995)^{499} \approx 0.204
          </me>.
          The probability of exactly two errors is
          <me>
            \binom{n}{2} q^{2}p^{n - 2}= \frac{500 \cdot 499}{2}(0.005)^{2}(0.995)^{498} \approx 0.257
          </me>.
          The probability of more than two errors is approximately
          <me>
            1 - 0.082 - 0.204 - 0.257 = 0.457
          </me>.
        </p>
      </example>
    </subsection>

    <subsection xml:id="algcodes-subsection-block-codes">
      <title>Block Codes</title>
      <p>
        If we are to develop efficient error-detecting and error-correcting codes,
        we will need more sophisticated mathematical tools.
        Group theory  will allow faster methods of encoding and decoding messages.
        A code is an <m>(n, m)</m>-<term>block code</term>
        if the information that is to be coded can be divided into blocks of <m>m</m> binary digits,
        each of which can be encoded into <m>n</m> binary digits.
        More specifically, an <m>(n, m)</m>-block code consists of an
        <term>encoding function</term>
        <me>
          E:{\mathbb Z}^{m}_{2} \rightarrow {\mathbb Z}^{n}_{2}
        </me>
        and a <term>decoding function</term>
        <me>
          D:{\mathbb Z}^{n}_{2} \rightarrow {\mathbb Z}^{m}_{2}
        </me>.
        A <term>codeword</term> is any element in the image of <m>E</m>.
        We also require that <m>E</m> be one-to-one so that two information blocks will not be encoded into the same codeword.
        If our code is to be error-correcting,
        then <m>D</m> must be onto.
      </p>

      <example xml:id="algcodes-example-block-code">
        <p>
          The even-parity coding system developed to detect single errors in <acro>ASCII</acro> characters is an <m>(8,7)</m>-block code.
          The encoding function is
          <me>
            E(x_7, x_6, \ldots, x_1) = (x_8, x_7,  \ldots, x_1)
          </me>,
          where <m>x_8 = x_7 + x_6 + \cdots + x_1</m> with addition in <m>{\mathbb Z}_2</m>.
        </p>
      </example>

      <p>
        Let <m>{\mathbf x} = (x_1, \ldots,
        x_n)</m> and <m>{\mathbf y} = (y_1, \ldots,
        y_n)</m> be binary <m>n</m>-tuples.
        The <term>Hamming distance</term><idx><h>Hamming distance</h></idx>
        or <term>distance</term>, <m>d({\mathbf x}, {\mathbf y})</m>,
        between <m>{\mathbf x}</m> and
        <m>{\mathbf y}</m> is the number of bits in which
        <m>{\mathbf x}</m> and <m>{\mathbf y}</m> differ.
        The distance between two codewords is the minimum number of transmission errors required to change one codeword into the other.
        The <term>minimum distance</term><idx><h>Code</h><h>minimum distance of</h></idx> for a code,
        <m>d_{\min}</m>,
        is the minimum of all distances <m>d({\mathbf x}, {\mathbf y})</m>,
        where <m>{\mathbf x}</m> and
        <m>{\mathbf y}</m> are distinct codewords.
        The <idx><h>Weight of a codeword</h></idx><term>weight</term>,   
        <m>w({\mathbf x})</m>, of a binary codeword
        <m>{\mathbf x}</m> is the number of <m>1</m>s in <m>{\mathbf x}</m>.
        Clearly, <m>w({\mathbf x}) = d({\mathbf x}, {\mathbf 0})</m>,
        where <m>{\mathbf 0} = (00 \cdots 0)</m>.

        <notation>
          <usage><m>d(\mathbf x, \mathbf y)</m></usage>
          <description>Hamming distance between <m>\mathbf x</m> and <m>\mathbf y</m></description>
        </notation>

        <notation>
          <usage><m>d_{\min}</m></usage>
          <description>the minimum distance of a code</description>
        </notation>

        <notation>
          <usage><m>w(\mathbf x)</m></usage>
          <description>the weight of <m>\mathbf x</m></description>
        </notation>

      </p>

      <example xml:id="algcodes-example-min-distance">
        <p>
          Let <m>{\mathbf x} = (10101)</m>,
          <m>{\mathbf y} = (11010)</m>,
          and <m>{\mathbf z} = (00011)</m> be all of the codewords in some code <m>C</m>.
          Then we have the following Hamming distances:
          <me>
            d({\mathbf x},{\mathbf y}) = 4, \qquad d({\mathbf x},{\mathbf z}) = 3, \qquad d({\mathbf y},{\mathbf z}) = 3
          </me>.
          The minimum distance  for this code is 3.
          We also have the following weights:
          <me>
            w({\mathbf x}) = 3, \qquad w({\mathbf y}) = 3, \qquad w({\mathbf z}) = 2
          </me>.
        </p>
      </example>

      <p>
        The following proposition lists some basic properties about the weight of a codeword and the distance between two codewords.
        The proof is left as an exercise.
      </p>

      <proposition>
        <statement>
          <p>
            Let <m>{\mathbf x}</m>, <m>{\mathbf y}</m>,
            and <m>{\mathbf z}</m> be binary <m>n</m>-tuples.
            Then

            <ol>
              <li>
                <p>
                  <m>w({\mathbf x}) = d( {\mathbf x}, {\mathbf 0})</m>;
                </p>
              </li>

              <li>
                <p>
                  <m>d( {\mathbf x}, {\mathbf y}) \geq 0</m>;
                </p>
              </li>

              <li>
                <p>
                  <m>d( {\mathbf x}, {\mathbf y}) = 0</m> exactly when <m>{\mathbf x} = {\mathbf y}</m>;
                </p>
              </li>

              <li>
                <p>
                  <m>d( {\mathbf x}, {\mathbf y})= d( {\mathbf y}, {\mathbf x})</m>;
                </p>
              </li>

              <li>
                <p>
                  <m>d( {\mathbf x}, {\mathbf y}) \leq d( {\mathbf x}, {\mathbf z}) + d( {\mathbf z}, {\mathbf y})</m>.
                </p>
              </li>
            </ol>
          </p>
        </statement>
      </proposition>

      <p>
        The weights in a particular code are usually much easier to compute than the Hamming distances between all codewords in the code.
        If a code is set up carefully,
        we can use this fact to our advantage.
      </p>

      <p>
        Suppose that <m>{\mathbf x} = (1101)</m> and
        <m>{\mathbf y} = (1100)</m> are codewords in some code.
        If we transmit <m>(1101)</m> and an error occurs in the rightmost bit,
        then (1100) will be received.
        Since <m>(1100)</m> is a codeword,
        the decoder will decode <m>(1100)</m> as the transmitted message.
        This code is clearly not very appropriate for error detection.
        The problem is that <m>d({\mathbf x}, {\mathbf y}) = 1</m>.
        If <m>{\mathbf x} = (1100)</m> and <m>{\mathbf y} = (1010)</m> are codewords,
        then <m>d({\mathbf x}, {\mathbf y}) = 2</m>.
        If <m>{\mathbf x}</m> is transmitted and a single error occurs,
        then <m>{\mathbf y}</m> can never be received.
        <xref ref="algcodes-table-4-bit-words"/> gives the distances between all 4-bit codewords in which the first three bits carry information and the fourth is an even parity check bit.
        We can see that the minimum distance here is <m>2</m>;
        hence, the code is suitable as a single error-detecting code.
      </p>

      <table xml:id="algcodes-table-4-bit-words">
        <title>Distances between 4-bit codewords</title>
        <tabular halign="center" top="medium" left="medium" right="medium">
          <row bottom="medium">
            <cell/>
            <cell><m>0000</m></cell>
            <cell><m>0011</m></cell>
            <cell><m>0101</m></cell>
            <cell><m>0110</m></cell>
            <cell><m>1001</m></cell>
            <cell><m>1010</m></cell>
            <cell><m>1100</m></cell>
            <cell><m>1111</m></cell>
          </row>
          <row>
            <cell><m>0000</m></cell>
            <cell><m>0</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>4</m></cell>
          </row>
          <row>
            <cell><m>0011</m></cell>
            <cell><m>2</m></cell>
            <cell><m>0</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>4 </m></cell>
            <cell><m>2</m></cell>
          </row>
          <row>
            <cell><m>0101</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>0</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>4</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
          </row>
          <row>
            <cell><m>0110</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>0</m></cell>
            <cell><m>4</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
          </row>
          <row>
            <cell><m>1001</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>4</m></cell>
            <cell><m>0</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
          </row>
          <row>
            <cell><m>1010</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>4</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>0</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
          </row>
          <row>
            <cell><m>1100</m></cell>
            <cell><m>2</m></cell>
            <cell><m>4</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>0</m></cell>
            <cell><m>2</m></cell>
          </row>
          <row bottom="medium">
            <cell><m>1111</m></cell>
            <cell><m>4</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>2</m></cell>
            <cell><m>0</m></cell>
          </row>
        </tabular>

      </table>

      <p>
        To determine exactly what the error-detecting and error-correcting capabilities for a code are,
        we need to analyze the minimum distance for the code.
        Let <m>{\mathbf x}</m> and <m>{\mathbf y}</m> be codewords.
        If <m>d({\mathbf x}, {\mathbf y}) = 1</m> and an error occurs where
        <m>{\mathbf x}</m> and <m>{\mathbf y}</m> differ,
        then <m>{\mathbf x}</m> is changed to <m>{\mathbf y}</m>.
        The received codeword is <m>{\mathbf y}</m> and no error message is given.
        Now suppose <m>d({\mathbf x}, {\mathbf y}) = 2</m>.
        Then a single error cannot change
        <m>{\mathbf x}</m> to <m>{\mathbf y}</m>.
        Therefore, if <m>d_{\min} = 2</m>,
        we have the ability to detect single errors.
        However, suppose that <m>d({\mathbf x}, {\mathbf y}) = 2</m>,
        <m>{\mathbf y}</m> is sent,
        and a noncodeword <m>{\mathbf z}</m> is received such that
        <me>
          d({\mathbf x}, {\mathbf z}) = d({\mathbf y}, {\mathbf z}) = 1
        </me>.
        Then the decoder cannot decide between
        <m>{\mathbf x}</m> and <m>{\mathbf y}</m>.
        Even though we are aware that an error has occurred,
        we do not know what the error is.
      </p>

      <p>
        Suppose <m>d_{\min} \geq 3</m>.
        Then the maximum-likelihood decoding scheme corrects all single errors.
        Starting with a codeword <m>{\mathbf x}</m>,
        an error in the transmission of a single bit gives
        <m>{\mathbf y}</m> with <m>d({\mathbf x}, {\mathbf y}) = 1</m>,
        but <m>d({\mathbf z}, {\mathbf y}) \geq 2</m> for any other codeword <m>{\mathbf z} \neq {\mathbf x}</m>.
        If we do not require the correction of errors,
        then we can detect multiple errors when a code has a minimum distance that is greater than or equal to <m>3</m>.
      </p>

      <theorem xml:id="algcodes-theorem-min-distance">
        <statement>
          <p>
            Let <m>C</m> be a code with <m>d_{\min} = 2n + 1</m>.
            Then <m>C</m> can correct any <m>n</m> or fewer errors.
            Furthermore,
            any <m>2n</m> or fewer errors can be detected in <m>C</m>.
          </p>
        </statement>

        <proof>
          <p>
            Suppose that a codeword <m>{\mathbf x}</m> is sent and the word
            <m>{\mathbf y}</m> is received with at most <m>n</m> errors.
            Then <m>d( {\mathbf x}, {\mathbf y}) \leq n</m>.
            If <m>{\mathbf z}</m> is any codeword other than <m>{\mathbf x}</m>, then
            <me>
              2n+1 \leq d( {\mathbf x}, {\mathbf z}) \leq d( {\mathbf x}, {\mathbf y}) + d( {\mathbf y}, {\mathbf z}) \leq n + d( {\mathbf y}, {\mathbf z})
            </me>.
            Hence, <m>d({\mathbf y}, {\mathbf z} ) \geq n+1</m> and
            <m>{\mathbf y}</m> will be correctly decoded as <m>{\mathbf x}</m>.
            Now suppose that <m>{\mathbf x}</m> is transmitted and
            <m>{\mathbf y}</m> is received and that at least one error  has occurred,
            but not more than <m>2n</m> errors.
            Then <m>1 \leq d( {\mathbf x}, {\mathbf y} ) \leq 2n</m>.
            Since the minimum distance between codewords is <m>2n +1</m>,
            <m>{\mathbf y}</m> cannot be a codeword.
            Consequently,
            the code can detect between <m>1</m> and <m>2n</m> errors.
          </p>
        </proof>
      </theorem>

      <example xml:id="algcodes-example-single-correct">
        <p>
          In <xref ref="algcodes-table-hamming-distance"/>,
          the codewords <m>{\mathbf c}_1 = (00000)</m>,
          <m>{\mathbf c}_2 = (00111)</m>,
          <m>{\mathbf c}_3 = (11100)</m>,
          and <m>{\mathbf c}_4 = (11011)</m> determine a single error-correcting code.
        </p>

        <table xml:id="algcodes-table-hamming-distance">
          <title>Hamming distances for an error-correcting code</title>
          <tabular halign="center" top="medium" left="medium" right="medium">
            <row bottom="medium">
              <cell/>
              <cell><m>00000</m></cell>
              <cell><m>00111</m></cell>
              <cell><m>11100</m></cell>
              <cell><m>11011</m></cell>
            </row>
            <row>
              <cell><m>00000 </m></cell>
              <cell><m>0</m></cell>
              <cell><m>3</m></cell>
              <cell><m>3</m></cell>
              <cell><m>4</m></cell>
            </row>
            <row>
              <cell><m>00111 </m></cell>
              <cell><m>3</m></cell>
              <cell><m>0</m></cell>
              <cell><m>4</m></cell>
              <cell><m>3</m></cell>
            </row>
            <row>
              <cell><m>11100 </m></cell>
              <cell><m>3</m></cell>
              <cell><m>4</m></cell>
              <cell><m>0</m></cell>
              <cell><m>3</m></cell>
            </row>
            <row bottom="medium">
              <cell><m>11011 </m></cell>
              <cell><m>4</m></cell>
              <cell><m>3</m></cell>
              <cell><m>3</m></cell>
              <cell><m>0</m></cell>
            </row>
          </tabular>

        </table>
      </example>


    </subsection>

    <subsection xml:id="algcodes-subsection-historical-note">
      <title>Historical Note</title>
      <p>
        Modern coding theory began in 1948 with C. Shannon's<idx><h>Shannon, C.</h></idx>
        paper, <q>A Mathematical Theory of Information</q>
        [7]. This paper offered an example of an algebraic code,
        and Shannon's Theorem proclaimed exactly how good codes could be expected to be.
        Richard Hamming<idx><h>Hamming, R.</h></idx>
        began working with linear codes at Bell Labs in the late 1940s and early 1950s after becoming frustrated because the programs that he was running could not recover from simple errors generated by noise.
        Coding theory has grown tremendously in the past several decades.
        <em>The Theory of Error-Correcting Codes</em>,
        by MacWilliams and Sloane [5], published in 1977,
        already contained over 1500 references.
        Linear codes (Reed-Muller <m>(32, 6)</m>-block codes) were used on NASA's Mariner space probes.
        More recent space probes such as Voyager have used what are called convolution codes.
        Currently, very active research is being done with Goppa codes,
        which are heavily dependent on algebraic geometry.
      </p>
    </subsection>
  </section>

  <section xml:id="algcodes-section-linear-codes">
    <title>Linear Codes</title>
    <introduction>
      <p>
        To gain more knowledge of a particular code and develop more efficient techniques of encoding,
        decoding,
        and error detection, we need to add additional structure to our codes.
        One way to accomplish this is to require that the code also be a group.
        A <term>group code</term><idx><h>Code</h><h>group</h></idx>
        is a code that is also a subgroup of <m>{\mathbb Z}_2^n</m>.
      </p>

      <p>
        To check that a code is a group code,
        we need only verify one thing.
        If we add any two elements in the code,
        the result must be an <m>n</m>-tuple that is again in the code.
        It is not necessary to check that the inverse of the <m>n</m>-tuple is in the code,
        since every codeword is its own inverse,
        nor is it necessary to check that <m>{\mathbf 0}</m> is a codeword.
        For instance,
        <me>
          (11000101) + (11000101) = (00000000)
        </me>.
      </p>

      <example xml:id="algcodes-example-weights">
        <p>
          Suppose that we have a code that consists of the following 7-tuples:
          <md>
            <mrow>&amp;(0000000) &amp; &amp; (0001111) &amp;  &amp; (0010101) &amp; &amp; (0011010)</mrow>
            <mrow>&amp;(0100110) &amp; &amp; (0101001) &amp; &amp; (0110011) &amp; &amp; (0111100)</mrow>
            <mrow>&amp;(1000011) &amp; &amp; (1001100) &amp; &amp; (1010110) &amp; &amp; (1011001)</mrow>
            <mrow>&amp;(1100101) &amp; &amp; (1101010) &amp; &amp; (1110000) &amp; &amp; (1111111)</mrow>
          </md>.
          It is a straightforward though tedious task to verify that this code is also a subgroup of
          <m>{\mathbb Z}_2^7</m> and, therefore, a group code.
          This code is a single error-detecting and single error-correcting  code,
          but it is a long and tedious process to compute all of the distances between  pairs of codewords to determine that <m>d_{\min} = 3</m>.
          It is much easier to see that the minimum weight of all the nonzero codewords is <m>3</m>.
          As we will soon see, this is no coincidence.
          However, the relationship between weights and distances in a particular code is heavily dependent on the fact that the code is a group.
        </p>
      </example>

      <lemma>
        <statement>
          <p>
            Let <m>{\mathbf x}</m> and
            <m>{\mathbf y}</m> be binary <m>n</m>-tuples.
            Then <m>w({\mathbf x} + {\mathbf y}) = d({\mathbf x}, {\mathbf y})</m>.
          </p>
        </statement>

        <proof>
          <p>
            Suppose that <m>{\mathbf x}</m> and
            <m>{\mathbf y}</m> are binary <m>n</m>-tuples.
            Then the distance between <m>{\mathbf x}</m> and
            <m>{\mathbf y}</m> is exactly the number of places in which
            <m>{\mathbf x}</m> and <m>{\mathbf y}</m> differ.
            But <m>{\mathbf x}</m> and
            <m>{\mathbf y}</m> differ in a particular coordinate exactly when the sum in the coordinate is <m>1</m>, since
            <md>
              <mrow>1 + 1 &amp; = 0</mrow>
              <mrow>0 + 0 &amp; = 0</mrow>
              <mrow>1 + 0 &amp; = 1</mrow>
              <mrow>0 + 1 &amp; = 1</mrow>
            </md>.
            Consequently,
            the weight of the sum must be the distance between the two codewords.
          </p>
        </proof>
      </lemma>

      <theorem>
        <statement>
          <p>
            Let <m>d_{\min}</m> be the minimum distance for a group code <m>C</m>.
            Then <m>d_{\min}</m> is the minimum weight of all the nonzero codewords in <m>C</m>.
            That is,
            <me>
              d_{\min} = \min\{ w({\mathbf x}) : { {\mathbf x} \neq {\mathbf 0} } \}
            </me>.
          </p>
        </statement>

        <proof>
          <p>
            Observe that
            <md>
              <mrow>d_{\min} &amp; =  \min \{ d({\mathbf x},{\mathbf y}) : {\mathbf x}\neq{\mathbf y} \}</mrow>
              <mrow>&amp;=  \min \{ d({\mathbf x},{\mathbf y}) : {\mathbf x}+{\mathbf y} \neq {\mathbf 0} \}</mrow>
              <mrow>&amp;= \min\{ w({\mathbf x} + {\mathbf y}) : {\mathbf x}+{\mathbf y}\neq {\mathbf 0} \}</mrow>
              <mrow>&amp; =  \min\{ w({\mathbf z}) : {\mathbf z} \neq {\mathbf 0} \}</mrow>
            </md>.
          </p>
        </proof>
      </theorem>
    </introduction>

    <subsection xml:id="algcodes-subsection-linear-codes">
      <title>Linear Codes</title>
      <p>
        From <xref ref="algcodes-example-weights"/>,
        it is now easy to check that the minimum nonzero weight is <m>3</m>;
        hence, the code does indeed detect and correct all single errors.
        We have now reduced the problem of finding <q>good</q>
        codes to that of generating group codes.
        One easy way to generate group codes is to employ a bit of matrix theory.
      </p>

      <p>
        Define the <term>inner product</term><idx><h>Inner product</h></idx>
        of two binary <m>n</m>-tuples to be
        <me>
          {\mathbf x} \cdot {\mathbf y} = x_1 y_1 + \cdots + x_n y_n
        </me>,
        where <m>{\mathbf x} = (x_1, x_2, \ldots,
        x_n)^\transpose</m> and <m>{\mathbf y} = (y_1, y_2, \ldots,
        y_n)^\transpose</m> are column vectors.<fn>
        Since we will be working with matrices,
        we will write binary <m>n</m>-tuples as column vectors for the remainder of this chapter.
        </fn> For example,
        if <m>{\mathbf x} = (011001)^\transpose</m> and <m>{\mathbf y} = (110101)^\transpose</m>,
        then <m>{\mathbf x} \cdot {\mathbf y} = 0</m>.
        We can also look at an inner product as the product of a row matrix with a column matrix; that is,
        <md>
          <mrow>{\mathbf x} \cdot {\mathbf y} &amp; = {\mathbf x}^\transpose  {\mathbf y}</mrow>
          <mrow>&amp; =
          \begin{pmatrix}
          x_1 &amp; x_2 &amp; \cdots &amp; x_n
          \end{pmatrix}
          \begin{pmatrix}
          y_1 \\ y_2 \\ \vdots \\ y_n
          \end{pmatrix}</mrow>
          <mrow>&amp; = x_{1}y_{1} + x_{2}y_{2} + \cdots + x_{n}y_{n}</mrow>
        </md>.
      </p>

      <example xml:id="algcodes-example-matrixcodes">
        <p>
          Suppose that the words to be encoded consist of all binary <m>3</m>-tuples and that our encoding scheme is even-parity.
          To encode an arbitrary <m>3</m>-tuple,
          we add a fourth bit to obtain an even number of <m>1</m>s.
          Notice that an arbitrary <m>n</m>-tuple
          <m>{\mathbf x} = (x_1, x_2, \ldots,
          x_n)^\transpose</m> has an even number of <m>1</m>s exactly when <m>x_1 + x_2 + \cdots + x_n = 0</m>;
          hence, a <m>4</m>-tuple <m>{\mathbf x} = (x_1, x_2, x_3, x_4)^\transpose</m> has an even number of <m>1</m>s if <m> x_1+ x_2+ x_3+ x_4 = 0</m>, or
          <me>
            {\mathbf x} \cdot {\mathbf 1} =  {\mathbf x}^\transpose {\mathbf 1} =
            \begin{pmatrix}
            x_1 &amp; x_2 &amp; x_3 &amp; x_4
            \end{pmatrix}
            \begin{pmatrix}
            1 \\ 1 \\ 1 \\ 1
            \end{pmatrix} = 0
          </me>.
          This example leads us to hope that there is a connection between matrices and coding theory.
        </p>
      </example>

      <p>
        Let <m>{\mathbb M}_{m \times n}({\mathbb Z}_2)</m> denote the set of all
        <m>m \times n</m> matrices with entries in <m>{\mathbb Z}_2</m>.
        We do matrix operations as usual except that all our addition and multiplication operations occur in <m>{\mathbb Z}_2</m>.
        Define the <term>null space</term><idx><h>Matrix</h><h>null space of</h></idx><idx><h>Null space</h><h>of a matrix</h></idx> of a matrix
        <m>H \in {\mathbb M}_{m \times n}({\mathbb Z}_2)</m> to be the set of all binary <m>n</m>-tuples
        <m>{\mathbf x}</m> such that <m>H{\mathbf x} = {\mathbf 0}</m>.
        We denote the null space of a matrix <m>H</m> by <m>\Null(H)</m>.

        <notation>
          <usage><m>\mathbb M_{m \times n}(\mathbf Z_2)</m></usage>
          <description>the set of <m>m \times n</m> matrices with entries in <m>\mathbb Z_2</m></description>
        </notation>

        <notation>
          <usage><m>\Null(H)</m></usage>
          <description>null space of a matrix <m>H</m></description>
        </notation>

      </p>

      <example xml:id="algcodes-example-group-code">
        <p>
          Suppose that
          <me>
            H =
            \begin{pmatrix}
            0 &amp; 1 &amp; 0 &amp; 1 &amp; 0 \\
            1 &amp; 1 &amp; 1 &amp; 1 &amp; 0 \\
            0 &amp; 0 &amp; 1 &amp; 1 &amp; 1
            \end{pmatrix}
          </me>.
          For a <m>5</m>-tuple <m>{\mathbf x} = (x_1, x_2, x_3, x_4, x_5)^\transpose</m> to be in the null space of <m>H</m>,
          <m>H{\mathbf x} = {\mathbf 0}</m>.
          Equivalently,
          the following system of equations must be satisfied:
          <md>
            <mrow>x_2 +  x_4  &amp; =  0</mrow>
            <mrow>x_1 + x_2 + x_3  + x_4 &amp; =  0</mrow>
            <mrow>x_3  + x_4  +  x_5 &amp; =  0</mrow>
          </md>.
          The set of binary <m>5</m>-tuples satisfying these equations is
          <me>
            (00000) \qquad (11110) \qquad (10101) \qquad (01011)
          </me>.
          This code is easily determined to be a group code.
        </p>
      </example>

      <theorem>
        <statement>
          <p>
            Let <m>H</m> be in <m>{\mathbb M}_{m \times n}({\mathbb Z}_2)</m>.
            Then the null space of <m>H</m> is a group code.
          </p>
        </statement>

        <proof>
          <p>
            Since each element of <m>{\mathbb Z}_2^n</m> is its own inverse,
            the only thing that really needs to be checked here is closure.
            Let <m>{\mathbf x}, {\mathbf y} \in \Null(H)</m> for some matrix <m>H</m> in <m>{\mathbb M}_{m \times n}({\mathbb Z}_2)</m>.
            Then <m>H{\mathbf x} = {\mathbf 0}</m> and <m>H{\mathbf y} = {\mathbf 0}</m>.
            So
            <me>
              H({\mathbf x}+{\mathbf y}) = H{\mathbf x} + H{\mathbf y} = {\mathbf 0} + {\mathbf 0} = {\mathbf 0}
            </me>.
            Hence, <m>{\mathbf x} + {\mathbf y}</m> is in the null space of <m>H</m> and therefore must be a codeword.
          </p>
        </proof>
      </theorem>
            <!-- typo correction.  Suggested by J. Buller. -->
            <!-- TWJ - 12/20/2011 -->
      <p>
        A code is a <term>linear code</term><idx><h>Code</h><h>linear</h></idx>
        if it is determined by the null space of some matrix <m>H \in {\mathbb M}_{m \times n}({\mathbb Z}_2)</m>.
      </p>

      <example xml:id="algcodes-example-linear-code">
        <p>
          Let <m>C</m> be the code given by the matrix
          <me>
            H =
            \begin{pmatrix}
            0 &amp; 0 &amp; 0 &amp; 1 &amp; 1 &amp; 1 \\
            0 &amp; 1 &amp; 1 &amp; 0 &amp; 1 &amp; 1 \\
            1 &amp; 0 &amp; 1 &amp; 0 &amp; 0 &amp; 1
            \end{pmatrix}
          </me>.
          Suppose that the <m>6</m>-tuple <m>{\mathbf x} = (010011)^\transpose</m> is received.
          It is a simple matter of matrix multiplication to determine whether or not <m>{\mathbf x}</m> is a codeword.
          Since
          <me>
            H{\mathbf x} =
            \begin{pmatrix} 
            0 \\ 1 \\ 1
            \end{pmatrix}
          </me>,
          the received word is not a codeword.
          We must either attempt to correct the word or request that it be transmitted again.
        </p>
      </example>
            <!-- typo correction.  Suggested by J. Buller. -->
            <!-- TWJ - 12/20/2011 -->
    </subsection>
  </section>

  <section xml:id="algcodes-section-parity-check">
    <title>Parity-Check and Generator Matrices</title>
    <p>
      We need to find a systematic way of generating linear codes as well as fast methods of decoding.
      By examining the properties of a matrix <m>H</m> and by carefully choosing <m>H</m>,
      it is possible to develop very efficient methods of encoding and decoding messages.
      To this end,
      we will introduce standard generator and canonical parity-check matrices.
    </p>

    <p>
      Suppose that <m>H</m> is an
      <m>m \times n</m> matrix with entries in <m>{\mathbb Z}_2</m> and <m>n \gt m</m>.
      If the last <m>m</m> columns of the matrix form the
      <m>m \times m</m> identity matrix, <m>I_m</m>,
      then the matrix is a <idx><h>Matrix</h><h>parity-check</h></idx><term>canonical parity-check matrix</term>.
      More specifically, <m>H= (A \mid I_m)</m>,
      where <m>A</m> is the <m>m \times (n-m)</m> matrix
      <me>
        \begin{pmatrix}
        a_{11} &amp; a_{12} &amp; \cdots &amp; a_{1,n-m} \\
        a_{21} &amp; a_{22} &amp; \cdots &amp; a_{2,n-m} \\
        \vdots &amp; \vdots &amp; \ddots &amp; \vdots    \\
        a_{m1} &amp; a_{m2} &amp; \cdots &amp; a_{m,n-m}
        \end{pmatrix}
      </me>
      and <m>I_m</m> is the <m>m \times m</m> identity matrix
      <me>
        \begin{pmatrix}
        1 &amp; 0 &amp; \cdots &amp; 0 \\
        0 &amp; 1 &amp; \cdots &amp; 0 \\
        \vdots &amp; \vdots &amp; \ddots &amp; \vdots \\
        0 &amp; 0 &amp; \cdots &amp; 1
        \end{pmatrix}
      </me>.
      With each canonical parity-check matrix we can associate an <m>n \times (n-m)</m>
      <term>standard generator matrix</term><idx><h>Matrix</h><h>generator</h></idx>
      <me>
        G = \left( \frac{I_{n-m}}{A} \right)
      </me>.
      Our goal will be to show that an <m>\mathbf x</m> satisfying
      <m>G {\mathbf x} = {\mathbf y}</m> exists if and only if <m>H{\mathbf y} = {\mathbf 0}</m>.
      Given a message block <m>{\mathbf x}</m> to be encoded,
      the matrix <m>G</m> will allow us to quickly encode it into a linear codeword <m>{\mathbf y}</m>.
    </p>

    <example xml:id="algcodes-example-parity-check">
      <p>
        Suppose that we have the following eight words to be encoded:
        <me>
          (000), (001), (010), \ldots, (111)
        </me>.
        For
        <me>
          A =
          \begin{pmatrix}
          0 &amp; 1 &amp; 1 \\
          1 &amp; 1 &amp; 0 \\
          1 &amp; 0 &amp; 1
          \end{pmatrix}
        </me>,
        the associated standard generator and canonical parity-check matrices are
        <me>
          G=
          \begin{pmatrix}
          1 &amp; 0 &amp; 0 \\
          0 &amp; 1 &amp; 0 \\
          0 &amp; 0 &amp; 1 \\
          0 &amp; 1 &amp; 1 \\
          1 &amp; 1 &amp; 0 \\
          1 &amp; 0 &amp; 1
          \end{pmatrix}
        </me>
        and
        <me>
          H =
          \begin{pmatrix}
          0 &amp; 1 &amp; 1 &amp; 1 &amp; 0 &amp; 0 \\
          1 &amp; 1 &amp; 0 &amp; 0 &amp; 1 &amp; 0 \\
          1 &amp; 0 &amp; 1 &amp; 0 &amp; 0 &amp; 1
          \end{pmatrix}
        </me>,
        respectively.
      </p>

      <p>
        Observe that the rows in <m>H</m>  represent the parity checks on certain bit positions in a <m>6</m>-tuple.
        The <m>1</m>s in the identity matrix serve as parity checks for the <m>1</m>s in the same row.
        If <m>{\mathbf x} = (x_1, x_2, x_3, x_4, x_5, x_6)</m>, then
        <me>
          {\mathbf 0}
          =
          H{\mathbf x}
          =
          \begin{pmatrix}
          x_2 + x_3 + x_4 \\
          x_1 + x_2 + x_5\\
          x_1 + x_3 + x_6
          \end{pmatrix}
        </me>,
        which yields a system of equations:
        <md>
          <mrow>x_2 + x_3 + x_4 &amp; = 0</mrow>
          <mrow>x_1 + x_2 + x_5 &amp; = 0</mrow>
          <mrow>x_1 + x_3 + x_6 &amp; = 0</mrow>
        </md>.
        Here <m>x_4</m> serves as a check bit for <m>x_2</m> and <m>x_3</m>;
        <m>x_5</m> is a check bit for <m>x_1</m> and <m>x_2</m>;
        and <m>x_6</m> is a check bit for <m>x_1</m> and <m>x_3</m>.
        The identity matrix keeps <m>x_4</m>,
        <m>x_5</m>, and <m>x_6</m> from having to check on each other.
        Hence, <m>x_1</m>, <m>x_2</m>,
        and <m>x_3</m> can be arbitrary but <m>x_4</m>,
        <m>x_5</m>, and <m>x_6</m> must be chosen to ensure parity.
        The null space of <m>H</m> is easily computed to be
        <me>
          \begin{array}{cccc}
          (000000) &amp; (001101) &amp; (010110) &amp; (011011) \\
          (100011) &amp; (101110) &amp; (110101) &amp; (111000).
          \end{array}
        </me>
        An even easier way to compute the null space is with the generator matrix <m>G</m> (<xref ref="algcodes-table-matrix-gen-code"/>).
      </p>
    </example>

    <table xml:id="algcodes-table-matrix-gen-code">
      <title>A matrix-generated code</title>
      <tabular halign="center" top="medium">
        <row bottom="medium">
          <cell>Message Word <m>\mathbf x</m></cell>
          <cell>Codeword <m>G \mathbf x</m></cell>
        </row>
        <row>
          <cell><m>000</m></cell>
          <cell><m>000000</m></cell>
        </row>
        <row>
          <cell><m>001</m></cell>
          <cell><m>001101</m></cell>
        </row>
        <row>
          <cell><m>010</m></cell>
          <cell><m>010110</m></cell>
        </row>
        <row>
          <cell><m>011</m></cell>
          <cell><m>011011</m></cell>
        </row>
        <row>
          <cell><m>100</m></cell>
          <cell><m>100011</m></cell>
        </row>
        <row>
          <cell><m>101</m></cell>
          <cell><m>101110</m></cell>
        </row>
        <row>
          <cell><m>110</m></cell>
          <cell><m>110101</m></cell>
        </row>
        <row bottom="medium">
          <cell><m>111</m></cell>
          <cell><m>111000</m></cell>
        </row>
      </tabular>

    </table>

    <theorem>
      <statement>
        <p>
          If <m>H \in {\mathbb M}_{m \times n}({\mathbb Z}_2)</m> is a canonical parity-check matrix,
          then <m>\Null(H)</m> consists of all
          <m>{\mathbf x} \in {\mathbb Z}_2^n</m> whose first <m>n-m</m> bits are arbitrary but whose last <m>m</m> bits are determined by <m>H{\mathbf x} = {\mathbf 0}</m>.
          Each of the last <m>m</m> bits serves as an even parity check bit for some of the first <m>n-m</m> bits.
          Hence, <m>H</m> gives rise to an <m>(n, n-m)</m>-block code.
        </p>
      </statement>
    </theorem>

    <p>
      We leave the proof of this theorem as an exercise.
      In light of the theorem,
      the first <m>n - m</m> bits in <m>{\mathbf x}</m> are called
      <term>information bits</term>
      and the last <m>m</m> bits are called <term>check bits</term>.
      In <xref ref="algcodes-example-parity-check"/>,
      the first three bits are the information bits and the last three are the check bits.
    </p>

    <theorem>
      <statement>
        <p>
          Suppose that <m>G</m> is an
          <m>n \times k</m> standard generator matrix.
          Then <m>C = \left\{{\mathbf y} : G{\mathbf x} ={\mathbf y}\text{ for }{\mathbf x}\in {\mathbb Z}_2^k\right\}</m> is an <m>(n,k)</m>-block code.
          More specifically, <m>C</m> is a group code.
        </p>
      </statement>

      <proof>
        <p>
          Let <m>G {\mathbf x}_1 = {\mathbf y}_1</m> and <m>G {\mathbf x}_2 ={\mathbf y}_2</m> be two codewords.
          Then <m>{\mathbf y}_1 + {\mathbf y}_2</m> is in <m>C</m> since
          <me>
            G( {\mathbf x}_1 + {\mathbf x}_2) = G {\mathbf x}_1 + G {\mathbf x}_2 = {\mathbf y}_1 + {\mathbf y}_2
          </me>.
          We must also show that two message blocks cannot be encoded into the same codeword.
          That is, we must show that if <m>G {\mathbf x} = G {\mathbf y}</m>,
          then <m>{\mathbf x} = {\mathbf y}</m>.
          Suppose that <m>G {\mathbf x} = G {\mathbf y}</m>.
          Then
          <me>
            G {\mathbf x} - G {\mathbf y} = G( {\mathbf x} - {\mathbf y}) = {\mathbf 0}
          </me>.
          However, the first <m>k</m> coordinates in
          <m>G( {\mathbf x} - {\mathbf y})</m> are exactly <m>x_1 -y_1, \ldots,
          x_k - y_k</m>,
          since they are determined by the identity matrix,
          <m>I_k</m>, part of <m>G</m>.
          Hence, <m>G( {\mathbf x} - {\mathbf y}) = {\mathbf 0}</m> exactly when <m>{\mathbf x} = {\mathbf y}</m>.
        </p>
      </proof>
    </theorem>

    <p>
      Before we can prove the relationship between canonical parity-check matrices and standard generating matrices,
      we need to prove a lemma.
    </p>

    <lemma xml:id="algcodes-lemma-parity-check">
      <statement>
        <p>
          Let <m>H = (A \mid I_m )</m> be an
          <m>m \times n</m> canonical parity-check matrix and <m>G = \left( \frac{I_{n-m} }{A} \right)</m> be the corresponding
          <m>n \times (n-m)</m> standard generator matrix.
          Then <m>HG = {\mathbf 0}</m>.
        </p>
      </statement>

      <proof>
        <p>
          Let <m>C = HG</m>.
          The <m>ij</m>th entry in <m>C</m> is
          <md>
            <mrow>c_{ij} &amp; = \sum_{k=1}^n h_{ik} g_{kj}</mrow>
            <mrow>&amp; =  \sum_{k=1}^{n-m} h_{ik} g_{kj} + \sum_{k=n-m+1}^n h_{ik} g_{kj}</mrow>
            <mrow>&amp; = \sum_{k=1}^{n-m} a_{ik} \delta_{kj} + \sum_{k=n-m+1}^n \delta_{i-(m-n),k} a_{kj}</mrow>
            <mrow>&amp; =  a_{ij} + a_{ij}</mrow>
            <mrow>&amp; = 0</mrow>
          </md>,
          where
          <me>
            \delta_{ij} =
            \begin{cases}
            1 &amp; i = j \\
            0 &amp; i \neq j
            \end{cases}
          </me>
          is the <idx><h>Kronecker delta</h></idx>Kronecker delta.
          <notation>
            <usage><m>\delta_{ij}</m></usage>
            <description>Kronecker delta</description>
          </notation>

        </p>
      </proof>
    </lemma>

    <theorem>
      <statement>
        <p>
          Let <m>H = (A \mid I_m )</m> be an
          <m>m \times n</m> canonical parity-check matrix and let <m>G = \left( \frac{I_{n-m} }{A} \right) </m> be the
          <m>n \times (n-m)</m> standard generator matrix associated with <m>H</m>.
          Let <m>C</m> be the code generated by <m>G</m>.
          Then <m>{\mathbf y}</m> is in <m>C</m> if and only if <m>H {\mathbf y} = {\mathbf 0}</m>.
          In particular,
          <m>C</m> is a linear code with canonical parity-check matrix <m>H</m>.
        </p>
      </statement>

      <proof>
        <p>
          First suppose that <m>{\mathbf y} \in C</m>.
          Then <m>G {\mathbf x} = {\mathbf y}</m> for some <m>{\mathbf x} \in {\mathbb Z}_2^m</m>.
          By <xref ref="algcodes-lemma-parity-check"/>,
          <m>H {\mathbf y} = HG {\mathbf x} = {\mathbf 0}</m>.
        </p>

        <p>
          Conversely, suppose that <m>{\mathbf y} = (y_1, \ldots,
          y_n)^\transpose</m> is in the null space of <m>H</m>.
          We need to find an <m>{\mathbf x}</m> in
          <m>{\mathbb Z}_2^{n-m}</m> such that <m>G {\mathbf x}^\transpose = {\mathbf y}</m>.
          Since <m>H {\mathbf y} = {\mathbf 0}</m>,
          the following set of equations must be satisfied:
          <md>
            <mrow>a_{11} y_1 + a_{12} y_2 + \cdots + a_{1, n-m} y_{n-m} + y_{n-m+1} &amp; = 0</mrow>
            <mrow>a_{21} y_1 + a_{22} y_2 + \cdots + a_{2, n-m} y_{n-m} + y_{n-m+2} &amp; = 0</mrow>
            <mrow>&amp; \vdots</mrow>
            <mrow>a_{m1} y_1 + a_{m2} y_2 + \cdots + a_{m, n-m} y_{n-m} + y_{n-m+m} &amp; = 0</mrow>
          </md>.
          Equivalently, <m>y_{n-m+1}, \ldots,
          y_n</m> are determined by <m>y_1, \ldots, y_{n-m}</m>:
          <md>
            <mrow>y_{n-m+1} &amp; = a_{11} y_1 + a_{12} y_2 + \cdots + a_{1, n-m} y_{n-m}</mrow>
            <mrow>y_{n-m+2} &amp; = a_{21} y_1 + a_{22} y_2 + \cdots + a_{2, n-m} y_{n-m}</mrow>
            <mrow>&amp; \vdots</mrow>
            <mrow>y_{n} &amp; = a_{m1} y_1 + a_{m2} y_2 + \cdots + a_{m, n-m} y_{n-m}</mrow>
          </md>.
          Consequently,
          we can let <m>x_i = y_i</m> for <m>i= 1, \ldots, n - m</m>.
        </p>
      </proof>
    </theorem>

    <p>
      It would be helpful if we could compute the minimum distance of a linear code directly from its matrix <m>H</m> in order to determine the error-detecting and error-correcting capabilities of the code.
      Suppose that
      <md>
        <mrow>{\mathbf e}_1 &amp; = (100 \cdots 00)^\transpose</mrow>
        <mrow>{\mathbf e}_2 &amp; = (010 \cdots 00)^\transpose</mrow>
        <mrow>&amp; \vdots</mrow>
        <mrow>{\mathbf e}_n &amp; = (000 \cdots 01)^\transpose</mrow>
      </md>
      are the <m>n</m>-tuples in <m>{\mathbb Z}_2^n</m> of weight <m>1</m>.
      For an <m>m \times n</m> binary matrix <m>H</m>,
      <m>H{\mathbf e}_i</m> is exactly the <m>i</m>th column of the matrix <m>H</m>.
    </p>

    <example xml:id="algcodes-example-ith-column">
      <p>
        Observe that
        <me>
          \begin{pmatrix}
          1 &amp; 1 &amp; 1 &amp; 0 &amp; 0 \\
          1 &amp; 0 &amp; 0 &amp; 1 &amp; 0 \\
          1 &amp; 1 &amp; 0 &amp; 0 &amp; 1
          \end{pmatrix}
          \begin{pmatrix}
          0 \\ 1 \\ 0 \\ 0 \\ 0
          \end{pmatrix}
          =
          \begin{pmatrix}
          1 \\ 0 \\ 1
          \end{pmatrix}
        </me>.
      </p>
    </example>

    <p>
      We state this result in the following proposition and leave the proof as an exercise.
    </p>

    <proposition xml:id="algcodes-proposition-column">
      <statement>
        <p>
          Let <m>{\mathbf e}_i</m> be the binary <m>n</m>-tuple with a <m>1</m> in the <m>i</m>th coordinate and <m>0</m>'s elsewhere and suppose that <m>H \in {\mathbb M}_{m \times n}({\mathbb Z}_2)</m>.
          Then <m>H{\mathbf e}_i</m> is the <m>i</m>th column of the matrix <m>H</m>.
        </p>
      </statement>
    </proposition>

    <theorem xml:id="algcodes-theorem-single-error">
      <statement>
        <p>
          Let <m>H</m> be an <m>m \times n</m> binary matrix.
          Then the null space of <m>H</m> is a single error-detecting code if and only if no column of <m>H</m> consists entirely of zeros.
        </p>
      </statement>

      <proof>
        <p>
          Suppose that <m>\Null(H)</m> is a single error-detecting code.
          Then the minimum distance of the code must be at least <m>2</m>.
          Since the null space is a group code,
          it is sufficient to require that the code contain no codewords of less than weight <m>2</m> other than the zero codeword.
          That is, <m>{\mathbf e}_i</m> must not be a codeword for <m>i = 1, \ldots, n</m>.
          Since <m>H{\mathbf e}_i</m> is the <m>i</m>th column of <m>H</m>,
          the only way in which <m>{\mathbf e}_i</m> could be in the null space of <m>H</m> would be if the <m>i</m>th column were all zeros,
          which is impossible;
          hence, the code must have the capability to detect at least single errors.
        </p>

        <p>
          Conversely, suppose that no column of <m>H</m> is the zero column.
          By <xref ref="algcodes-proposition-column"/>, <m>H{\mathbf e}_i \neq {\mathbf 0}</m>.
        </p>
      </proof>
    </theorem>

    <example xml:id="algcodes-example-null-space">
      <p>
        If we consider the matrices
        <me>
          H_1 =
          \begin{pmatrix}
          1 &amp; 1 &amp; 1 &amp; 0 &amp; 0 \\
          1 &amp; 0 &amp; 0 &amp; 1 &amp; 0 \\
          1 &amp; 1 &amp; 0 &amp; 0 &amp; 1
          \end{pmatrix}
        </me>
        and
        <me>
          H_2 =
          \begin{pmatrix}
          1 &amp; 1 &amp; 1 &amp; 0 &amp; 0 \\
          1 &amp; 0 &amp; 0 &amp; 0 &amp; 0 \\
          1 &amp; 1 &amp; 0 &amp; 0 &amp; 1
          \end{pmatrix}
        </me>,
        then the null space of <m>H_1</m> is a single error-detecting code and the null space of <m>H_2</m> is not.
      </p>
    </example>

    <p>
      We can even do better than <xref ref="algcodes-theorem-single-error"/>.
      This theorem gives us conditions on a matrix <m>H</m> that tell us when the minimum weight of the code formed by the null space of <m>H</m> is <m>2</m>.
      We can also determine when the minimum distance of a linear code is <m>3</m> by examining the corresponding matrix.
    </p>

    <example xml:id="algcodes-example-check-matrix">
      <p>
        If we let
        <me>
          H =
          \begin{pmatrix}
          1 &amp; 1 &amp; 1 &amp; 0 \\
          1 &amp; 0 &amp; 0 &amp; 1 \\
          1 &amp; 1 &amp; 0 &amp; 0
          \end{pmatrix}
        </me>
        and want to determine whether or not <m>H</m> is the canonical parity-check matrix for an error-correcting code,
        it is necessary to make certain that
        <m>\Null(H)</m> does not contain any <m>4</m>-tuples of weight <m>2</m>.
        That is, <m>(1100)</m>, <m>(1010)</m>,
        <m>(1001)</m>, <m>(0110)</m>,
        <m>(0101)</m>, and <m>(0011)</m> must not be in <m>\Null(H)</m>.
        The next theorem states that we can indeed determine that the code generated by <m>H</m> is error-correcting by examining the columns of <m>H</m>.
        Notice in this example that not only does <m>H</m> have no zero columns,
        but also that no two columns are the same.
      </p>
    </example>

    <theorem>
      <statement>
        <p>
          Let <m>H</m> be a binary matrix.
          The null space of <m>H</m> is a single error-correcting code if and only if <m>H</m> does not contain any zero columns and no two columns of <m>H</m> are identical.
        </p>
      </statement>

      <proof>
        <p>
          The <m>n</m>-tuple <m>{\mathbf e}_{i} +{\mathbf e}_{j}</m> has <m>1</m>s in the <m>i</m>th and <m>j</m>th entries and 0s elsewhere,
          and <m>w( {\mathbf e}_{i} +{\mathbf e}_{j}) = 2</m> for <m>i \neq j</m>.
          Since
          <me>
            {\mathbf 0} = H({\mathbf e}_{i} +{\mathbf e}_{j}) = H{\mathbf e}_{i} + H{\mathbf e}_{j}
          </me>
          can only occur if the <m>i</m>th and <m>j</m>th columns are identical,
          the null space of <m>H</m> is a single error-correcting code.
        </p>
      </proof>
    </theorem>

    <p>
      Suppose now that we have a canonical parity-check matrix <m>H</m> with three rows.
      Then we might ask how many more columns we can add to the matrix and still have a null space that is a single error-detecting and single error-correcting code.
      Since each column has three entries,
      there are <m>2^3 = 8</m> possible distinct columns.
      We cannot add the columns
      <me>
        \begin{pmatrix}
        0 \\ 0 \\ 0 
        \end{pmatrix},
        \begin{pmatrix}
        1 \\ 0 \\ 0 
        \end{pmatrix},
        \begin{pmatrix}
        0 \\ 1 \\ 0 
        \end{pmatrix},
        \begin{pmatrix}
        0 \\ 0 \\ 1 
        \end{pmatrix}
      </me>.
      So we can add as many as four columns and still maintain a minimum distance of <m>3</m>.
    </p>

    <p>
      In general, if <m>H</m> is an
      <m>m \times n</m> canonical parity-check matrix,
      then there are <m>n-m</m> information positions in each codeword.
      Each column has <m>m</m> bits,
      so there are <m>2^m</m> possible distinct columns.
      It is necessary that the columns <m>{\mathbf 0}, {\mathbf e}_1, \ldots, {\mathbf e}_m</m> be excluded,
      leaving <m>2^m - (1 + m)</m> remaining columns for information if we are still to maintain the ability not only to detect but also to correct single errors.
    </p>
        <!-- typo correction.  Suggested by G. Cheng. -->
        <!-- TWJ - 10/1/2014 -->
  </section>

  <section xml:id="algcodes-section-efficient-decoding">
    <title>Efficient Decoding</title>
    <introduction>
      <p>
        We are now at the stage where we are able to generate linear codes that detect and correct errors fairly easily,
        but it is still a time-consuming process to decode a received <m>n</m>-tuple and determine which is the closest codeword,
        because the received <m>n</m>-tuple must be compared to each possible codeword to determine the proper decoding.
        This can be a serious impediment if the code is very large.
      </p>

      <example xml:id="algcodes-example-syndrome">
        <p>
          Given the binary matrix
          <me>
            H =
            \begin{pmatrix}
            1 &amp; 1 &amp; 1 &amp; 0 &amp; 0 \\
            0 &amp; 1 &amp; 0 &amp; 1 &amp; 0 \\
            1 &amp; 0 &amp; 0 &amp; 0 &amp; 1
            \end{pmatrix}
          </me>
          and the <m>5</m>-tuples <m>{\mathbf x} = (11011)^\transpose</m> and <m>{\mathbf y} = (01011)^\transpose</m>,
          we can compute
          <me>
            H{\mathbf x} =
            \begin{pmatrix}
            0 \\ 0 \\ 0 
            \end{pmatrix}
            \qquad
            \text{and}
            \qquad
            H{\mathbf y} =
            \begin{pmatrix}
            1 \\ 0 \\ 1 
            \end{pmatrix}
          </me>.
          Hence, <m>{\mathbf x}</m> is a codeword and <m>{\mathbf y}</m> is not,
          since <m>{\mathbf x}</m> is in the null space and <m>{\mathbf y}</m> is not.
          Notice that <m>H{\mathbf y}</m> is identical to the first column of <m>H</m>.
          In fact, this is where the error occurred.
          If we flip the first bit in
          <m>{\mathbf y}</m> from <m>0</m> to <m>1</m>,
          then we obtain <m>{\mathbf x}</m>.
        </p>
      </example>
            <!-- typo correction.  Suggested by E. Martin. -->
            <!-- TWJ - 1/2/2013 -->
      <p>
        If <m>H</m> is an <m>m \times n</m> matrix and <m>{\mathbf x} \in {\mathbb Z}_2^n</m>,
        then we say that the <term>syndrome</term><idx><h>Syndrome of a code</h></idx>
        of <m>{\mathbf x}</m> is <m>H{\mathbf x}</m>.
        The following proposition allows the quick detection and correction of errors.
      </p>

      <proposition xml:id="algcodes-proposition-syndrome">
        <statement>
          <p>
            Let the <m>m \times n</m> binary matrix <m>H</m> determine a linear code and let
            <m>{\mathbf x}</m> be the received <m>n</m>-tuple.
            Write <m>{\mathbf x}</m> as <m>{\mathbf x} = {\mathbf c} +{\mathbf e}</m>,
            where <m>{\mathbf c}</m> is the transmitted codeword and
            <m>{\mathbf e}</m> is the transmission error.
            Then the syndrome <m>H{\mathbf x}</m> of the received codeword
            <m>{\mathbf x}</m> is also the syndrome of the error <m>{\mathbf e}</m>.
          </p>
        </statement>

        <proof>
          <p>
            The proof follows from the fact that
            <me>
              H{\mathbf x} = H({\mathbf c} +{\mathbf e}) = H{\mathbf c} + H{\mathbf e} = {\mathbf 0} + H{\mathbf e} = H{\mathbf e}
            </me>.
          </p>
        </proof>
      </proposition>
            <!--Made the proof into a complete sentence.  TWJ 3/6/2014-->
      <p>
        This proposition tells us that the syndrome of a received word depends solely on the error and not on the transmitted codeword.
        The proof of the following theorem follows immediately from <xref ref="algcodes-proposition-syndrome"/> and from the fact that
        <m>H{\mathbf e}</m> is the <m>i</m>th column of the matrix <m>H</m>.
      </p>

      <theorem>
        <statement>
          <p>
            Let <m>H \in {\mathbb M}_{ m \times n} ( {\mathbb Z}_2)</m> and suppose that the linear code corresponding to <m>H</m> is single error-correcting.
            Let <m>{\mathbf r}</m> be a received <m>n</m>-tuple that was transmitted with at most one error.
            If the syndrome of <m>{\mathbf r}</m> is <m>{\mathbf 0}</m>,
            then no error has occurred;
            otherwise, if the syndrome of
            <m>{\mathbf r}</m> is equal to some column of <m>H</m>,
            say the <m>i</m>th column, then the error has occurred in the <m>i</m>th bit.
          </p>
        </statement>
      </theorem>

      <example xml:id="algcodes-example-detecting-errors">
        <p>
          Consider the matrix
          <me>
            H =
            \begin{pmatrix}
            1 &amp; 0 &amp; 1 &amp; 1 &amp; 0 &amp; 0 \\
            0 &amp; 1 &amp; 1 &amp; 0 &amp; 1 &amp; 0 \\
            1 &amp; 1 &amp; 1 &amp; 0 &amp; 0 &amp; 1
            \end{pmatrix}
          </me>
          and suppose that the  <m>6</m>-tuples <m>{\mathbf x} = (111110)^\transpose</m>,
          <m>{\mathbf y} = (111111)^\transpose</m>,
          and <m>{\mathbf z} = (010111)^\transpose</m> have been received.
          Then
          <me>
            H{\mathbf x} =
            \begin{pmatrix}
            1 \\ 1 \\ 1 
            \end{pmatrix},
            H{\mathbf y} =
            \begin{pmatrix}
            1 \\ 1 \\ 0 
            \end{pmatrix},
            H{\mathbf z} =
            \begin{pmatrix}
            1 \\ 0 \\ 0
            \end{pmatrix}
          </me>.
          Hence, <m>{\mathbf x}</m> has an error in the third bit and
          <m>{\mathbf z}</m> has an error in the fourth bit.
          The transmitted codewords for
          <m>{\mathbf x}</m> and <m>{\mathbf z}</m> must have been
          <m>(110110)</m> and <m>(010011)</m>, respectively.
          The syndrome of <m>{\mathbf y}</m> does not occur in any of the columns of the matrix <m>H</m>,
          so multiple errors must have occurred to produce <m>{\mathbf y}</m>.
        </p>
      </example>
    </introduction>

    <subsection xml:id="algcodes-subsection-coset-decoding">
      <title>Coset Decoding</title>
      <p>
        We can use group theory to obtain another way of decoding messages.
        A linear code <m>C</m> is a subgroup of <m>{\mathbb Z}_2^n</m>. <term>Coset</term><idx><h>Coset decoding</h></idx>
        or <term>standard decoding</term><idx><h>Standard decoding</h></idx>
        uses the cosets of <m>C</m> in
        <m>{\mathbb Z}_2^n</m> to implement maximum-likelihood decoding.
        Suppose that <m>C</m> is an <m>(n,m)</m>-linear code.
        A coset of <m>C</m> in <m>{\mathbb Z}_2^n</m> is written in the form <m>{\mathbf x} + C</m>,
        where <m>{\mathbf x} \in {\mathbb Z}_2^n</m>.
        By Lagrange's Theorem (<xref ref="cosets-theorem-lagrange"/>),
        there are <m>2^{n - (n - m)} = 2^m</m> distinct cosets of <m>C</m> in <m>{\mathbb Z}_2^n</m>.
      </p>

      <example xml:id="algcodes-example-coset-decoding">
        <p>
          Let <m>C</m> be the <m>(5,3)</m>-linear code given by the parity-check matrix
          <me>
            H =
            \begin{pmatrix}
            0 &amp; 1 &amp; 1 &amp; 0 &amp; 0 \\
            1 &amp; 0 &amp; 0 &amp; 1 &amp; 0 \\
            1 &amp; 1 &amp; 0 &amp; 0 &amp; 1
            \end{pmatrix}
          </me>.
          The code consists of the codewords
          <me>
            (00000) \quad (01101) \quad (10011) \quad (11110)
          </me>.
          There are <m>2^{5-2} = 2^3</m> cosets of <m>C</m> in <m>{\mathbb Z}_2^5</m>,
          each with order <m>2^2 =4</m>.
          These cosets are listed in <xref ref="algcodes-table-cosets-of-c"/>.
        </p>
      </example>

      <table xml:id="algcodes-table-cosets-of-c">
        <title>Cosets of <m>C</m></title>
        <tabular halign="center" top="medium">
          <row>
            <cell>Coset</cell>
            <cell>Coset</cell>
          </row>
          <row bottom="medium">
            <cell>Representative</cell>
            <cell/>
          </row>
          <row>
            <cell><m>C</m></cell>
            <cell><m>(00000)  (01101)  (10011)  (11110)</m></cell>
          </row>
          <row>
            <cell><m>(10000) + C</m></cell>
            <cell><m>(10000)  (11101)  (00011)  (01110)</m></cell>
          </row>
          <row>
            <cell><m>(01000) + C</m></cell>
            <cell><m>(01000)  (00101)  (11011)  (10110)</m></cell>
          </row>
          <row>
            <cell><m>(00100) + C</m></cell>
            <cell><m>(00100)  (01001)  (10111)  (11010)</m></cell>
          </row>
          <row>
            <cell><m>(00010) + C</m></cell>
            <cell><m>(00010)  (01111)  (10001)  (11100)</m></cell>
          </row>
          <row>
            <cell><m>(00001) + C</m></cell>
            <cell><m>(00001)  (01100)  (10010)  (11111)</m></cell>
          </row>
          <row>
            <cell><m>(10100) + C</m></cell>
            <cell><m>(00111)  (01010)  (10100)  (11001)</m></cell>
          </row>
          <row bottom="medium">
            <cell><m>(00110) + C</m></cell>
            <cell><m>(00110)  (01011)  (10101)  (11000)</m></cell>
          </row>
        </tabular>

      </table>

      <p>
        Our task is to find out how knowing the cosets might help us to decode a message.
        Suppose that <m>{\mathbf x}</m> was the original codeword sent and that
        <m>{\mathbf r}</m> is the <m>n</m>-tuple received.
        If <m>{\mathbf e}</m> is the transmission error,
        then <m>{\mathbf r} = {\mathbf e} + {\mathbf x}</m> or,
        equivalently, <m>{\mathbf x} = {\mathbf e} + {\mathbf r}</m>.
        However, this is exactly the statement that
        <m>{\mathbf r}</m> is an element in the coset <m>{\mathbf e} + C</m>.
        In maximum-likelihood decoding we expect the error
        <m>{\mathbf e}</m> to be as small as possible;
        that is, <m>{\mathbf e}</m> will have the least weight.
        An <m>n</m>-tuple of least weight in a coset is called a
        <idx><h>Coset</h><h>leader</h></idx><term>coset leader</term>.  
        Once we have determined a coset leader for each coset,
        the decoding process becomes a task of calculating
        <m>{\mathbf r} + {\mathbf e}</m> to obtain <m>{\mathbf x}</m>.
      </p>

      <example xml:id="algcodes-example-representative">
        <p>
          In <xref ref="algcodes-table-cosets-of-c"/>,
          notice that we have chosen a representative of the least possible weight for each coset.
          These representatives are coset leaders.
          Now suppose that <m>{\mathbf r} = (01111)</m> is the received word.
          To decode <m>{\mathbf r}</m>,
          we find that it is in the coset <m>(00010) + C</m>;
          hence, the originally transmitted codeword must have been <m>(01101) = (01111) + (00010)</m>.
        </p>
      </example>

      <p>
        A potential problem with this method of decoding is that we might have to examine every coset for the received codeword.
        The following proposition gives a method of implementing coset decoding.
        It states that we can associate a syndrome with each coset;
        hence, we can make a table that designates a coset leader corresponding to each syndrome.
        Such a list is called a  <idx><h>Decoding table</h></idx><term>decoding table</term>.
      </p>

      <table xml:id="algcodes-table-syndrome">
        <title>Syndromes for each coset</title>
        <tabular halign="center" top="medium">
          <row bottom="medium">
            <cell>Syndrome</cell>
            <cell>Coset Leader</cell>
          </row>
          <row>
            <cell><m>(000)</m></cell>
            <cell><m>(00000)</m></cell>
          </row>
          <row>
            <cell><m>(001)</m></cell>
            <cell><m>(00001)</m></cell>
          </row>
          <row>
            <cell><m>(010)</m></cell>
            <cell><m>(00010)</m></cell>
          </row>
          <row>
            <cell><m>(011)</m></cell>
            <cell><m>(10000)</m></cell>
          </row>
          <row>
            <cell><m>(100)</m></cell>
            <cell><m>(00100)</m></cell>
          </row>
          <row>
            <cell><m>(101)</m></cell>
            <cell><m>(01000)</m></cell>
          </row>
          <row>
            <cell><m>(110)</m></cell>
            <cell><m>(00110)</m></cell>
          </row>
          <row bottom="medium">
            <cell><m>(111)</m></cell>
            <cell><m>(10100)</m></cell>
          </row>
        </tabular>

      </table>

      <proposition>
        <statement>
          <p>
            Let <m>C</m> be an <m>(n,k)</m>-linear code given by the matrix <m>H</m> and suppose that <m>{\mathbf x}</m> and
            <m>{\mathbf y}</m> are in <m>{\mathbb Z}_2^n</m>.
            Then <m>{\mathbf x}</m> and
            <m>{\mathbf y}</m> are in the same coset of <m>C</m> if and only if <m>H{\mathbf x} = H{\mathbf y}</m>.
            That is, two <m>n</m>-tuples are in the same coset if and only if their syndromes are the same.
          </p>
        </statement>

        <proof>
          <p>
            Two <m>n</m>-tuples <m>{\mathbf x}</m> and
            <m>{\mathbf y}</m> are in the same coset of <m>C</m> exactly when <m>{\mathbf x} - {\mathbf y} \in C</m>;
            however, this is equivalent to
            <m>H({\mathbf x} - {\mathbf y}) = 0</m> or <m>H {\mathbf x} = H{\mathbf y}</m>.
          </p>
        </proof>
      </proposition>

      <example xml:id="algcodes-example-decoding-table">
        <p>
          <xref ref="algcodes-table-syndrome"/> is a decoding table for the code <m>C</m> given in <xref ref="algcodes-example-coset-decoding"/>.
          If <m>{\mathbf x} = (01111)</m> is received,
          then its syndrome can be computed to be
          <me>
            H {\mathbf x} =
            \begin{pmatrix}
            0 \\ 1 \\ 0
            \end{pmatrix}
          </me>.
          Examining the decoding table,
          we determine that the coset leader is <m>(00010)</m>.
          It is now easy to decode the received codeword.
        </p>
      </example>

      <p>
        Given an <m>(n,k)</m>-block code,
        the question arises of whether or not coset decoding is a manageable scheme.
        A decoding table requires a list of cosets and syndromes,
        one for each of the <m>2^{n - k}</m> cosets of <m>C</m>.
        Suppose that we have a <m>(32, 24)</m>-block code.
        We have a huge number of codewords, <m>2^{24}</m>,
        yet there are only <m>2^{32 - 24} = 2^{8} = 256</m> cosets.
      </p>
      <paragraphs component="sage-blurb">
        <title>Sage</title>
        <p>
          Sage has a substantial repertoire of commands for coding theory,
          including the ability to build many different families of codes.
        </p>
      </paragraphs>
    </subsection>
  </section>

    <reading-questions>

    <exercise label="rq-algcodes-min-distance">
      <statement>
        <p>
          Suppose a binary code has minimum distance <m>d=6</m>.
          How many errors can be detected?
          How many errors can be corrected?
        </p>
      </statement>
      <response/>
    </exercise>

    <exercise label="rq-algcodes-ascii-check">
      <statement>
        <p>
          Explain why it is impossible for the 8-bit string with decimal value <m>56_{10}</m> to be an <acro>ASCII</acro> code for a character.
          Assume the leftmost bit of the string is being used as a parity-check bit.
        </p>
      </statement>
      <response/>
    </exercise>

    <exercise label="rq-algcodes-min-error">
      <statement>
        <p>
          Suppose we receive the 8-bit string with decimal value <m>56_{10}</m> when we are expecting <acro>ASCII</acro> characters with a parity-check bit in the first bit
          (leftmost).
          We know an error has occurred in transmission.
          Give one of the probable guesses for the character which was actually sent
          (other than <sq>8</sq>),
          under the assumption that any individual bit is rarely sent in error.
          Explain the logic of your answer.
          (You may need to consult a table of <acro>ASCII</acro> values online.)
        </p>
      </statement>
      <response/>
    </exercise>

    <exercise label="rq-algcodes-linear-code">
      <statement>
        <p>
          Suppose a linear code <m>C</m> is created as the null space of the parity-check matrix
          <me>
            H =\begin{bmatrix}
            0 \amp 1 \amp 0 \amp 1 \amp 0 \\
            1 \amp 1 \amp 1 \amp 1 \amp 0 \\
            0 \amp 0 \amp 1 \amp 1 \amp 1
            \end{bmatrix}
          </me>
          Then <m>x=11100</m> is not a codeword.
          Describe a computation, and give the result of that computation,
          which verifies that <m>x</m> is not a codeword of the code <m>C</m>.
        </p>
      </statement>
      <response/>
    </exercise>

    <exercise label="rq-algcodes-max-likelihood">
      <statement>
        <p>
          For <m>H</m> and <m>x</m> as in the previous question,
          suppose that <m>x</m> is received as a message.
          Give a maximum likelihood decoding of the received message.
        </p>
      </statement>
      <response/>
    </exercise>

  </reading-questions>
    <!-- Exercises with Solutions  -->
    <!-- File: algcodes.xml  -->
    <!-- Title: Algebraic Coding Theory -->
  <exercises xml:id="algcodes-exercises" filenamebase="algcodes">
    <title>Exercises</title>
    <exercise number="1" xml:id="algcodes-exercise-acceptable-codes">
      <statement>
        <p>
          Why is the following encoding scheme not acceptable?
        </p>

        <tabular halign="center" top="medium">
          <row bottom="medium">
            <cell>Information</cell>
            <cell><m>0</m></cell>
            <cell><m>1</m></cell>
            <cell><m>2</m></cell>
            <cell><m>3</m></cell>
            <cell><m>4</m></cell>
            <cell><m>5</m></cell>
            <cell><m>6</m></cell>
            <cell><m>7</m></cell>
            <cell><m>8</m></cell>
          </row>
          <row bottom="medium">
            <cell>Codeword</cell>
            <cell><m>000</m></cell>
            <cell><m>001</m></cell>
            <cell><m>010</m></cell>
            <cell><m>011</m></cell>
            <cell><m>101</m></cell>
            <cell><m>110</m></cell>
            <cell><m>111</m></cell>
            <cell><m>000</m></cell>
            <cell><m>001</m></cell>
          </row>
        </tabular>
      </statement>
    </exercise>

    <exercise number="2" xml:id="algcodes-exercise-not-a-group-code">
      <statement>
        <p>
          Without doing any addition,
          explain why the following set of <m>4</m>-tuples in
          <m>{\mathbb Z}_2^4</m> cannot be a group code.
          <me>
            (0110) \quad (1001) \quad (1010) \quad (1100)
          </me>
        </p>
      </statement>
      <hint>
        <p>
          This cannot be a group code since <m>(0000) \notin C</m>.
        </p>
      </hint>
    </exercise>

    <exercise number="3" xml:id="algcodes-exercise-hamming-distance">
      <statement>
        <p>
          Compute the Hamming distances between the following pairs of <m>n</m>-tuples.

          <ol cols="2">
            <li>
              <p>
                <m>(011010), (011100)</m>
              </p>
            </li>

            <li>
              <p>
                <m>(11110101), (01010100)</m>
              </p>
            </li>

            <li>
              <p>
                <m>(00110), (01111)</m>
              </p>
            </li>

            <li>
              <p>
                <m>(1001), (0111)</m>
              </p>
            </li>
          </ol>
        </p>
      </statement>
      <hint>
        <p>
          (a) <m>2</m>; (c) <m>2</m>.
        </p>
      </hint>
    </exercise>

    <exercise number="4" xml:id="algcodes-exercise-weights">
      <statement>
        <p>
          Compute the weights of the following <m>n</m>-tuples.

          <ol cols="2">
            <li>
              <p>
                <m>(011010)</m>
              </p>
            </li>

            <li>
              <p>
                <m>(11110101)</m>
              </p>
            </li>

            <li>
              <p>
                <m>(01111)</m>
              </p>
            </li>

            <li>
              <p>
                <m>(1011)</m>
              </p>
            </li>
          </ol>
        </p>
      </statement>
      <hint>
        <p>
          (a) <m>3</m>; (c) <m>4</m>.
        </p>
      </hint>
    </exercise>

    <exercise number="5" xml:id="algcodes-exercise-minimum-weight-7">
      <statement>
        <p>
          Suppose that a linear code <m>C</m> has a minimum weight of <m>7</m>.
          What are the error-detection and error-correction capabilities of <m>C</m>?
        </p>
      </statement>
    </exercise>

    <exercise number="6" xml:id="algcodes-exercise-minimum-distance">
      <statement>
        <p>
          In each of the following codes,
          what is the minimum distance for the code?
          What is the best situation we might hope for in connection with error detection and error correction?

          <ol>
            <li>
              <p>
                <m>(011010) \; (011100) \; (110111) \; (110000)</m>
              </p>
            </li>

            <li>
              <p>
                <m>(011100) \; (011011) \; (111011) \; (100011) \\ (000000) \; (010101) \; (110100) \; (110011)</m>
              </p>
            </li>

            <li>
              <p>
                <m>(000000) \; (011100) \; (110101) \; (110001)</m>
              </p>
            </li>

            <li>
              <p>
                <m>(0110110) \; (0111100) \; (1110000) \; (1111111) \\ (1001001) \; (1000011) \; (0001111) \; (0000000)</m>
              </p>
            </li>
          </ol>
        </p>
      </statement>
      <hint>
        <p>
          (a) <m>d_{\min} = 2</m>; (c) <m>d_{\min} = 1</m>.
        </p>
      </hint>
    </exercise>

    <exercise number="7" xml:id="algcodes-exercise-null-space">
      <statement>
        <p>
          Compute the null space of each of the following matrices.
          What type of <m>(n,k)</m>-block codes are the null spaces?
          Can you find a matrix
          (not necessarily a standard generator matrix)
          that generates each code?
          Are your generator matrices unique?

          <ol cols="2">
            <li>
              <p>
                <me>
                  \begin{pmatrix}
                  0 &amp; 1 &amp; 0 &amp; 0 &amp; 0 \\
                  1 &amp; 0 &amp; 1 &amp; 0 &amp; 1 \\
                  1 &amp; 0 &amp; 0 &amp; 1 &amp; 0
                  \end{pmatrix}
                </me>
              </p>
            </li>

            <li>
              <p>
                <me>
                  \begin{pmatrix}
                  1 &amp; 0 &amp; 1 &amp; 0 &amp; 0 &amp; 0 \\
                  1 &amp; 1 &amp; 0 &amp; 1 &amp; 0 &amp; 0 \\
                  0 &amp; 1 &amp; 0 &amp; 0 &amp; 1 &amp; 0 \\
                  1 &amp; 1 &amp; 0 &amp; 0 &amp; 0 &amp; 1
                  \end{pmatrix}
                </me>
              </p>
            </li>

            <li>
              <p>
                <me>
                  \begin{pmatrix}
                  1 &amp; 0 &amp; 0 &amp; 1 &amp; 1 \\
                  0 &amp; 1 &amp; 0 &amp; 1 &amp; 1
                  \end{pmatrix}
                </me>
              </p>
            </li>

            <li>
              <p>
                <me>
                  \begin{pmatrix}
                  0 &amp; 0 &amp; 0 &amp; 1 &amp; 1 &amp; 1 &amp; 1 \\
                  0 &amp; 1 &amp; 1 &amp; 0 &amp; 0 &amp; 1 &amp; 1 \\
                  1 &amp; 0 &amp; 1 &amp; 0 &amp; 1 &amp; 0 &amp; 1 \\
                  0 &amp; 1 &amp; 1 &amp; 0 &amp; 0 &amp; 1 &amp; 1
                  \end{pmatrix}
                </me>
              </p>
            </li>
          </ol>
        </p>
      </statement>
      <hint>
        <p>
          <ol>
            <li>
              <p>
                <m>(00000), (00101), (10011), (10110)</m>
                <me>
                  G = 
                  \begin{pmatrix}
                  0 &amp; 1 \\
                  0 &amp; 0 \\
                  1 &amp; 0 \\
                  0 &amp; 1 \\
                  1 &amp; 1
                  \end{pmatrix}
                </me>
              </p>
            </li>

            <li>
              <p>
                <m>(000000), (010111), (101101), (111010)</m>
                <me>
                  G = 
                  \begin{pmatrix}
                  1 &amp; 0 \\
                  0 &amp; 1 \\
                  1 &amp; 0 \\
                  1 &amp; 1 \\ 
                  0 &amp; 1 \\
                  1 &amp; 1
                  \end{pmatrix}
                </me>
              </p>
            </li>
          </ol>
        </p>
      </hint>
    </exercise>

    <exercise number="8" xml:id="algcodes-exercise-5-2-block-code">
      <statement>
        <p>
          Construct a <m>(5,2)</m>-block code.
          Discuss both the error-detection and error-correction capabilities of your code.
        </p>
      </statement>
    </exercise>

    <exercise number="9" xml:id="algcodes-exercise-null-space-code">
      <statement>
        <p>
          Let <m>C</m> be the code obtained from the null space of the matrix
          <me>
            H =
            \begin{pmatrix}
            0 &amp; 1 &amp; 0 &amp; 0 &amp; 1 \\
            1 &amp; 0 &amp; 1 &amp; 0 &amp; 1 \\
            0 &amp; 0 &amp; 1 &amp; 1 &amp; 1
            \end{pmatrix}
          </me>.
          Decode the message
          <me>
            01111 \quad 10101 \quad 01110 \quad 00011
          </me>
          if possible.
        </p>
      </statement>
      <hint>
        <p>
          Multiple errors occur in one of the received words.
        </p>
      </hint>
    </exercise>

    <exercise number="10" xml:id="algcodes-exercise-probability">
      <statement>
        <p>
          Suppose that a <m>1000</m>-bit binary message is transmitted.
          Assume that the probability of a single error is <m>p</m> and that the errors occurring in different bits are independent of one another.
          If <m>p = 0.01</m>,
          what is the probability of more than one error occurring?
          What is the probability of exactly two errors occurring?
          Repeat this problem for <m>p = 0.0001</m>.
        </p>
      </statement>
    </exercise>

    <exercise number="11" xml:id="algcodes-exercise-check-matrices">
      <statement>
        <p>
          Which matrices are canonical parity-check matrices?
          For those matrices that are canonical parity-check matrices,
          what are the corresponding standard generator matrices?
          What are the error-detection and error-correction capabilities of the code generated by each of these matrices?

          <ol cols="2">
            <li>
              <p>
                <me>
                  \begin{pmatrix}
                  1 &amp; 1 &amp; 0 &amp; 0 &amp; 0 \\
                  0 &amp; 0 &amp; 1 &amp; 0 &amp; 0 \\
                  0 &amp; 0 &amp; 0 &amp; 1 &amp; 0 \\
                  1 &amp; 0 &amp; 0 &amp; 0 &amp; 1
                  \end{pmatrix}
                </me>
              </p>
            </li>

            <li>
              <p>
                <me>
                  \begin{pmatrix}
                  0 &amp; 1 &amp; 1 &amp; 0 &amp; 0 &amp; 0 \\
                  1 &amp; 1 &amp; 0 &amp; 1 &amp; 0 &amp; 0 \\
                  0 &amp; 1 &amp; 0 &amp; 0 &amp; 1 &amp; 0 \\
                  1 &amp; 1 &amp; 0 &amp; 0 &amp; 0 &amp; 1
                  \end{pmatrix}
                </me>
              </p>
            </li>

            <li>
              <p>
                <me>
                  \begin{pmatrix}
                  1 &amp; 1 &amp; 1 &amp; 0 \\
                  1 &amp; 0 &amp; 0 &amp; 1
                  \end{pmatrix}
                </me>
              </p>
            </li>

            <li>
              <p>
                <me>
                  \begin{pmatrix}
                  0 &amp; 0 &amp; 0 &amp; 1 &amp; 0 &amp; 0 &amp; 0 \\
                  0 &amp; 1 &amp; 1 &amp; 0 &amp; 1 &amp; 0 &amp; 0 \\
                  1 &amp; 0 &amp; 1 &amp; 0 &amp; 0 &amp; 1 &amp; 0 \\
                  0 &amp; 1 &amp; 1 &amp; 0 &amp; 0 &amp; 0 &amp; 1
                  \end{pmatrix}
                </me>
              </p>
            </li>
          </ol>
        </p>
      </statement>
      <hint>
        <p>
          (a) A canonical parity-check matrix with standard generator matrix
          <me>
            G = 
            \begin{pmatrix}
            1 \\ 1 \\ 0 \\ 0 \\ 1
            \end{pmatrix}
          </me>.
        </p>

        <p>
          (c) A canonical parity-check matrix with standard generator matrix
          <me>
            G = 
            \begin{pmatrix}
            1 &amp; 0 \\
            0 &amp; 1 \\
            1 &amp; 1 \\
            1 &amp; 0
            \end{pmatrix}
          </me>.
        </p>
      </hint>
    </exercise>

    <exercise number="12" xml:id="algcodes-exercise-syndromes">
      <statement>
        <p>
          List all possible syndromes for the codes generated by each of the matrices in <xref ref="algcodes-exercise-check-matrices"/>.
        </p>
      </statement>
      <hint>
        <p>
          (a) All possible syndromes occur.
        </p>
      </hint>
    </exercise>

    <exercise number="13" xml:id="algcodes-exercise-syndrome-transmission-error">
      <statement>
        <p>
          Let
          <me>
            H =
            \begin{pmatrix}
            0 &amp; 1 &amp; 1 &amp; 1 &amp; 1 \\
            0 &amp; 0 &amp; 0 &amp; 1 &amp; 1 \\
            1 &amp; 0 &amp; 1 &amp; 0 &amp; 1
            \end{pmatrix}
          </me>.
          Compute the syndrome caused by each of the following transmission errors.

          <ol>
            <li>
              <p>
                An error in the first bit.
              </p>
            </li>

            <li>
              <p>
                An error in the third bit.
              </p>
            </li>

            <li>
              <p>
                An error in the last bit.
              </p>
            </li>

            <li>
              <p>
                Errors in the third and fourth bits.
              </p>
            </li>
          </ol>
        </p>
      </statement>
    </exercise>

    <exercise number="14" xml:id="algcodes-exercise-cosets-Z8">
      <statement>
        <p>
          Let <m>C</m> be the group code in
          <m>{\mathbb Z}_2^3</m> defined by the codewords <m>(000)</m> and <m>(111)</m>.
          Compute the cosets of <m>C</m> in <m>{\mathbb Z}_2^3</m>.
          Why was there no need to specify right or left cosets?
          Give the single transmission error, if any,
          to which each coset corresponds.
        </p>
      </statement>
    </exercise>

    <exercise number="15" xml:id="algcodes-exercise-find-cosets">
      <statement>
        <p>
          For each of the following matrices,
          find the cosets of the corresponding code <m>C</m>.
          Give a decoding table for each code if possible.

          <ol cols="2">
            <li>
              <p>
                <me>
                  \begin{pmatrix}
                  0 &amp; 1 &amp; 0 &amp; 0 &amp; 0 \\
                  1 &amp; 0 &amp; 1 &amp; 0 &amp; 1 \\
                  1 &amp; 0 &amp; 0 &amp; 1 &amp; 0
                  \end{pmatrix}
                </me>
              </p>
            </li>

            <li>
              <p>
                <me>
                  \begin{pmatrix}
                  0 &amp; 0 &amp; 1 &amp; 0 &amp; 0  \\
                  1 &amp; 1 &amp; 0 &amp; 1 &amp; 0 \\
                  0 &amp; 1 &amp; 0 &amp; 1 &amp; 0 \\
                  1 &amp; 1 &amp; 0 &amp; 0 &amp; 1
                  \end{pmatrix}
                </me>
              </p>
            </li>

            <li>
              <p>
                <me>
                  \begin{pmatrix}
                  1 &amp; 0 &amp; 0 &amp; 1 &amp; 1 \\
                  0 &amp; 1 &amp; 0 &amp; 1 &amp; 1
                  \end{pmatrix}
                </me>
              </p>
            </li>

            <li>
              <p>
                <me>
                  \begin{pmatrix}
                  1 &amp; 0 &amp; 0 &amp; 1 &amp; 1 &amp; 1 &amp; 1 \\
                  1 &amp; 1 &amp; 1 &amp; 0 &amp; 0 &amp; 1 &amp; 1 \\
                  1 &amp; 0 &amp; 1 &amp; 0 &amp; 1 &amp; 0 &amp; 1 \\
                  1 &amp; 1 &amp; 1 &amp; 0 &amp; 0 &amp; 1 &amp; 0
                  \end{pmatrix}
                </me>
              </p>
            </li>
          </ol>
        </p>
      </statement>
      <hint>
        <p>
          (a) <m>C</m>, <m>(10000) + C</m>,
          <m>(01000) + C</m>, <m>(00100) + C</m>, <m>(00010) + C</m>,
          <m>(11000) + C</m>, <m>(01100) + C</m>, <m>(01010) + C</m>.
          A decoding table does not exist for <m>C</m> since this is only a single error-detecting code.
        </p>
      </hint>
    </exercise>

    <exercise number="16" xml:id="algcodes-exercise-proof-distance">
      <statement>
        <p>
          Let <m>{\mathbf x}</m>, <m>{\mathbf y}</m>,
          and <m>{\mathbf z}</m> be binary <m>n</m>-tuples.
          Prove each of the following statements.

          <ol>
            <li>
              <p>
                <m>w({\mathbf x}) = d( {\mathbf x}, {\mathbf 0})</m>
              </p>
            </li>

            <li>
              <p>
                <m>d( {\mathbf x}, {\mathbf y}) = d( {\mathbf x} + {\mathbf z}, {\mathbf y} + {\mathbf z} )</m>
              </p>
            </li>

            <li>
              <p>
                <m>d({\mathbf x}, {\mathbf y}) = w({\mathbf x}- {\mathbf y})</m>
              </p>
            </li>
          </ol>
        </p>
      </statement>
    </exercise>

    <exercise number="17" xml:id="algcodes-exercise-metric">
      <statement>
        <p>
          A <term>metric</term> on a set <m>X</m> is a map
          <m>d: X \times X \rightarrow {\mathbb R}</m> satisfying the following conditions.

          <ol>
            <li>
              <p>
                <m>d( {\mathbf x}, {\mathbf y}) \geq 0</m> for all <m>{\mathbf x}, {\mathbf y} \in X</m>;
              </p>
            </li>

            <li>
              <p>
                <m>d( {\mathbf x}, {\mathbf y}) = 0</m> exactly when <m>{\mathbf x} = {\mathbf y}</m>;
              </p>
            </li>

            <li>
              <p>
                <m>d( {\mathbf x}, {\mathbf y})= d( {\mathbf y}, {\mathbf x})</m>;
              </p>
            </li>

            <li>
              <p>
                <m>d( {\mathbf x}, {\mathbf y}) \leq d( {\mathbf x}, {\mathbf z}) + d( {\mathbf z}, {\mathbf y})</m>.
              </p>
            </li>
          </ol>

          In other words,
          a metric is simply a generalization of the notion of distance.
          Prove that Hamming distance is a metric on <m>{\mathbb Z}_2^n</m>.
          Decoding a message actually reduces to deciding which is the closest codeword in terms of distance.
        </p>
      </statement>
    </exercise>

    <exercise number="18" xml:id="algcodes-exercise-i-th-coordinate">
      <statement>
        <p>
          Let <m>C</m> be a linear code.
          Show that either the <m>i</m>th coordinates in the codewords of <m>C</m> are all zeros or exactly half of them are zeros.
        </p>
      </statement>
    </exercise>

    <exercise number="19" xml:id="algcodes-exercise-even-weight">
      <statement>
        <p>
          Let <m>C</m> be a linear code.
          Show that either every codeword has even weight or exactly half of the codewords have even weight.
        </p>
      </statement>
      <hint>
        <p>
          Let <m>{\mathbf x} \in C</m> have odd weight and define a map from the set of odd codewords to the set of even codewords by <m>{\mathbf y} \mapsto {\mathbf x} + {\mathbf y}</m>.
          Show that this map is a bijection.
        </p>
      </hint>
    </exercise>

    <exercise number="20" xml:id="algcodes-exercise-even-weight-linear-code">
      <statement>
        <p>
          Show that the codewords of even weight in a linear code <m>C</m> are also a linear code.
        </p>
      </statement>
    </exercise>

    <exercise number="21" xml:id="algcodes-exercise-ascii-characters">
      <statement>
        <p>
          If we are to use an error-correcting linear code to transmit the 128 <acro>ASCII</acro> characters,
          what size matrix must be used?
          What size matrix must be used to transmit the extended <acro>ASCII</acro> character set of 256 characters?
          What if we require only error detection in both cases?
        </p>
      </statement>
    </exercise>

    <exercise number="22" xml:id="algcodes-exercise-canonical-parity-check-matrix">
      <statement>
        <p>
          Find the canonical parity-check matrix that gives the even parity check bit code with three information positions.
          What is the matrix for seven information positions?
          What are the corresponding standard generator matrices?
        </p>
      </statement>
    </exercise>

    <exercise number="23" xml:id="algcodes-exercise-20-information-positions">
      <statement>
        <p>
          How many check positions are needed for a single error-correcting code with 20 information positions?
          With 32 information positions?
        </p>
      </statement>
      <hint>
        <p>
          For <m>20</m> information positions,
          at least 6 check bits are needed to ensure an error-correcting code.
        </p>
      </hint>
    </exercise>

    <exercise number="24" xml:id="algcodes-exercise-binary-code">
      <statement>
        <p>
          Let <m>{\mathbf e}_i</m> be the binary <m>n</m>-tuple with a <m>1</m> in the <m>i</m>th coordinate and <m>0</m>'s elsewhere and suppose that <m>H \in {\mathbb M}_{m \times n}({\mathbb Z}_2)</m>.
          Show that <m>H{\mathbf e}_i</m> is the <m>i</m>th column of the matrix <m>H</m>.
        </p>
      </statement>
    </exercise>

    <exercise number="25" xml:id="algcodes-exercise-dual-code">
      <statement>
        <p>
          Let <m>C</m> be an <m>(n,k)</m>-linear code.
          Define the <term>dual</term> or
          <term>orthogonal code</term> of <m>C</m>  to be
          <me>
            C^\perp = \{ {\mathbf x} \in {\mathbb Z}_2^n :  {\mathbf x} \cdot {\mathbf y} = 0 \text{ for all } {\mathbf y} \in C \}
          </me>.
          <ol>
            <li>
              <p>
                Find the dual code of the linear code <m>C</m> where <m>C</m> is given by the matrix
                <me>
                  \begin{pmatrix}
                  1 &amp; 1 &amp; 1 &amp; 0 &amp; 0 \\
                  0 &amp; 0 &amp; 1 &amp; 0 &amp; 1 \\
                  1 &amp; 0 &amp; 0 &amp; 1 &amp; 0
                  \end{pmatrix}
                </me>.
              </p>
            </li>

            <li>
              <p>
                Show that <m>C^\perp</m> is an <m>(n, n-k)</m>-linear code.
              </p>
            </li>

            <li>
              <p>
                Find the standard generator and parity-check matrices of <m>C</m> and <m>C^\perp</m>.
                What happens in general?
                Prove your conjecture.
              </p>
            </li>
          </ol>
        </p>
      </statement>
    </exercise>

    <exercise number="26" xml:id="algcodes-exercise-hamming-code">
      <statement>
        <p>
          Let <m>H</m> be an <m>m \times n</m> matrix over <m>{\mathbb Z}_2</m>,
          where the <m>i</m>th column is the number <m>i</m> written in binary with <m>m</m> bits.
          The null space of such a matrix is called a
          <term>Hamming code</term>.

          <ol>
            <li>
              <p>
                Show  that the matrix
                <me>
                  H =
                  \begin{pmatrix}
                  0 &amp; 0 &amp; 0 &amp; 1 &amp; 1 &amp; 1 \\
                  0 &amp; 1 &amp; 1 &amp; 0 &amp; 0 &amp; 1 \\
                  1 &amp; 0 &amp; 1 &amp; 0 &amp; 1 &amp; 0
                  \end{pmatrix}
                </me>
                generates a Hamming code.
                What are the error-correcting properties of a Hamming code?
              </p>
            </li>

            <li>
              <p>
                The column corresponding to the syndrome also marks the bit that was in error;
                that is, the <m>i</m>th column of the matrix is <m>i</m> written as a binary number,
                and the syndrome immediately tells us which bit is in error.
                If the received word is <m>(101011)</m>, compute the syndrome.
                In which bit did the error occur in this case,
                and what codeword was originally transmitted?
              </p>
            </li>

            <li>
              <p>
                Give a binary matrix <m>H</m> for the Hamming code with six information positions and four check positions.
                What are the check positions and what are the information positions?
                Encode the messages <m>(101101)</m> and <m>(001001)</m>.
                Decode the received words <m>(0010000101)</m> and <m>(0000101100)</m>.
                What are the possible syndromes for this code?
              </p>
            </li>

            <li>
              <p>
                What is the number of check bits and the number of information bits in an <m>(m,n)</m>-block Hamming code?
                Give both an upper and a lower bound on the number of information bits in terms of the number of check bits.
                Hamming codes having the maximum possible number of information bits with <m>k</m> check bits are called <term>perfect</term>.
                Every possible syndrome except <m>{\mathbf 0}</m> occurs as a column.
                If the number of information bits is less than the maximum,
                then the code is called <term>shortened</term>.
                In this case,
                give an example showing that some syndromes can represent multiple errors.
              </p>
            </li>
          </ol>
        </p>
      </statement>
    </exercise>

  </exercises>
  <exercises xml:id="algcodes-exercises-programming">
    <title>Programming Exercises</title>
    <exercise>
      <statement>
        <p>
          Write a program to implement a <m>(16, 12)</m>-linear code.
          Your program should be able to encode and decode messages using coset decoding.
          Once your program is written,
          write a program to simulate a binary symmetric channel with transmission noise.
          Compare the results of your simulation with the theoretically predicted error probability.
        </p>
      </statement>
    </exercise>

  </exercises>

  <references xml:id="algcodes-references">
    <title>References and Suggested Readings</title>
    <biblio type="raw">
<!-- was [1] -->
      Blake, I. F. <q>Codes and Designs,</q>
      <title>Mathematics Magazine</title>
      <volume>52</volume>
      (1979), 81<ndash/>95.
    </biblio>
        <!-- Reference updated - TWJ 6/1/2010 -->
    <biblio type="raw">
<!-- was [2] -->
      Hill, R.
      <title>A First Course in Coding Theory</title>. Oxford University Press, Oxford, 1990.
    </biblio>

    <biblio type="raw">
<!-- was [3] -->
      Levinson, N. <q>Coding Theory: A Counterexample to G. H. Hardy's Conception of Applied Mathematics,</q>
      <title>American Mathematical Monthly</title>
      <volume>77</volume>
      (1970), 249<ndash/>58.
    </biblio>
        <!-- Reference updated - TWJ 6/1/2010 -->
    <biblio type="raw">
<!-- was [4] -->
      Lidl, R. and Pilz, G.
      <title>Applied Abstract Algebra</title>. 2nd ed. Springer, New York, 1998.
    </biblio>
        <!-- Reference updated - TWJ 6/1/2010 -->
    <biblio type="raw">
<!-- was [5] -->
      MacWilliams, F. J. and Sloane, N. J. A.
      <title>The Theory of Error-Correcting Codes</title>. North-Holland Mathematical Library, 16, Elsevier, Amsterdam, 1983.
    </biblio>

    <biblio type="raw">
<!-- was [6] -->
      Roman, S.
      <title>Coding and Information Theory</title>. Springer-Verlag, New York, 1992.
    </biblio>

    <biblio type="raw">
<!-- was [7] -->
      Shannon, C. E. <q>A Mathematical Theory of Communication,</q>
      <title>Bell System Technical Journal</title>
      <volume>27</volume>
      (1948), 379<ndash/>423, 623<ndash/>56.
    </biblio>

    <biblio type="raw">
<!-- was [8] -->
      Thompson, T. M.
      <title>From Error-Correcting Codes through Sphere Packing to Simple Groups</title>. Carus Monograph Series, No. 21. Mathematical Association of America, Washington, DC, 1983.
    </biblio>
        <!-- Reference updated - TWJ 6/1/2010 -->
    <biblio type="raw">
<!-- was [9] -->
      van Lint, J. H.
      <title>Introduction to Coding Theory</title>. Springer, New York, 1999.
    </biblio>
  </references>



  <section xml:id="algcodes-sage" component="sage-info">
    <title>Sage</title>
    <introduction>
      <p>
        Sage has a full suite of linear codes and a variety of methods that may be used to investigate them.
      </p>
    </introduction>

    <subsection>
      <title>Constructing Linear Codes</title>
      <p>
        The <c>codes</c> object can be used to get a concise listing of the available implemented codes.
        Type <c>codes.</c> and press the <c>Tab</c> key and most interfaces to Sage will give you a list.
        You can then use a question mark at the end of a method name to learn about the various parameters.
      </p>

            <sage doctest="not tested">
                <input>codes.</input>
            </sage>

      <p>
        We will use the classic binary Hamming <m>(7,4)</m> code as an illustration.  <q>Binary</q>
        means we have vectors with just 0's and 1's, the <m>7</m> is the length and means the vectors have <m>7</m> coordinates,
        and the <m>4</m> is the dimension,
        meaning this code has <m>2^4=16</m> vectors comprising the code.
        The documentation assumes we know a few things from later in the course.
        We use <c>GF(2)</c> to specify that our code is binary <mdash/> this will make more sense at the end of the course.
        A second parameter is <c>r</c> and we can see from the formulas in the documenation that setting <c>r=3</c> will give length <m>7</m>.
      </p>

            <sage>
                <input>H = codes.HammingCode(GF(2), 3); H</input>
                <output>[7, 4] Hamming Code over GF(2)</output>
            </sage>
    </subsection>

    <subsection>
      <title>Properties of Linear Codes</title>
      <p>
        We can examine the Hamming code we just built.
        First the dimension.
      </p>

            <sage>
                <input>H.dimension()</input>
                <output>4</output>
            </sage>

      <p>
        The code is small enough that we can list all the codewords.
      </p>

            <sage>
                <input>H.list()</input>
                <output>
            [(0, 0, 0, 0, 0, 0, 0), (1, 0, 0, 0, 0, 1, 1), (0, 1, 0, 0, 1, 0, 1),
             (1, 1, 0, 0, 1, 1, 0), (0, 0, 1, 0, 1, 1, 0), (1, 0, 1, 0, 1, 0, 1),
             (0, 1, 1, 0, 0, 1, 1), (1, 1, 1, 0, 0, 0, 0), (0, 0, 0, 1, 1, 1, 1),
             (1, 0, 0, 1, 1, 0, 0), (0, 1, 0, 1, 0, 1, 0), (1, 1, 0, 1, 0, 0, 1),
             (0, 0, 1, 1, 0, 0, 1), (1, 0, 1, 1, 0, 1, 0), (0, 1, 1, 1, 1, 0, 0),
             (1, 1, 1, 1, 1, 1, 1)]
            </output>
            </sage>

      <p>
        The minimum distance is perhaps one of the most important properties.
        Hamming codes always have minimum distance <m>d=3</m>,
        so they are always single error-correcting.
      </p>

            <sage>
                <input>H.minimum_distance()</input>
                <output>3</output>
            </sage>

      <p>
        We know that the parity-check matrix and the generator matrix are useful for the construction,
        description and analysis of linear codes.
        The Sage method names are just a bit cryptic.
        Sage has extensive routines for analyzing matrices with elements from different fields,
        so we perform much of the subsequent analysis of these matrices within Sage.
      </p>

            <sage>
                <input>C = H.parity_check_matrix(); C</input>
                <output>
            [1 0 1 0 1 0 1]
            [0 1 1 0 0 1 1]
            [0 0 0 1 1 1 1]
            </output>
            </sage>

      <p>
        The generator matrix here in the text has
        <em>columns</em> that are codewords,
        and linear combinations of the columns
        (the column space of the matrix)
        are codewords.
        In Sage the generator matrix has <em>rows</em>
        that are codewords and the row space of the matrix is the code.
        So here is another place where we need to mentally translate between a choice made in the text and a choice made by the Sage developers.
      </p>

            <sage>
                <input>G = H.generator_matrix(); G</input>
                <output>
            [1 0 0 0 0 1 1]
            [0 1 0 0 1 0 1]
            [0 0 1 0 1 1 0]
            [0 0 0 1 1 1 1]
            </output>
            </sage>

      <p>
        Here is a partial test that these two matrices are correct,
        exercising <xref ref="algcodes-lemma-parity-check"/>.
        Notice that we need to use the transpose of the generator matrix,
        for reasons described above.
      </p>

            <sage>
                <input>C*G.transpose() == zero_matrix(3, 4)</input>
                <output>True</output>
            </sage>

      <p>
        Note that the parity-check may not be canonical and the generator matrix may not be standard.
        Sage can produce a generator matrix that has a set of columns that forms an identity matrix,
        though no guarantee is made that these columns are the first columns.
        (Columns, not rows.)
        Such a matrix is said to be <term>systematic</term>,
        and the Sage method is <c>.systematic_generator_matrix()</c>.
      </p>

            <sage>
                <input>H.systematic_generator_matrix()</input>
                <output>
            [1 0 0 0 0 1 1]
            [0 1 0 0 1 0 1]
            [0 0 1 0 1 1 0]
            [0 0 0 1 1 1 1]
            </output>
            </sage>
    </subsection>

    <subsection>
      <title>Decoding with a Linear Code</title>
      <p>
        We can decode received messages originating from a linear code.
        Suppose we receive the length <m>7</m> binary vector <c>r</c>.
      </p>

            <sage>
                <input>r = vector(GF(2), [1, 1, 1, 1, 0, 0, 1]); r</input>
                <output>(1, 1, 1, 1, 0, 0, 1)</output>
            </sage>

      <p>
        We can recognize that one or more errors has occured,
        since <c>r</c> is not in the code,
        as the next computation does not yield the zero vector.
      </p>

            <sage>
                <input>C*r</input>
                <output>(1, 1, 0)</output>
            </sage>

      <p>
        A linear code has a <c>.decode</c> method.
        You may choose from several different algorithms,
        while the Hamming codes have their own custom algorithm.
        The default algorithm is syndrome decoding.
      </p>

            <sage>
                <input>H.decode_to_code(r)</input>
                <output>(1, 1, 0, 1, 0, 0, 1)</output>
            </sage>

      <p>
        So if we are willing to assume that only one error occured
        (which we might,
        if the probability of an indivual entry of the vector being in error is very low),
        then we see that an error occured in the third position.
      </p>

      <p>
        Remember that it could happen that there was more than just one error.
        For example,
        suppose the message was the same as before and errors occurred in the third,
        fifth and sixth locations.
      </p>

            <sage>
                <input>
            message = vector(GF(2), [1, 1, 0, 1, 0, 0, 1])
            errors = vector(GF(2), [0, 0, 1, 0, 1, 1, 0])
            received = message + errors
            received
            </input>
                <output>(1, 1, 1, 1, 1, 1, 1)</output>
            </sage>

      <p>
        It then appears that we have received a codeword,
        so we assume no errors at all, and decode incorrectly.
      </p>

            <sage>
                <input>H.decode_to_code(received) == message</input>
                <output>False</output>
            </sage>
            <sage>
                <input>H.decode_to_code(received) == received</input>
                <output>True</output>
            </sage>
    </subsection>
  </section>
  <exercises xml:id="algcodes-sage-exercises" component="sage-exercises">
    <title>Sage Exercises</title>
    <exercise number="1" xml:id="algcodes-sage-exercise-golay-code">
      <statement>
        <p>
          Create the (binary) Golay code with the <c>codes.GolayCode()</c> constructor.
          Read the documentation to be sure you build the binary version
          (not ternary),
          and do not build the extended version
          (which is the default).
        </p>

        <ol>
          <li>
            <p>
              Use Sage methods to compute the length,
              dimension and minimum distance of the code.
            </p>
          </li>

          <li>
            <p>
              How many errors can this code detect?
              How many can it correct?
            </p>
          </li>

          <li>
            <p>
              Find a nonzero codeword and introduce three errors by adding a vector with three 1's
              (your choice)
              to create a received message.
              Show that the message is decoded properly.
            </p>
          </li>

          <li>
            <p>
              Recycle your choices from the previous part,
              but now add one more error.
              Does the new received message get decoded properly?
            </p>
          </li>
        </ol>
      </statement>
    </exercise>

    <exercise number="2" xml:id="algcodes-sage-exercise-parity-check-bit">
      <statement>
        <p>
          One technique for improving the characteristics of a code is to add an overall parity-check bit,
          much like the lone parity-check bit of the <acro>ASCII</acro> code described in <xref ref="algcodes-example-even-parity"/>.
          Such codes are referred to as the
          <term>extended</term> version of the original.
        </p>

        <ol>
          <li>
            <p>
              Construct the (binary) Golay code and obtain the parity-check matrix.
              Use Sage commands to enlarge this matrix to create a new parity check matrix that has an additional overall parity-check bit.
              You may find the matrix methods <c>.augment()</c> and <c>.stack()</c> useful,
              as well as the constructors <c>zero_vector()</c> and <c>ones_matrix()</c>(remembering that we specify the binary entries as being from the field <c>GF(2)</c>.)
            </p>

            <p>
              Create the extended code by supplying your enlarged parity-check matrix to the <c>codes.from_parity_check_matrix()</c> constructor and compute the length,
              dimension and minimum distance of the extended code.
            </p>
          </li>

          <li>
            <p>
              How are the properties of this new code better?
              At what cost?
            </p>
          </li>

          <li>
            <p>
              Now create the extended (binary) Golay code with the Sage constructor <c>codes.GolayCode()</c> and the correct keyword to obtain the extended version.
              With luck, the sorted lists of your codewords and Sage's codewords will be equal.
              If not, the linear code method <c>.is_permutation_equivalent()</c> should return <c>True</c> to indicate that your code and Sage's are just rearrangements of each other.
            </p>
          </li>
        </ol>
      </statement>
    </exercise>

    <exercise number="3" xml:id="algcodes-sage-exercise-buggy-sage-code">
      <statement>
        <p>
          <em>Note:</em> This problem is on holiday
          (as of Sage 6.7),
          while some buggy Sage code for the minimum distance of a Hamming code gets sorted out.
          The <c>r = 2</c> case produces an error message and for <c>r &gt; 5</c> the computation of the minimum distance has become intolerably slow.
          So it is a bit harder to make a reasonable conjecture from just <m>3</m> cases.
        </p>

        <p>
          The dual of an <m>(n,k)</m> block code is formed as all the set of all binary vectors which are orthogonal to every vector of the original code.
          <xref ref="algcodes-exercise-dual-code"/> describes this construction and asks about some of its properties.
        </p>

        <p>
          You can construct the dual of a code in Sage with the <c>.dual_code()</c> method.
          Construct the binary Hamming codes, and their duals,
          with the parameter <c>r</c> ranging from <c>2</c> to <c>5</c>,
          inclusive.
          Build a table with six columns
          (perhaps employing the <c>html.table()</c> function)
          that lists <m>r</m>,
          the length of the codes,
          the dimensions of the original and the dual,
          and the minimum distances of the orginal and the dual.
        </p>

        <p>
          Conjecture formulas for the dimension and minimum distance of the dual of the Hamming code as expressions in the parameter <m>r</m>.
        </p>
      </statement>
    </exercise>

    <exercise number="4" xml:id="algcodes-sage-exercise-perfect-code">
      <statement>
        <p>
          A code with minimum distance <m>d</m> is called <term>perfect</term>
          if every possible vector is within Hamming distance <m>(d-1)/2</m> of some codeword.
          If we expand our notion of geometry to account for the Hamming distance as the metric,
          then we can speak of a sphere of radius <m>r</m> around a vector (or codeword.
          For a code of length <m>n</m>,
          such a sphere will contain
          <me>
            1 + {n\choose 1} + {n\choose 2} + \cdots + {n\choose r}
          </me>
          vectors within in it.
          For a perfect code,
          the spheres of radius <m>d</m> centered at the codewords of the code will exactly partition the entire set of all possible vectors.
          (This is the connection that means that coding theory meshes with sphere packing problems.)
        </p>

        <p>
          A consequence of a code of dimension <m>k</m> being perfect is that
          <me>
            2^k\left({n\choose 0} + {n\choose 1} + {n\choose 2} + \cdots + {n\choose \frac{d-1}{2}}\right) = 2^n
          </me>
          Conversely, if a code has minimum distance <m>d</m> and the condition above is true,
          then the code is perfect.
        </p>

        <p>
          Write a Python function,
          named <c>is_perfect()</c> which accepts a linear code as input and returns <c>True</c> or <c>False</c>.
          Demonstrate your function by checking that the (binary) Golay code is perfect,
          and then use a loop to verify that the (binary) Hamming codes are perfect for all lengths below <m>32</m>.
        </p>
      </statement>
    </exercise>
        <!-- Potential exercise 5: probabilty and Shannon bound -->
  </exercises>
</chapter>