(some) LaTeX environments for Jupyter notebook

In [1]:
%%html
<style>
    .prompt{
        display: none;
    }    

</style>

\subsection{ What's new }

July 27, 2016 -

  • In this version we have reworked equation numbering. In the previous version, we have implemented a specialized counter and detected equations rendering for updating the counter. Meanwhile, this feature has been introduced in MathJax and now we rely on MathJax implementation. We still have keep the capability of displaying only equation labels (instead of numbers). The numbering is automatically updated and is document-wide.
  • We have completely reworked the notebook conversion to plain $\LaTeX$ and html. We provide specialized exporters, pre and post processors, templates. We also added entry-points to simplify the conversion process. It is now as simple as
    jupyter nbconvert --to html_lenvs FILE.ipynb
    
    to convert FILE.ipynb into html while keeping all the features of the latex_envs notebook extension in the converted version.

Presentation and main features

This extension for IPython 3.x or Jupyter enables to use some LaTeX commands and environments in the notebook's markdown cells.

\begin{enumerate} \item **LaTeX commands and environments** \begin{itemize} \item support for some LaTeX commands within markdown cells, *e.g.* `\textit`, `\textbf`, `\underline` \item support for **theorems-like environments** \item support for **lists**: *enumerate, itemize*, \item limited support for a **figure environment**, \item support for an environment *listing*, \item additional *textboxa* environment \end{itemize} \item **Citations and bibliography** \begin{itemize} \item support for `\cite` with creation of a References section \end{itemize} \item **Document-wide numbering of equations, support for `\label` and `\ref`** \item **Configuration toolbar** \item Styles can be customized in the `latex_env.css` stylesheet \end{enumerate}

A simple illustration is as follows: on can type the following in a markdown cell

\begin{listing} \begin{theorem} \label{theo:dotp} Let $u$ and $v$ be two vectors of $\mathbb{R}^n$. The dot product can be expressed as \begin{equation} \label{eq:dotp} u^Tv = |u||v| \cos \theta, \end{equation} where $\theta$ is the angle between $u$ and $v$ ... \end{theorem} Then one can reference the equation (\ref{eq:dotp}) in theorem \ref{theo:dotp}. \end{listing}

and have it rendered as

\begin{theorem} \label{theo:dotp} Let $u$ and $v$ be two vectors of $\mathbb{R}^n$. The dot product can be expressed as \begin{equation} \label{eq:dotp} u^Tv = |u||v| \cos \theta, \end{equation} where $\theta$ is the angle between $u$ and $v$ ... \end{theorem}

Then one can reference the equation (\ref{eq:dotp}) in theorem \ref{theo:dotp}.

Implementation principle

The main idea is to override the standard Markdown renderer in order to add a small parsing of LaTeX expressions and environments. This heavily uses regular expressions. The LaTeX expression are then rendered using an html version. For instance \underline {something} is rendered as <u> something </u>, that is \underline{something}. The environments are replaced by an html tag with a class derived from the name of the environment. For example, a definition denvronment will be replaced by an html rendering corresponding to the class latex_definition. The styles associated with the different classes are sepcified in latex_env.css. These substitutions are implemented in thsInNb4.js.

Support for simple LaTeX commands

We also added some LaTeX commands (e.g. \textit, \textbf, \underline) -- this is useful in the case of copy-paste from a LaTeX document. Labels and references are supported, including for equations.

Available environments

  • theorems-like environments: property, theorem, lemma, corollary, proposition, definition,remark, problem, exercise, example,
  • lists: enumerate, itemize,
  • limited support for a figure environment,
  • an environment listing,
  • textboxa, wich is a textbox environment defined as a demonstration (see below).

More environments can be added easily in the javascript source file thmsInNb4.js. The rendering is done according to the stylesheet latex_env.css, which can be customized.

\begin{remark} When exporting to html, the `latex_env.css` file honored is the file on the Jupyter-notebook-extensions CDN. However, customized css can be added in a `custom.css` file that must reside in the same directory as the notebook itself. \end{remark}

Automatic numerotation, labels and references

Counters for numerotation are implemented: one for theorems-like environments, a second for exercises-like environments and a third one for numbering figures.
Mathjax-equations with a label are also numbered document-wide. An anchor is created for any label which enables to links things within the document: \label and \ref are both supported. A limitation is that numbering is updated (incremented) each time a cell is rendered. A toolbar button is provided to reset the counters and refresh the rendering of the whole document.

\label{example:mixing} A simple example is as follows, featuring automatic numerotation, and the use of labels and references. Also note that standard markdown can be present in the environment and is interpreted. The rendering is done according to the stylesheet latex_env.css, which of course, can be tailored to specific uses and tastes.

\begin{listing} \begin{definition} \label{def:FT} Let $x[n]$ be a sequence of length $N$. Then, its **Fourier transform** is given by \begin{equation} \label{eq:FT} X[k]= \frac{1}{N} \sum_{n=0}^{N-1} x[n] e^{-j2\pi \frac{kn}{N}} \end{equation} \end{definition} \end{listing}
\begin{definition} \label{def:FT} Let $x[n]$ be a sequence of length $N$. Then, its **Fourier transform** is given by \begin{equation} \label{eq:FT2} X[k]= \frac{1}{N} \sum_{n=0}^{N-1} x[n] e^{-j2\pi \frac{kn}{N}} \end{equation} \end{definition}

It is now possible to refer to the definition and to the equation by their labels, as in:

\begin{listing} As an example of Definition \ref{def:FT}, consider the Fourier transform (\ref{eq:FT2}) of a pure cosine wave given by $$ x[n]= \cos(2\pi k_0 n/N), $$ where $k_0$ is an integer. \end{listing}

As an example of Definition \ref{def:FT}, consider the Fourier transform (\ref{eq:FT2}) of a pure cosine wave given by $$ x[n]= \cos(2\pi k_0 n/N), $$ where $k_0$ is an integer. Its Fourier transform is given by $$ X[k] = \frac{1}{2} \left( \delta[k-k_0] + \delta[k-k_0] \right), $$ modulo $N$.

Bibliography

Usage

It is possible to cite bibliographic references using the standard LaTeX \cite mechanism. The extension looks for the references in a bibTeX file, by default biblio.bib in the same directory as the notebook. The name of this file can be modified in the configuration toolbar. It is then possible to cite works in the notebook, e.g.

\begin{listing} The main paper on IPython is definitively \cite{PER-GRA:2007}. Other interesting references are certainly \cite{mckinney2012python, rossant2013learning}. Interestingly, a presentation of the IPython notebook has also be published recently in Nature \cite{shen2014interactive}. \end{listing}

The main paper on IPython is definitively \cite{PER-GRA:2007}. Other interesting references are certainly \cite{mckinney2012python, rossant2013learning}. Interestingly, a presentation of the IPython notebook has also be published recently in Nature \cite{shen2014interactive}.

Implementation

The implemention uses several snippets from the nice icalico-document-tools extension that also considers the rendering of citations in the notebook. We also use a modified version of the bibtex-js parser for reading the references in the bibTeX file. The different functions are implemented in bibInNb4.js. The rendering of citations calls can adopt three styles (Numbered, by key or apa-like) -- this can be selected in the configuration toolbar. It is also possible to customize the rendering of references in the reference list. A citation template is provided in the beginning of file latex_envs.js:

var cit_tpl = {
// feel free to add more types and customize the templates
    'INPROCEEDINGS': '%AUTHOR:InitialsGiven%, ``_%TITLE%_\'\', %BOOKTITLE%, %MONTH% %YEAR%.',
    ... etc

The keys are the main types of documents, eg inproceedings, article, inbook, etc. To each key is associated a string where the %KEYWORDS% are the fields of the bibtex entry. The keywords are replaced by the correponding bibtex entry value. The template string can formatted with additional words and effects (markdown or LaTeX are commands are supported)

Figure environment

Finally, it is sometimes useful to integrate a figure within a markdown cell. The standard markdown markup for that is ![link](image), but a limitation is that the image can not be resized, can not be referenced and is not numbered. Furthermore it can be useful for re-using existing code. Threfore we have added a limited support for the figure environment. This enables to do something like

\begin{listing} \begin{figure} \centerline{\includegraphics[width=10cm]{example.png}} \caption{\label{fig:example} This is an example of figure included using LaTeX commands.} \end{figure} \end{listing}

which renders as

\begin{figure} \centerline{\includegraphics[width=10cm]{example.png}} \caption{\label{fig:example} This is an example of figure included using LaTeX commands.} \end{figure}

Of course, this Figure can now be referenced:

\begin{listing} Figure \ref{fig:example} shows a second filter with input $X_2$, output $Y_2$ and an impulse response denoted as $h_2(n)$ \end{listing}

Figure \ref{fig:example} shows a second filter with input $X_2$, output $Y_2$ and an impulse response denoted as $h_2(n)$

figcaption

For Python user, we have added in passing a simple function in the latex_envs.py library. This function can be imported classicaly, eg from latex_envs import figcaption. Then, this function enables to specify a caption and a label for the next plot. In turn, when exporting to $\LaTeX$, the corresponding plot is converted to a nice figure environement with a label and a caption.

In [2]:
%matplotlib inline
import matplotlib.pyplot as plt
from jupyter_contrib_nbextensions.nbconvert_support.latex_envs import figcaption
from numpy import pi, sin, arange
figcaption("This is a nice sine wave", label="fig:mysin")
plt.plot(sin(2*pi*0.01*arange(100)))
Caption: This is a nice sine wave
Out[2]:
[<matplotlib.lines.Line2D at 0x7f8a1872ba20>]

Other features

  • As shown in the examples, eg \ref{example:mixing}, it is possible to mix LaTeX and markdown markup in environments
  • Environments can be nested. However, it is not possible, because of limlitations of regular expressions, to nest environments of the same king (eg itemize into itemize into itemize...)

User interface

Buttons on main toolbar

On the main toolbar, the extension provides three buttons three buttons The first one can be used to refresh the numerotation of equations and references in all the document. The second one fires the reading of the bibliography bibtex file and creates (or updates) the reference section. Finally the third one is a toogle button that opens or closes the configuration toolbar.

Configuration toolbar

The configuration toolbar configuration toolbar enables to enter some configuration options for the extension. First, one can indicate the name of the bibtex file. If this file is not found and the user creates the reference section, then this section will indicate that the file was not found. The references drop-down menu enables to choose the type of reference calls. The Equations input box enable to initiate numbering of equations at the given number (this may be useful for complex documents in several files/parts). Finally the last drop-down menu let the user choose to number equation or to display their label instead. These configuration options are then stored in the notebook's metadata (and restored on reload).

Installation, usage and further examples

Installation

The extension consists in several javascript scripts: latex_envs.js, thmsInNb4.js, bibInNb4.js and initNb.js, together with a stylesheet latex_envs.css. You may follow the instructions in the wiki to install the extension.

Definitively, the simplest way to install and enable the extension is to follow the instrutions in the Jupyter-notebook-extensions repo. Once this is done, you can open a tab at http://localhost:8888/nbextensions to anable and configurate the various extensions.

First example (continued)

We continue the first example on fthe Fourier transform definition \ref{def:FT} in order to show that, of course, we can illustrate things using a simple code. Since the Fourier transform is an essential tool in signal processing, We put this in evidence using the textboxa environment -- which is defined here in the css, and that one should define in the LaTeX counterpart:

\begin{listing} \begin{textboxa} The Fourier transform is an extremely useful tool to have in your toolbox! \end{textboxa} \end{listing}
\begin{textboxa} The Fourier transform is an extremely useful tool to have in your toolbox! \end{textboxa}

The Fourier transform of a pure cosine is given by $$ X[k] = \frac{1}{2} \left( \delta[k-k_0] + \delta[k-k_0] \right), $$ modulo $N$. This is illustrated in the following simple script:

In [21]:
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt 
from numpy.fft import fft
k0=4; N=128; n=np.arange(N); k=np.arange(N)
x=np.sin(2*np.pi*k0*n/N)
X=fft(x)
plt.stem(k,np.abs(X))
plt.xlim([0, 20])
plt.title("Fourier transform of a cosine")
_=plt.xlabel("Frequency index (k)")

Second example

This example shows a series of environments, with different facets; links, references, markdown or/and LaTeX formatting within environments. The listing of environments below is typed using the environment listing...

\begin{listing} \begin{definition} \label{def:diffeq} We call \textbf{difference equation} an equation of the form $$ \label{eq:diffeq} y[n]= \sum_{k=1}^{p} a_k y[n-k] + \sum_{i=0}^q b_i x[n-i] $$ \end{definition} \begin{property} If all the $a_k$ in equation (\ref{eq:diffeq}) of definition \ref{def:diffeq} are zero, then the filter has a **finite impulse response**. \end{property} \begin{proof} Let $\delta[n]$ denote the Dirac impulse. Take $x[n]=\delta[n]$ in (\ref{eq:diffeq}). This yields, by definition, the impulse response: $$ \label{eq:fir} h[n]= \sum_{i=0}^q b_i \delta[n-i], $$ which has finite support. \end{proof} \begin{theorem} The poles of a causal stable filter are located within the unit circle in the complex plane. \end{theorem} \begin{example} \label{ex:IIR1} Consider $y[n]= a y[n-1] + x[n]$. The pole of the transfer function is $z=a$. The impulse response $h[n]=a^n$ has infinite support. \end{example} In the following exercise, you will check that the filter is stable iff $a$<1. \begin{exercise}\label{ex:exofilter} Consider the filter defined in Example \ref{ex:IIR1}. Using the **function** `lfilter` of scipy, compute and plot the impulse response for several values of $a$. \end{exercise} \end{listing}

The lines above are rendered as follows (of course everything can be tailored in the stylesheet):

\begin{definition} \label{def:diffeq} We call \textbf{difference equation} an equation of the form \begin{equation} \label{eq:diffeq} y[n]= \sum_{k=1}^{p} a_k y[n-k] + \sum_{i=0}^q b_i x[n-i] \end{equation} \end{definition}

Properties of the filter are linked to the coefficients of the difference equation. For instance, an immediate property is

\begin{property} If all the $a_k$ in equation (\ref{eq:diffeq}) of definition \ref{def:diffeq} are zero, then the filter has a **finite impulse response**. \end{property}\begin{proof} Let $\delta[n]$ denote the Dirac impulse. Take $x[n]=\delta[n]$ in (\ref{eq:diffeq}). This yields, by definition, the impulse response: \begin{equation} \label{eq:fir} h[n]= \sum_{i=0}^q b_i \delta[n-i], \end{equation} which has finite support. \end{proof}\begin{theorem} The poles of a causal stable filter are located within the unit circle in the complex plane. \end{theorem}\begin{example} \label{ex:IIR1} Consider $y[n]= a y[n-1] + x[n]$. The pole of the transfer function is $z=a$. The impulse response $h[n]=a^n$ has infinite support. \end{example}

In the following exercise, you will check that the filter is stable iff $a$<1.

\begin{exercise}\label{ex:exofilter} Consider the filter defined in Example \ref{ex:IIR1}. Using the **function** `lfilter` of scipy, compute and plot the impulse response for several values of $a$. \end{exercise}
\begin{listing} The solution of exercise \ref{ex:exofilter}, which uses a difference equation as in Definition \ref{def:diffeq}: \end{listing}

The solution of exercise \ref{ex:exofilter}, which uses a difference equation as in Definition \ref{def:diffeq}:

In [22]:
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt 
from scipy.signal import lfilter
d=np.zeros(100); d[0]=1 #dirac impulse
alist=[0.2, 0.8, 0.9, 0.95, 0.99, 0.999, 1.001, 1.01]
for a in alist:
    h=lfilter([1], [1, -a],d)
    _=plt.plot(h, label="a={}".format(a))
plt.ylim([0,1.5])
plt.xlabel('Time')
_=plt.legend()

Third example:

This example shows that environments like itemize or enumerate are also available. As already indicated, this is useful for copying text from a TeX file. Following the same idea, text formating commands \textit, \textbf, \underline, etc are also available.

\begin{listing} The following \textit{environments} are available: \begin{itemize} \item \textbf{Theorems and likes} \begin{enumerate} \item theorem, \item lemma, \item corollary \item ... \end{enumerate} \item \textbf{exercises} \begin{enumerate} \item problem, \item example, \item exercise \end{enumerate} \end{itemize} \end{listing}

which gives...

The following \textit{environments} are available:

\begin{itemize} \item \textbf{Theorems and likes} \begin{enumerate} \item theorem, \item lemma, \item corollary \item ... \end{enumerate} \item \textbf{exercises} \begin{enumerate} \item problem, \item example, \item exercise \end{enumerate} \end{itemize}

Conversion to LaTeX and HTML

The extension works in the live-notebook. Since it relies on a bunch of javascript, the notebook does not render as is in services such as nbviewer or github viewer. Similarly, nbconvert does not know of the LaTeX constructs which are used here and therefore does not fully convert notebooks using this extension.

Therefore, we provide specialized templates and exporters to achieve these conversions.

Conversion to html

We provide a template latex_envs.tpl and an exporter class LenvsHTMLExporter (in library latex_envs.py). Using that class, conversion simply amounts to

jupyter nbconvert --to latex_envs.LenvsHTMLExporter FILE.ipynb

A shortcut is also provided

jupyter nbconvert --to html_lenvs FILE.ipynb

It should be noted that the rendering is done exactly in the same way as in the livenotebook. Actually, it is the very same javascript which is run in the html file. The javascript functions are available on the jupyter_notebook_extensions CDN, which means that the rendering of the html files requires an internet connection (this is also true for the rendering of equations with MathJax).

Conversion to LaTeX

We provide two templates thmsInNb_article.tplx and thmsInNb_report.tplx for article and report styles respectively. Anyway one can also use the standard article, report, book templates provided with nbconvert. Simply, we have improved some of the internals styles. More importantly, we provide an exporter class LenvsLatexExporter (also in library latex_envs.py). Using that class, conversion simply amounts to

jupyter nbconvert --to latex_envs.LenvsLatexExporter FILE.ipynb

A shortcut is also provided

jupyter nbconvert --to latex_lenvs FILE.ipynb

In addition, we provide several further options:

  • removeHeaders: Remove headers and footers, (default false)
  • figcaptionProcess: Process figcaptions, (default true)
  • tocrefRemove Remove tocs and ref sections, + some cleaning, (default true),

These options can be specified on the command line as, eg,

jupyter nbconvert --to latex_lenvs --LenvsLatexExporter.removeHeaders=True --  --LenvsLatexExporter.tocrefRemove=False FILE.ipynb
\begin{example} As for an example, the present document has been converted using jupyter nbconvert --to latex_lenvs --LenvsLatexExporter.removeHeaders=True latex_env_doc.ipynb Then the resulting file (without header/footer) has been included in the main file `documentation.tex`, where some LaTeX definitions of environments are done (namely listings, colors, etc) and compiled using xelatex documentation The output can be consulted [here](documentation.pdf). \end{example}

Disclaimer, sources and thanks

This is done in th ehope it can be useful. However there are many impovements possible, in the code and in the documentation.

Contributions will be welcome and deeply appreciated.

Originally, I used a piece of code from the nice online markdown editor stackedit https://github.com/benweet/stackedit/issues/187, where the authors also considered the problem of incorporating LaTeX markup in their markdown.

I also studied and used examples and code from https://github.com/ipython-contrib/IPython-notebook-extensions.

References

(P\'erez and Granger, 2007) P\'erez Fernando and Granger Brian E., ``IPython: a System for Interactive Scientific Computing'', Computing in Science and Engineering, vol. 9, number 3, pp. 21--29, May 2007. online

(McKinney, 2012) Wes McKinney, ``Python for data analysis: Data wrangling with Pandas, NumPy, and IPython'', 2012.

(Rossant, 2013) Cyrille Rossant, ``Learning IPython for interactive computing and data visualization'', 2013.

(Shen, 2014) Shen Helen, ``Interactive notebooks: Sharing the code'', Nature, vol. 515, number 7525, pp. 151--152, 2014.