TL;DR I have been thinking about alternative frontends to the Jupyter notebook. The first step is to abstract communications with Jupyter kernels. I have started writing one of these.

• Project repository
• docs.rs
• crates.io

I am a huge fan of the Jupyter notebook environment. It is a fantastic way to explore a new data analysis approach, and keep results together with the implementation and explanation.

I wanted to understand how the Jupyter notebook/lab frontend interacts with the Kernel backends, and as such have started writing a client library.

Architecture

The Jupyter suite consists of two parts:

• Frontend
• Kernel

Within the frontend is typically a kernel communication abstraction component, what I call the “client”:

The frontend handles nicely presenting the execution results and collecting input from the user (i.e. the notebook). The kernel is the execution engine that can be written in many languages. The client handles communicating with a kernel. It communicates over 4¹ ZeroMQ sockets:

• shell: for client -> kernel requests (and responses)
• iopub: for kernel -> client messages
• heartbeat: for health checks
• stdin: for kernel -> client requests (and responses from the frontend)

This abstraction is what allows the frontend to communicate with different kernels.

Writing a client library

The documentation on the Jupyter wire protocol is pretty good at describing how messages are sent to and from the kernel over the “wire” (the various ZeroMQ sockets). This documentation was used to build a WireMessage abstraction, which:

• is created from a Command enum variant (conversion method),
• gets passed to a Socket; an abstraction around a ZeroMQ socket,
• gets sent over the wire to the kernel,
• a response is sent back and converted to a WireMessage (conversion performed here),
• which is then turned back into a Response object and generally passed back to the user.

I hope that this library will form the beginning of a new Rust-based Jupyter frontend. I have some ideas but that’s for another time.

Links

• Project repository
• docs.rs
• crates.io
• Description of the Jupyter wire protocol
• Creating language kernels for IPython