Release 1.0.0: An Afternoon Hack
IPython 1.0 requires Python ≥ 2.6.5 or ≥ 3.2.1.
It does not support Python 3.0, 3.1, or 2.5.
This is a big release. The principal milestone is the addition of IPython.nbconvert,
but there has been a great deal of work improving all parts of IPython as well.
The previous version (0.13) was released on June 30, 2012,
and in this development cycle we had:
- ~12 months of work.
- ~700 pull requests merged.
- ~600 issues closed (non-pull requests).
- contributions from ~150 authors.
- ~4000 commits.
The amount of work included in this release is so large that we can only cover
here the main highlights; please see our detailed release statistics for links to every issue and pull request closed on GitHub
as well as a full list of individual contributors.
It includes
Reorganization
There have been two major reorganizations in IPython 1.0:
- Added IPython.kernel for all kernel-related code.
This means that IPython.zmq has been removed,
and much of it is now in IPython.kernel.zmq,
some of it being in the top-level IPython.kernel.
- We have removed the frontend subpackage,
as it caused unnecessary depth. So what was IPython.frontend.qt
is now IPython.qt, and so on. The one difference is that
the notebook has been further flattened, so that
IPython.frontend.html.notebook is now just IPython.html.
There is a shim module, so IPython.frontend is still
importable in 1.0, but there will be a warning.
- The IPython sphinx directives are now installed in IPython.sphinx,
so they can be imported by other projects.
Public APIs
For the first time since 0.10 (sorry, everyone),
there is an official public API for starting IPython:
from IPython import start_ipython
start_ipython()
This is what packages should use that start their own IPython session,
but don’t actually want embedded IPython (most cases).
IPython.embed() is used for embedding IPython into the calling namespace,
similar to calling Pdb.set_trace(), whereas start_ipython()
will start a plain IPython session, loading config and startup files as normal.
We also have added:
from IPython import get_ipython
Which is a library function for getting the current IPython instance,
and will return None if no IPython instance is running.
This is the official way to check whether your code is called from inside an IPython session.
If you want to check for IPython without unnecessarily importing IPython,
use this function:
def get_ipython():
"""return IPython instance if there is one, None otherwise"""
import sys
if "IPython" in sys.modules:
import IPython
return IPython.get_ipython()
Core
- The input transformation framework has been reworked. This fixes some corner
cases, and adds more flexibility for projects which use IPython, like SymPy &
SAGE. For more details, see Custom input transformation.
- Exception types can now be displayed with a custom traceback, by defining a
_render_traceback_() method which returns a list of strings, each
containing one line of the traceback.
- A new command, ipython history trim can be used to delete everything but
the last 1000 entries in the history database.
- __file__ is defined in both config files at load time,
and .ipy files executed with %run.
- %logstart and %logappend are no longer broken.
- Add glob expansion for %run, e.g. %run -g script.py *.txt.
- Expand variables ($foo) in Cell Magic argument line.
- By default, iptest will exclude various slow tests.
All tests can be run with iptest –all.
- SQLite history can be disabled in the various cases that it does not behave well.
- %edit works on interactively defined variables.
- editor hooks have been restored from quarantine, enabling TextMate as editor,
etc.
- The env variable PYTHONSTARTUP is respected by IPython.
- The %matplotlib magic was added, which is like the old %pylab magic,
but it does not import anything to the interactive namespace.
It is recommended that users switch to %matplotlib and explicit imports.
- The --matplotlib command line flag was also added. It invokes the new
%matplotlib magic and can be used in the same way as the old --pylab
flag. You can either use it by itself as a flag (--matplotlib), or you
can also pass a backend explicitly (--matplotlib qt or
--matplotlib=wx, etc).
Backwards incompatible changes
- Calling InteractiveShell.prefilter() will no longer perform static
transformations - the processing of escaped commands such as %magic and
!system, and stripping input prompts from code blocks. This functionality
was duplicated in IPython.core.inputsplitter, and the latter version
was already what IPython relied on. A new API to transform input will be ready
before release.
- Functions from IPython.lib.inputhook to control integration with GUI
event loops are no longer exposed in the top level of IPython.lib.
Code calling these should make sure to import them from
IPython.lib.inputhook.
- For all kernel managers, the sub_channel attribute has been renamed to
iopub_channel.
- Users on Python versions before 2.6.6, 2.7.1 or 3.2 will now need to call
IPython.utils.doctestreload.doctest_reload() to make doctests run
correctly inside IPython. Python releases since those versions are unaffected.
For details, see PR #3068 and Python issue 8048.
- The InteractiveShell.cache_main_mod() method has been removed, and
new_main_mod() has a
different signature, expecting a filename where earlier versions expected
a namespace. See PR #3555 for details.
- The short-lived plugin system has been removed. Extensions are the way to go.
NbConvert
The major milestone for IPython 1.0 is the addition of IPython.nbconvert - tools for converting
IPython notebooks to various other formats.
Warning
nbconvert is α-level preview code in 1.0
To use nbconvert to convert various file formats:
ipython nbconvert --to html *.ipynb
See ipython nbconvert --help for more information.
nbconvert depends on pandoc for many of the translations to and from various formats.
Notebook
Major changes to the IPython Notebook in 1.0:
- The notebook is now autosaved, by default at an interval of two minutes.
When you press ‘save’ or Ctrl-S, a checkpoint is made, in a hidden folder.
This checkpoint can be restored, so that the autosave model is strictly safer
than traditional save. If you change nothing about your save habits,
you will always have a checkpoint that you have written,
and an autosaved file that is kept up to date.
- The notebook supports raw_input() / input(), and thus also %debug,
and many other Python calls that expect user input.
- You can load custom javascript and CSS in the notebook by editing the files
$(ipython locate profile)/static/custom/custom.js,css.
- Add %%html, %%svg, %%javascript, and %%latex cell magics
for writing raw output in notebook cells.
- add a redirect handler and anchors on heading cells, so you can link
across notebooks, directly to heading cells in other notebooks.
- Images support width and height metadata,
and thereby 2x scaling (retina support).
- _repr_foo_ methods can return a tuple of (data, metadata),
where metadata is a dict containing metadata about the displayed object.
This is used to set size, etc. for retina graphics. To enable retina matplotlib figures,
simply set InlineBackend.figure_format = 'retina' for 2x PNG figures,
in your IPython config file or via the %config magic.
- Add display.FileLink and FileLinks for quickly displaying HTML links to local files.
- Cells have metadata, which can be edited via cell toolbars.
This metadata can be used by external code (e.g. reveal.js or exporters),
when examining the notebook.
- Fix an issue parsing LaTeX in markdown cells, which required users to type \\\,
instead of \\.
- Notebook templates are rendered with Jinja instead of Tornado.
- %%file has been renamed %%writefile (%%file is deprecated).
- ANSI (and VT100) color parsing has been improved in both performance and
supported values.
- The static files path can be found as IPython.html.DEFAULT_STATIC_FILES_PATH,
which may be changed by package managers.
- IPython’s CSS is installed in static/css/style.min.css
(all style, including bootstrap), and static/css/ipython.min.css,
which only has IPython’s own CSS. The latter file should be useful for embedding
IPython notebooks in other pages, blogs, etc.
- The Print View has been removed. Users are encouraged to test ipython
nbconvert to generate a static view.
Javascript Components
The javascript components used in the notebook have been updated significantly.
- updates to jQuery (2.0) and jQueryUI (1.10)
- Update CodeMirror to 3.14
- Twitter Bootstrap (2.3) for layout
- Font-Awesome (3.1) for icons
- highlight.js (7.3) for syntax highlighting
- marked (0.2.8) for markdown rendering
- require.js (2.1) for loading javascript
Some relevant changes that are results of this:
- Markdown cells now support GitHub-flavored Markdown (GFM),
which includes ```python code blocks and tables.
- Notebook UI behaves better on more screen sizes.
- Various code cell input issues have been fixed.
Kernel
The kernel code has been substantially reorganized.
New features in the kernel:
- Kernels support ZeroMQ IPC transport, not just TCP
- The message protocol has added a top-level metadata field,
used for information about messages.
- Add a data_pub message that functions much like display_pub,
but publishes raw (usually pickled) data, rather than representations.
- Ensure that sys.stdout.encoding is defined in Kernels.
- Stdout from forked subprocesses should be forwarded to frontends (instead of crashing).
IPEP 13
The KernelManager has been split into a KernelManager and a KernelClient.
The Manager owns a kernel and starts / signals / restarts it. There is always zero or one
KernelManager per Kernel. Clients communicate with Kernels via zmq channels,
and there can be zero-to-many Clients connected to a Kernel at any given time.
The KernelManager now automatically restarts the kernel when it dies,
rather than requiring user input at the notebook or QtConsole UI
(which may or may not exist at restart time).
In-process kernels
The Python-language frontends, particularly the Qt console, may now communicate
with in-process kernels, in addition to the traditional out-of-process
kernels. An in-process kernel permits direct access to the kernel namespace,
which is necessary in some applications. It should be understood, however, that
the in-process kernel is not robust to bad user input and will block the main
(GUI) thread while executing. Developers must decide on a case-by-case basis
whether this tradeoff is appropriate for their application.
Parallel
IPython.parallel has had some refactoring as well.
There are many improvements and fixes, but these are the major changes:
- Connections have been simplified. All ports and the serialization in use
are written to the connection file, rather than the initial two-stage system.
- Serialization has been rewritten, fixing many bugs and dramatically improving
performance serializing large containers.
- Load-balancing scheduler performance with large numbers of tasks has been dramatically improved.
- There should be fewer (hopefully zero) false-positives for engine failures.
- Increased compatibility with various use cases that produced serialization / argument errors
with map, etc.
- The controller can attempt to resume operation if it has crashed,
by passing ipcontroller --restore.
- Engines can monitor the Hub heartbeat, and shutdown if the Hub disappears for too long.
- add HTCondor support in launchers
QtConsole
Various fixes, including improved performance with lots of text output,
and better drag and drop support.
The initial window size of the qtconsole is now configurable via IPythonWidget.width
and IPythonWidget.height.
