=============================
More details of ØMQ and pyzmq
=============================

ØMQ is a fast, lightweight messaging library, and pyzmq is the Python bindings.

ØMQ in 100 words (From the excellent `Guide <zguide>`_):

.. _zguide: http://zguide.zeromq.org/page:all#-MQ-in-a-Hundred-Words

ØMQ (ZeroMQ, 0MQ, zmq) looks like an embeddable networking library but acts like a
concurrency framework. It gives you sockets that carry whole messages across various
transports like in-process, inter-process, TCP, and multicast. You can connect sockets
N-to-N with patterns like fanout, pub-sub, task distribution, and request-reply. It's fast
enough to be the fabric for clustered products. Its asynchronous I/O model gives you
scalable multicore applications, built as asynchronous message-processing tasks. It has a
score of language APIs and runs on most operating systems. ØMQ is from iMatix and is LGPL
open source.

ØMQ works with a variety of socket types, each of which specifies a different send/recv
pattern:

REQ : send/recv/send (counterpart: REP)

REP : recv/send/recv (counterpart: REQ)

XREQ: LRU load-balanced requests (counterpart: XREP)

XREP: IDENTITY-routed replies (counterpart: XREQ)

PUB : send-only, to all peers (counterpart: SUB)

SUB : recv-only, from all peers (counterpart: PUB)

PUSH: send-only, load-balanced across peers (counterpart: PULL)

PULL: recv-only, fair-queued among peers (counterpart: PUSH)

PAIR: symmetric exclusive-pair

XREQ is also called a DEALER, and XREP is called ROUTER, which are new, more descriptive
names.


.. figure:: figs/allconnections.png

    IPython's connection diagram

IPython uses MonitoredQueue devices, as schedulers that relay messages between engines and
clients, but *also* send every message to the Hub, which uses the information to track the
state of the cluster.

Creating some Sockets
=====================

With ØMQ, you always create a context first. Typically, you have just one Context per
process, and can use the classmethod :meth:`zmq.Context.instance` to create it.  Then
you use the Context's :meth:`socket` method to create sockets of various types.

.. sourcecode:: python

    import zmq
    
    ctx = zmq.Context.instance()
    
    a = ctx.socket(zmq.PAIR)
    b = ctx.socket(zmq.PIAR)
    
    p = a.bind_to_random_port('tcp://127.0.0.1')
    b.connect('tcp://127.0.0.1:%i'%p)
    
    a.send('hi')
    print b.recv()
    # 'hi'


non-copying sends
=================

pyzmq supports non-copying sends of objects that provide the buffer interface, which includes
numpy arrays.

.. sourcecode:: python

    A = numpy.ones(16)
    a.send(A, copy=False)
    

And on the other side, when you do a non-copying recv, you get a Message object, which also
provides the buffer interface:

.. sourcecode:: python

    msg = b.recv(copy=False)
    B = numpy.frombuffer(msg)
    
.. note::

    noncopying in code directory

Exercise
========

Can you write a pair of function for sending and receiving numpy arrays without copying?

`noncopying solution <_static/soln/serialsocket.py>`_

Using pyzmq for inter-engine communication
==========================================

Can we design some inter-engine communication with pyzmq?

What sorts of send/recv patterns might be useful?

How do we determine who connects to whom?