{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using third-party Native Libraries\n",
    "\n",
    "Sometimes, the functionality you need is only available in third-party native libraries. These libraries can still be used from within Pythran, using Pythran's support for capsules. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pythran Code\n",
    "\n",
    "The pythran code requires function pointers to the third-party functions, passed as parameters to your pythran routine, as in the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pythran\n",
    "%load_ext pythran.magic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%pythran \n",
    "#pythran export pythran_cbrt(float64(float64), float64)\n",
    "\n",
    "def pythran_cbrt(libm_cbrt, val):\n",
    "    return libm_cbrt(val)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In that case ``libm_cbrt`` is expected to be a capsule containing the function pointer to ``libm``'s ``cbrt`` (cube root) function.\n",
    "\n",
    "This capsule can be created using ``ctypes``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import ctypes\n",
    "\n",
    "# capsulefactory\n",
    "PyCapsule_New = ctypes.pythonapi.PyCapsule_New\n",
    "PyCapsule_New.restype = ctypes.py_object\n",
    "PyCapsule_New.argtypes = ctypes.c_void_p, ctypes.c_char_p, ctypes.c_void_p\n",
    "\n",
    "# load libm\n",
    "libm = ctypes.CDLL('libm.so.6')\n",
    "\n",
    "# extract the proper symbol\n",
    "cbrt = libm.cbrt\n",
    "\n",
    "# wrap it\n",
    "cbrt_capsule = PyCapsule_New(cbrt, \"double(double)\".encode(), None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The capsule is not usable from Python context (it's some kind of opaque box) but Pythran knows how to use it. beware, it does not try to do any kind of type verification. It trusts your ``#pythran export`` line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.0"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pythran_cbrt(cbrt_capsule, 8.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## With Pointers\n",
    "\n",
    "Now, let's try to use the ``sincos`` function. It's C signature is ``void sincos(double, double*, double*)``. How do we pass that to Pythran?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%pythran\n",
    "\n",
    "#pythran export pythran_sincos(None(float64, float64*, float64*), float64)\n",
    "def pythran_sincos(libm_sincos, val):\n",
    "    import numpy as np\n",
    "    val_sin, val_cos = np.empty(1), np.empty(1)\n",
    "    libm_sincos(val, val_sin, val_cos)\n",
    "    return val_sin[0], val_cos[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is some magic happening here:\n",
    "\n",
    "- ``None`` is used to state the function pointer does not return anything.\n",
    "\n",
    "- In order to create pointers, we actually create empty one-dimensional array and let pythran handle them as pointer. Beware that you're in charge of all the memory checking stuff!\n",
    "\n",
    "Apart from that, we can now call our function with the proper capsule parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "sincos_capsule = PyCapsule_New(libm.sincos, \"unchecked anyway\".encode(), None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pythran_sincos(sincos_capsule, 0.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## With Pythran\n",
    "\n",
    "It is naturally also possible to use capsule generated by Pythran. In that case, no type shenanigans is required, we're in our small world.\n",
    "\n",
    "One just need to use the ``capsule`` keyword to indicate we want to generate a capsule."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%pythran\n",
    "\n",
    "## This is the capsule.\n",
    "#pythran export capsule corp((int, str), str set)\n",
    "def corp(param, lookup):\n",
    "    res, key = param\n",
    "    return res if key in lookup else -1\n",
    "\n",
    "## This is some dummy callsite\n",
    "#pythran export brief(int, int((int, str), str set)):\n",
    "def brief(val, capsule):\n",
    "    return capsule((val, \"doctor\"), {\"some\"})\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's not possible to call the capsule directly, it's an opaque structure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'PyCapsule' object is not callable\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    corp((1,\"some\"),set())\n",
    "except TypeError as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's possible to pass it to the according pythran function though."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-1"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "brief(1, corp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## With Cython\n",
    "\n",
    "The capsule pythran uses may come from Cython-generated code. This uses a little-known feature from cython: ``api`` and ``__pyx_capi__``. ``nogil`` is of importance here: Pythran releases the GIL, so **better not call a cythonized function that uses it**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "!find -name 'cube*' -delete"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing cube.pyx\n"
     ]
    }
   ],
   "source": [
    "%%file cube.pyx\n",
    "#cython: language_level=3\n",
    "cdef api double cube(double x) nogil:\n",
    "    return x * x * x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Compiling cube.pyx because it changed.\n",
      "[1/1] Cythonizing cube.pyx\n"
     ]
    }
   ],
   "source": [
    "from setuptools import setup\n",
    "from Cython.Build import cythonize\n",
    "\n",
    "_ = setup(\n",
    "    name='cube',\n",
    "    ext_modules=cythonize(\"cube.pyx\"),\n",
    "    zip_safe=False,\n",
    "    # fake CLI call\n",
    "    script_name='setup.py',\n",
    "    script_args=['--quiet', 'build_ext', '--inplace']\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The cythonized module has a special dictionary that holds the capsule we're looking for."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'PyCapsule'>\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "sys.path.insert(0, '.')\n",
    "import cube\n",
    "print(type(cube.__pyx_capi__['cube']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8.0"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cython_cube = cube.__pyx_capi__['cube']\n",
    "pythran_cbrt(cython_cube, 2.)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
