gpu-jupyter/extra/Getting_Started/GPU-processing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# GPU-Jupyter\n",
    "\n",
    "This Jupyterlab Instance is connected to the GPU via CUDA drivers. In this notebook, we test the installation and perform some basic operations on the GPU."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test GPU connection\n",
    "\n",
    "#### Using the following command, your GPU type and its NVIDIA-SMI driver version should be listed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mon Jun 22 11:24:08 2020       \n",
      "+-----------------------------------------------------------------------------+\n",
      "| NVIDIA-SMI 440.82       Driver Version: 440.82       CUDA Version: 10.2     |\n",
      "|-------------------------------+----------------------+----------------------+\n",
      "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
      "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
      "|===============================+======================+======================|\n",
      "|   0  GeForce RTX 207...  Off  | 00000000:01:00.0  On |                  N/A |\n",
      "|  0%   49C    P0    38W / 215W |    430MiB /  7974MiB |      5%      Default |\n",
      "+-------------------------------+----------------------+----------------------+\n",
      "                                                                               \n",
      "+-----------------------------------------------------------------------------+\n",
      "| Processes:                                                       GPU Memory |\n",
      "|  GPU       PID   Type   Process name                             Usage      |\n",
      "|=============================================================================|\n",
      "+-----------------------------------------------------------------------------+\n"
     ]
    }
   ],
   "source": [
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Now, test if PyTorch can access the GPU via CUDA:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "torch.cuda.is_available()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[PhysicalDevice(name='/physical_device:XLA_GPU:0', device_type='XLA_GPU')]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[name: \"/device:CPU:0\"\n",
       " device_type: \"CPU\"\n",
       " memory_limit: 268435456\n",
       " locality {\n",
       " }\n",
       " incarnation: 12436949185972503812,\n",
       " name: \"/device:XLA_CPU:0\"\n",
       " device_type: \"XLA_CPU\"\n",
       " memory_limit: 17179869184\n",
       " locality {\n",
       " }\n",
       " incarnation: 9674938692146126962\n",
       " physical_device_desc: \"device: XLA_CPU device\",\n",
       " name: \"/device:XLA_GPU:0\"\n",
       " device_type: \"XLA_GPU\"\n",
       " memory_limit: 17179869184\n",
       " locality {\n",
       " }\n",
       " incarnation: 7870544216044264725\n",
       " physical_device_desc: \"device: XLA_GPU device\"]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow.python.client import device_lib\n",
    "print(tf.config.list_physical_devices('XLA_GPU'))\n",
    "device_lib.list_local_devices()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.0399, 0.1738, 0.2486],\n",
       "        [0.7464, 0.1461, 0.8991],\n",
       "        [0.7264, 0.9835, 0.8844],\n",
       "        [0.4544, 0.8331, 0.8435],\n",
       "        [0.0109, 0.0689, 0.2997]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from __future__ import print_function\n",
    "import numpy as np\n",
    "import torch\n",
    "a = torch.rand(5, 3)\n",
    "a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Performance test\n",
    "\n",
    "#### Now we want to know how much faster a typical operation is using GPU. Therefore we do the same operation in numpy, PyTorch and PyTorch with CUDA. The test operation is the calculation of the prediction matrix that is done in a linear regression."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1) Numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.random.rand(10000, 256)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "276 ms ± 9.97 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "H = x.dot(np.linalg.inv(x.transpose().dot(x))).dot(x.transpose())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2) PyTorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.rand(10000, 256)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "82.1 ms ± 1.85 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "# Calculate the projection matrix of x on the CPU\n",
    "H = x.mm( (x.t().mm(x)).inverse() ).mm(x.t())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3) PyTorch on GPU via CUDA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0.2854, 0.3384, 0.6473, 0.0433, 0.5640],\n",
      "        [0.3960, 0.0449, 0.6597, 0.5347, 0.8402],\n",
      "        [0.0048, 0.9231, 0.0311, 0.2545, 0.0409],\n",
      "        [0.6506, 0.8651, 0.7558, 0.1086, 0.8135],\n",
      "        [0.1083, 0.0039, 0.6049, 0.3596, 0.1359]], device='cuda:0')\n",
      "tensor([[0.2854, 0.3384, 0.6473, 0.0433, 0.5640],\n",
      "        [0.3960, 0.0449, 0.6597, 0.5347, 0.8402],\n",
      "        [0.0048, 0.9231, 0.0311, 0.2545, 0.0409],\n",
      "        [0.6506, 0.8651, 0.7558, 0.1086, 0.8135],\n",
      "        [0.1083, 0.0039, 0.6049, 0.3596, 0.1359]], dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    "# let us run this cell only if CUDA is available\n",
    "# We will use ``torch.device`` objects to move tensors in and out of GPU\n",
    "if torch.cuda.is_available():\n",
    "    device = torch.device(\"cuda\")          # a CUDA device object\n",
    "    x = torch.rand(10000, 256, device=device) # directly create a tensor on GPU\n",
    "    y = x.to(device)                       # or just use strings ``.to(\"cuda\")``\n",
    "    print(x[0:5, 0:5])\n",
    "    print(y.to(\"cpu\", torch.double)[0:5, 0:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "11.4 ms ± 28.8 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "# Calculate the projection matrix of x on the GPU\n",
    "H = x.mm( (x.t().mm(x)).inverse() ).mm(x.t())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exhaustive Testing on GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let us run this cell only if CUDA is available\n",
    "# We will use ``torch.device`` objects to move tensors in and out of GPU\n",
    "import torch\n",
    "if torch.cuda.is_available():\n",
    "    device = torch.device(\"cuda\")          # a CUDA device object\n",
    "    x = torch.rand(10000, 10, device=device) # directly create a tensor on GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0.1101, 0.7887, 0.0641, 0.1327, 0.1681],\n",
      "        [0.7914, 0.7248, 0.7731, 0.2662, 0.4908],\n",
      "        [0.2451, 0.3568, 0.4006, 0.2099, 0.5212],\n",
      "        [0.6195, 0.5120, 0.5212, 0.7321, 0.2272],\n",
      "        [0.2374, 0.4540, 0.0868, 0.9393, 0.1561]], device='cuda:0')\n"
     ]
    }
   ],
   "source": [
    "if torch.cuda.is_available():\n",
    "    y = x.to(device)                       # or just use strings ``.to(\"cuda\")``\n",
    "    print(x[0:5, 0:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "if torch.cuda.is_available():\n",
    "    # Here is the memory of the GPU a border. \n",
    "    # A matrix with 100000 lines requires 37 GB, but only 8 GB are available.\n",
    "    H = x.mm( (x.t().mm(x)).inverse() ).mm(x.t())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 6.4681e-04, -1.5392e-05,  3.3608e-04,  2.1025e-04,  8.0912e-05],\n",
      "        [-1.5392e-05,  5.0718e-04, -1.1769e-04, -2.3084e-05, -2.3264e-04],\n",
      "        [ 3.3608e-04, -1.1769e-04,  6.9678e-04,  2.2663e-04, -1.8900e-04],\n",
      "        [ 2.1025e-04, -2.3084e-05,  2.2663e-04,  6.0036e-04,  2.7787e-04],\n",
      "        [ 8.0912e-05, -2.3264e-04, -1.8900e-04,  2.7787e-04,  1.4208e-03]],\n",
      "       device='cuda:0')\n"
     ]
    }
   ],
   "source": [
    "if torch.cuda.is_available():\n",
    "    print(H[0:5, 0:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 6.4681e-04, -1.5392e-05,  3.3608e-04,  2.1025e-04,  8.0912e-05],\n",
      "        [-1.5392e-05,  5.0718e-04, -1.1769e-04, -2.3084e-05, -2.3264e-04],\n",
      "        [ 3.3608e-04, -1.1769e-04,  6.9678e-04,  2.2663e-04, -1.8900e-04],\n",
      "        [ 2.1025e-04, -2.3084e-05,  2.2663e-04,  6.0036e-04,  2.7787e-04],\n",
      "        [ 8.0912e-05, -2.3264e-04, -1.8900e-04,  2.7787e-04,  1.4208e-03]],\n",
      "       dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    "if torch.cuda.is_available():\n",
    "    # This operation is difficult, as an symmetric matrix is transferred \n",
    "    # back to the CPU. Is possible up to 30000 rows.\n",
    "    print(H.to(\"cpu\", torch.double)[0:5, 0:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}