gpu-jupyter/extra/Getting_Started/GPU-processing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# GPU-Jupyter\n",
    "\n",
    "This Jupyterlab Instance is connected to the GPU via CUDA drivers. In this notebook, we test the installation and perform some basic operations on the GPU."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test GPU connection\n",
    "\n",
    "#### Using the following command, your GPU type and its NVIDIA-SMI driver version should be listed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tue Mar 10 17:55:25 2020       \n",
      "+-----------------------------------------------------------------------------+\n",
      "| NVIDIA-SMI 440.48.02    Driver Version: 440.48.02    CUDA Version: 10.2     |\n",
      "|-------------------------------+----------------------+----------------------+\n",
      "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
      "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
      "|===============================+======================+======================|\n",
      "|   0  GeForce RTX 207...  Off  | 00000000:01:00.0 Off |                  N/A |\n",
      "|  0%   41C    P8     1W / 215W |    215MiB /  7974MiB |      0%      Default |\n",
      "+-------------------------------+----------------------+----------------------+\n",
      "                                                                               \n",
      "+-----------------------------------------------------------------------------+\n",
      "| Processes:                                                       GPU Memory |\n",
      "|  GPU       PID   Type   Process name                             Usage      |\n",
      "|=============================================================================|\n",
      "+-----------------------------------------------------------------------------+\n"
     ]
    }
   ],
   "source": [
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Now, test if PyTorch can access the GPU via CUDA:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "torch.cuda.is_available()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[name: \"/device:CPU:0\"\n",
       " device_type: \"CPU\"\n",
       " memory_limit: 268435456\n",
       " locality {\n",
       " }\n",
       " incarnation: 933763008911863935,\n",
       " name: \"/device:XLA_CPU:0\"\n",
       " device_type: \"XLA_CPU\"\n",
       " memory_limit: 17179869184\n",
       " locality {\n",
       " }\n",
       " incarnation: 12790964875098705008\n",
       " physical_device_desc: \"device: XLA_CPU device\",\n",
       " name: \"/device:GPU:0\"\n",
       " device_type: \"GPU\"\n",
       " memory_limit: 6940531098\n",
       " locality {\n",
       "   bus_id: 1\n",
       "   links {\n",
       "   }\n",
       " }\n",
       " incarnation: 4940791198162309705\n",
       " physical_device_desc: \"device: 0, name: GeForce RTX 2070 SUPER, pci bus id: 0000:01:00.0, compute capability: 7.5\",\n",
       " name: \"/device:XLA_GPU:0\"\n",
       " device_type: \"XLA_GPU\"\n",
       " memory_limit: 17179869184\n",
       " locality {\n",
       " }\n",
       " incarnation: 6996862811697216940\n",
       " physical_device_desc: \"device: XLA_GPU device\"]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow.python.client import device_lib\n",
    "print(tf.test.is_gpu_available(cuda_only=True))\n",
    "device_lib.list_local_devices()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.8519, 0.7682, 0.3258],\n",
       "        [0.1957, 0.4073, 0.6085],\n",
       "        [0.9164, 0.8401, 0.4548],\n",
       "        [0.9011, 0.8838, 0.9559],\n",
       "        [0.4692, 0.3993, 0.4313]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from __future__ import print_function\n",
    "import numpy as np\n",
    "import torch\n",
    "a = torch.rand(5, 3)\n",
    "a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Performance test\n",
    "\n",
    "#### Now we want to know how much faster a typical operation is using GPU. Therefore we do the same operation in numpy, PyTorch and PyTorch with CUDA. The test operation is the calculation of the prediction matrix that is done in a linear regression."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1) Numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.random.rand(10000, 256)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "362 ms ± 86.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "H = x.dot(np.linalg.inv(x.transpose().dot(x))).dot(x.transpose())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2) PyTorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.rand(10000, 256)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "135 ms ± 3.48 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "# Calculate the projection matrix of x\n",
    "H = x.mm( (x.t().mm(x)).inverse() ).mm(x.t())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3) PyTorch on GPU via CUDA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0.2812, 0.3255, 0.5715, 0.1665, 0.6951],\n",
      "        [0.5562, 0.9592, 0.0911, 0.9672, 0.3311],\n",
      "        [0.6711, 0.0422, 0.5091, 0.6653, 0.9234],\n",
      "        [0.1029, 0.1447, 0.8385, 0.7580, 0.7998],\n",
      "        [0.7787, 0.0114, 0.4865, 0.4171, 0.7066]], device='cuda:0')\n",
      "tensor([[0.2812, 0.3255, 0.5715, 0.1665, 0.6951],\n",
      "        [0.5562, 0.9592, 0.0911, 0.9672, 0.3311],\n",
      "        [0.6711, 0.0422, 0.5091, 0.6653, 0.9234],\n",
      "        [0.1029, 0.1447, 0.8385, 0.7580, 0.7998],\n",
      "        [0.7787, 0.0114, 0.4865, 0.4171, 0.7066]], dtype=torch.float64)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.7/site-packages/torch/cuda/__init__.py:134: UserWarning: \n",
      "    Found GPU0 GeForce RTX 2070 SUPER which requires CUDA_VERSION >= 10000 to\n",
      "     work properly, but your PyTorch was compiled\n",
      "     with CUDA_VERSION 9000. Please install the correct PyTorch binary\n",
      "     using instructions from https://pytorch.org\n",
      "    \n",
      "  warnings.warn(incorrect_binary_warn % (d, name, 10000, CUDA_VERSION))\n"
     ]
    }
   ],
   "source": [
    "# let us run this cell only if CUDA is available\n",
    "# We will use ``torch.device`` objects to move tensors in and out of GPU\n",
    "if torch.cuda.is_available():\n",
    "    device = torch.device(\"cuda\")          # a CUDA device object\n",
    "    x = torch.rand(10000, 256, device=device) # directly create a tensor on GPU\n",
    "    y = x.to(device)                       # or just use strings ``.to(\"cuda\")``\n",
    "    print(x[0:5, 0:5])\n",
    "    print(y.to(\"cpu\", torch.double)[0:5, 0:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "12.8 ms ± 564 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "H = x.mm( (x.t().mm(x)).inverse() ).mm(x.t())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exhaustive Testing on GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let us run this cell only if CUDA is available\n",
    "# We will use ``torch.device`` objects to move tensors in and out of GPU\n",
    "import torch\n",
    "if torch.cuda.is_available():\n",
    "    device = torch.device(\"cuda\")          # a CUDA device object\n",
    "    x = torch.rand(10000, 10, device=device) # directly create a tensor on GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0.6760, 0.8890, 0.7271, 0.4208, 0.1131],\n",
      "        [0.4036, 0.8012, 0.3448, 0.4120, 0.2439],\n",
      "        [0.6088, 0.4356, 0.9391, 0.1366, 0.4379],\n",
      "        [0.4540, 0.5981, 0.3885, 0.2473, 0.5938],\n",
      "        [0.2976, 0.8384, 0.6107, 0.6882, 0.9593]], device='cuda:0')\n"
     ]
    }
   ],
   "source": [
    "if torch.cuda.is_available():\n",
    "    y = x.to(device)                       # or just use strings ``.to(\"cuda\")``\n",
    "    print(x[0:5, 0:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "if torch.cuda.is_available():\n",
    "    # Here is the memory of the GPU a border. \n",
    "    # A matrix with 100000 lines requires 37 GB, but only 8 GB are available.\n",
    "    H = x.mm( (x.t().mm(x)).inverse() ).mm(x.t())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 1.1191e-03,  1.6152e-04, -2.1592e-04,  1.4253e-04, -4.0365e-04],\n",
      "        [ 1.6151e-04,  5.5901e-04,  2.6872e-04, -3.1842e-06,  2.8985e-04],\n",
      "        [-2.1592e-04,  2.6872e-04,  1.0728e-03, -3.5968e-05,  5.5613e-04],\n",
      "        [ 1.4253e-04, -3.1840e-06, -3.5968e-05,  6.5156e-04, -3.1820e-04],\n",
      "        [-4.0365e-04,  2.8985e-04,  5.5613e-04, -3.1820e-04,  1.4067e-03]],\n",
      "       device='cuda:0')\n"
     ]
    }
   ],
   "source": [
    "if torch.cuda.is_available():\n",
    "    print(H[0:5, 0:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 1.1191e-03,  1.6152e-04, -2.1592e-04,  1.4253e-04, -4.0365e-04],\n",
      "        [ 1.6151e-04,  5.5901e-04,  2.6872e-04, -3.1842e-06,  2.8985e-04],\n",
      "        [-2.1592e-04,  2.6872e-04,  1.0728e-03, -3.5968e-05,  5.5613e-04],\n",
      "        [ 1.4253e-04, -3.1840e-06, -3.5968e-05,  6.5156e-04, -3.1820e-04],\n",
      "        [-4.0365e-04,  2.8985e-04,  5.5613e-04, -3.1820e-04,  1.4067e-03]],\n",
      "       dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    "if torch.cuda.is_available():\n",
    "    # This operation is difficult, as an symmetric matrix is transferred \n",
    "    # back to the CPU. Is possible up to 30000 rows.\n",
    "    print(H.to(\"cpu\", torch.double)[0:5, 0:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}