436 lines
12 KiB
Plaintext
436 lines
12 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# GPU-Jupyter\n",
|
|
"\n",
|
|
"This Jupyterlab Instance is connected to the GPU via CUDA drivers. In this notebook, we test the installation and perform some basic operations on the GPU."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Test GPU connection\n",
|
|
"\n",
|
|
"#### Using the following command, your GPU type and its NVIDIA-SMI driver version should be listed:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Mon Jun 22 11:24:08 2020 \n",
|
|
"+-----------------------------------------------------------------------------+\n",
|
|
"| NVIDIA-SMI 440.82 Driver Version: 440.82 CUDA Version: 10.2 |\n",
|
|
"|-------------------------------+----------------------+----------------------+\n",
|
|
"| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
|
|
"| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n",
|
|
"|===============================+======================+======================|\n",
|
|
"| 0 GeForce RTX 207... Off | 00000000:01:00.0 On | N/A |\n",
|
|
"| 0% 49C P0 38W / 215W | 430MiB / 7974MiB | 5% Default |\n",
|
|
"+-------------------------------+----------------------+----------------------+\n",
|
|
" \n",
|
|
"+-----------------------------------------------------------------------------+\n",
|
|
"| Processes: GPU Memory |\n",
|
|
"| GPU PID Type Process name Usage |\n",
|
|
"|=============================================================================|\n",
|
|
"+-----------------------------------------------------------------------------+\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"!nvidia-smi"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### Now, test if PyTorch can access the GPU via CUDA:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"True"
|
|
]
|
|
},
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"import torch\n",
|
|
"torch.cuda.is_available()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[PhysicalDevice(name='/physical_device:XLA_GPU:0', device_type='XLA_GPU')]\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"[name: \"/device:CPU:0\"\n",
|
|
" device_type: \"CPU\"\n",
|
|
" memory_limit: 268435456\n",
|
|
" locality {\n",
|
|
" }\n",
|
|
" incarnation: 12436949185972503812,\n",
|
|
" name: \"/device:XLA_CPU:0\"\n",
|
|
" device_type: \"XLA_CPU\"\n",
|
|
" memory_limit: 17179869184\n",
|
|
" locality {\n",
|
|
" }\n",
|
|
" incarnation: 9674938692146126962\n",
|
|
" physical_device_desc: \"device: XLA_CPU device\",\n",
|
|
" name: \"/device:XLA_GPU:0\"\n",
|
|
" device_type: \"XLA_GPU\"\n",
|
|
" memory_limit: 17179869184\n",
|
|
" locality {\n",
|
|
" }\n",
|
|
" incarnation: 7870544216044264725\n",
|
|
" physical_device_desc: \"device: XLA_GPU device\"]"
|
|
]
|
|
},
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"import tensorflow as tf\n",
|
|
"from tensorflow.python.client import device_lib\n",
|
|
"print(tf.config.list_physical_devices('XLA_GPU'))\n",
|
|
"device_lib.list_local_devices()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"tensor([[0.0399, 0.1738, 0.2486],\n",
|
|
" [0.7464, 0.1461, 0.8991],\n",
|
|
" [0.7264, 0.9835, 0.8844],\n",
|
|
" [0.4544, 0.8331, 0.8435],\n",
|
|
" [0.0109, 0.0689, 0.2997]])"
|
|
]
|
|
},
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"from __future__ import print_function\n",
|
|
"import numpy as np\n",
|
|
"import torch\n",
|
|
"a = torch.rand(5, 3)\n",
|
|
"a"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Performance test\n",
|
|
"\n",
|
|
"#### Now we want to know how much faster a typical operation is using GPU. Therefore we do the same operation in numpy, PyTorch and PyTorch with CUDA. The test operation is the calculation of the prediction matrix that is done in a linear regression."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 1) Numpy"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"x = np.random.rand(10000, 256)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"276 ms ± 9.97 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"%%timeit\n",
|
|
"H = x.dot(np.linalg.inv(x.transpose().dot(x))).dot(x.transpose())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 2) PyTorch"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"x = torch.rand(10000, 256)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"82.1 ms ± 1.85 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"%%timeit\n",
|
|
"# Calculate the projection matrix of x on the CPU\n",
|
|
"H = x.mm( (x.t().mm(x)).inverse() ).mm(x.t())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 3) PyTorch on GPU via CUDA"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"tensor([[0.2854, 0.3384, 0.6473, 0.0433, 0.5640],\n",
|
|
" [0.3960, 0.0449, 0.6597, 0.5347, 0.8402],\n",
|
|
" [0.0048, 0.9231, 0.0311, 0.2545, 0.0409],\n",
|
|
" [0.6506, 0.8651, 0.7558, 0.1086, 0.8135],\n",
|
|
" [0.1083, 0.0039, 0.6049, 0.3596, 0.1359]], device='cuda:0')\n",
|
|
"tensor([[0.2854, 0.3384, 0.6473, 0.0433, 0.5640],\n",
|
|
" [0.3960, 0.0449, 0.6597, 0.5347, 0.8402],\n",
|
|
" [0.0048, 0.9231, 0.0311, 0.2545, 0.0409],\n",
|
|
" [0.6506, 0.8651, 0.7558, 0.1086, 0.8135],\n",
|
|
" [0.1083, 0.0039, 0.6049, 0.3596, 0.1359]], dtype=torch.float64)\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# let us run this cell only if CUDA is available\n",
|
|
"# We will use ``torch.device`` objects to move tensors in and out of GPU\n",
|
|
"if torch.cuda.is_available():\n",
|
|
" device = torch.device(\"cuda\") # a CUDA device object\n",
|
|
" x = torch.rand(10000, 256, device=device) # directly create a tensor on GPU\n",
|
|
" y = x.to(device) # or just use strings ``.to(\"cuda\")``\n",
|
|
" print(x[0:5, 0:5])\n",
|
|
" print(y.to(\"cpu\", torch.double)[0:5, 0:5])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"11.4 ms ± 28.8 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"%%timeit\n",
|
|
"# Calculate the projection matrix of x on the GPU\n",
|
|
"H = x.mm( (x.t().mm(x)).inverse() ).mm(x.t())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Exhaustive Testing on GPU"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# let us run this cell only if CUDA is available\n",
|
|
"# We will use ``torch.device`` objects to move tensors in and out of GPU\n",
|
|
"import torch\n",
|
|
"if torch.cuda.is_available():\n",
|
|
" device = torch.device(\"cuda\") # a CUDA device object\n",
|
|
" x = torch.rand(10000, 10, device=device) # directly create a tensor on GPU"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"tensor([[0.1101, 0.7887, 0.0641, 0.1327, 0.1681],\n",
|
|
" [0.7914, 0.7248, 0.7731, 0.2662, 0.4908],\n",
|
|
" [0.2451, 0.3568, 0.4006, 0.2099, 0.5212],\n",
|
|
" [0.6195, 0.5120, 0.5212, 0.7321, 0.2272],\n",
|
|
" [0.2374, 0.4540, 0.0868, 0.9393, 0.1561]], device='cuda:0')\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"if torch.cuda.is_available():\n",
|
|
" y = x.to(device) # or just use strings ``.to(\"cuda\")``\n",
|
|
" print(x[0:5, 0:5])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"if torch.cuda.is_available():\n",
|
|
" # Here is the memory of the GPU a border. \n",
|
|
" # A matrix with 100000 lines requires 37 GB, but only 8 GB are available.\n",
|
|
" H = x.mm( (x.t().mm(x)).inverse() ).mm(x.t())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"tensor([[ 6.4681e-04, -1.5392e-05, 3.3608e-04, 2.1025e-04, 8.0912e-05],\n",
|
|
" [-1.5392e-05, 5.0718e-04, -1.1769e-04, -2.3084e-05, -2.3264e-04],\n",
|
|
" [ 3.3608e-04, -1.1769e-04, 6.9678e-04, 2.2663e-04, -1.8900e-04],\n",
|
|
" [ 2.1025e-04, -2.3084e-05, 2.2663e-04, 6.0036e-04, 2.7787e-04],\n",
|
|
" [ 8.0912e-05, -2.3264e-04, -1.8900e-04, 2.7787e-04, 1.4208e-03]],\n",
|
|
" device='cuda:0')\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"if torch.cuda.is_available():\n",
|
|
" print(H[0:5, 0:5])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 15,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"tensor([[ 6.4681e-04, -1.5392e-05, 3.3608e-04, 2.1025e-04, 8.0912e-05],\n",
|
|
" [-1.5392e-05, 5.0718e-04, -1.1769e-04, -2.3084e-05, -2.3264e-04],\n",
|
|
" [ 3.3608e-04, -1.1769e-04, 6.9678e-04, 2.2663e-04, -1.8900e-04],\n",
|
|
" [ 2.1025e-04, -2.3084e-05, 2.2663e-04, 6.0036e-04, 2.7787e-04],\n",
|
|
" [ 8.0912e-05, -2.3264e-04, -1.8900e-04, 2.7787e-04, 1.4208e-03]],\n",
|
|
" dtype=torch.float64)\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"if torch.cuda.is_available():\n",
|
|
" # This operation is difficult, as an symmetric matrix is transferred \n",
|
|
" # back to the CPU. Is possible up to 30000 rows.\n",
|
|
" print(H.to(\"cpu\", torch.double)[0:5, 0:5])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.7.6"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
}
|