Wednesday, April 19, 2017

Clean Ubuntu 16.04 Tensorflow 1.0 CUDA 8.0 CUDNN 5 Python 3.5 Ubuntu 16.04 installation instructions-finally

Ignore this post if you can use use NVidia Docker...


 ##############################################

 This is under test by a few people.

When Tensorflow changes its Nvidia driver, cuDNN, and cuda version requirements this post will have to be updated
Install Tensorflow 1.3
Ubuntu/Linux 64-bit, GPU enabled, Python 3.5
Requires CUDA toolkit 8.0 and CuDNN v5. 

https://www.tensorflow.org/install/install_linux#InstallingVirtualenv

Uses:
Ubuntu 16.04 (Nvidia cuda 8.0 supports 14.04 and 16.04)
Python 3.5
Tensorflow 1.3 (requires cuda 8.0 and cudnn 6.0)
Nvidia driver version 387
Cuda 8.0
cuDNN 6.0
Keras 2.0

 Tensorflow has Nvidia driver and lib deps

 https://www.tensorflow.org/install/install_linux

NVIDIA requirements to run TensorFlow with GPU support

If you are installing TensorFlow with GPU support using one of the mechanisms described in this guide, then the following NVIDIA software must be installed on your system:

  • CUDA® Toolkit 8.0. For details, see NVIDIA's documentation. Ensure that you append the relevant Cuda pathnames to the LD_LIBRARY_PATH environment variable as described in the NVIDIA documentation.
  • The NVIDIA drivers associated with CUDA Toolkit 8.0.
  • cuDNN v6. For details, see NVIDIA's documentation. Ensure that you create the CUDA_HOME environment variable as described in the NVIDIA documentation.
  • GPU card with CUDA Compute Capability 3.0 or higher. See NVIDIA documentation for a list of supported GPU cards.
  • The libcupti-dev library, which is the NVIDIA CUDA Profile Tools Interface. This library provides advanced profiling support. To install this library, issue the following command:
    

    

    

    
$ sudo apt-get install libcupti-dev
# 1. Setup the virtual env
sudo apt-get -y install python-pip 
sudo apt-get -y install virtualenv
sudo apt-get -y install virtualenvwrapper
pip install virtualenv
pip install virtualenvwrapper

find / -name virtualenvwrapper.sh 2> /dev/null

# /usr/share/virtualenvwrapper/virtualenvwrapper.sh
# ls /usr/share/virtualenvwrapper/virtualenvwrapper.sh
# ls /home/depappas/.local/bin/virtualenvwrapper.sh


# 2. Install OpenCVsudo apt-get install  libopencv-dev
sudo apt-get -y install python-opencv

 #3. Set  up the environment

 # Keras/Tensorflow setup in ~/.bashrc

 # Tensorflow setup
export CUDA_ROOT=/usr/local/cuda
export LD_LIBRARY_PATH=$CUDA_ROOT/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$CUDA_ROOT/lib64:$CUDA_ROOT/extras/CUPTI/lib64"
export CUDNN_ROOT=/usr/local/cuda
export CPATH=$CUDNN_ROOT/include:$CPATH
export KERAS_BACKEND=tensorflow
export CUDA_ROOT=/usr/local/cuda                                                                      
# Python VM
# OSX source /usr/local/bin/virtualenvwrapper.sh
source /home/depappas/.local/bin/virtualenvwrapper.sh # Ubuntu

export WORKON_HOME=~/python_virtual_env

 # 4.
source ~/.bashrc
cd ~
mkdir python_virtual_env
cd python_virtual_env
virtualenv --system-site-packages -p python3.5   deep_learning_3.5
workon deep_learning_3.5
# now you are using the pip in the .virtualenvs/test tree

# if pip install does not work upgrade pip

easy_install --upgrade pip

# 5. Install net-tools and configure ssh

sudo apt -y install net-tools

http://ubuntuhandbook.org/index.php/2016/04/enable-ssh-ubuntu-16-04-lts/
sudo apt-get -y install openssh-server
sudo service ssh status
sudo emacs /etc/ssh/sshd_config
# Uncomment  PermitRootLogin prohibit-password
sudo service ssh restart

# set up passwordless login
http://programmingmatrix.blogspot.com/2017/10/ssh-keyless-login-on-ubuntu.html

# 6. Check NVIDIA Compute Capability of your GPU card
# 
# https://developer.nvidia.com/cuda-gpus

# if the machine locks up or you have an older driver installed
sudo apt-get purge nvidia* # or what ever old driver you are using

To add the Proprietary GPU Drivers PPA in Ubuntu or Linux Mint and update the software sources, use the following commands:
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt update
sudo apt-get install nvidia-387


# alternatively

2. Install (and activate) the latest Nvidia graphics drivers
From System Settings or directly from the menu / Dash, open Software & Updates, click on the "Additional Drivers" tab, select the driver (I selected 387 which works on Ubuntu 16.04 and a 1060 GPU) you want to use, and click "Apply changes":



# 7. Install the Nvidia 384 driver, CUDA 8.0, and CUDNN 5.1 (may change this to 6.0)



http://programmingmatrix.blogspot.com/2017/10/ubuntu-1604-nvidia-gpu-driver-cuda-and.html

# 8. Install the libcupti-dev library, which is the NVIDIA CUDA Profile Tools Interface.
# This library provides advanced profiling support.
# To install this library, issue the following command:
sudo apt-get -y install libcupti-dev

# 9. Check that the GPU is detected
# https://github.com/tensorflow/tensorflow/issues/394
# 
sudo reboot

 nvidia-smi

Found 1 NVIDIA devices
Device ID: 0
Device name: GeForce GTX TITAN X (*PrimaryCard)
GPU internal ID: 0420115018258

Tue Dec 15 23:56:17 2015
+------------------------------------------------------+
| NVIDIA-SMI 352.63 Driver Version: 352.63 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX TIT... Off | 0000:04:00.0 On | N/A |
| 22% 33C P8 17W / 250W | 441MiB / 12287MiB | 1% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 1270 G /usr/bin/X 174MiB |
| 0 2183 G compiz 112MiB |
| 0 2575 G ...ves-passed-by-fd --v8-snapshot-passed-by- 127MiB |


# 9. Misc. install
sudo apt-get -y install pandoc
sudo apt-get -y install graphviz
sudo apt-get -y install pandoc

# 10. Install Python packages
pip3 install keras 
pip3 install h5py
pip3 install numpy
pip3 install matplotlib 
pip3 install gensim 
pip3 install ioutils 
pip3 install Cython
# https://github.com/jazzsaxmafia/video_to_sequence/issues/3
pip3 install opencv-python
pip3 install keras
pip3 install sklearn
pip3 install pypandoc
pip3 install pandoc
pip3 install keras_diagram
pip3 install tensorflow-gpu
pip3 install h5py
pip3 install seaborn
pip3 install python-flake8
pip3 install pandas
pip3 install pydot
pip3 install pydot-ng

 # 11. Install Tensorflow

# don't do this!
# pip3 install tensorflow
# pip3 install --upgrade tensorflow-gpu

https://stackoverflow.com/questions/39817645/cuda-cudnn-installed-but-tensorflow-cant-use-the-gpu


https://www.tensorflow.org/install/install_linux#InstallingVirtualenv

################################################################################

Don't do the following, this is just an example:
  1. (Optional) If Step 4 failed (typically because you invoked a pip version lower than 8.1), install TensorFlow in the active virtualenv environment by issuing a command of the following format:
    

    

    

    
(tensorflow)$ pip install --upgrade tfBinaryURL   # Python 2.7
 (tensorflow)$ pip3 install --upgrade tfBinaryURL  # Python 3.n
    where tfBinaryURL identifies the URL of the TensorFlow Python package. The appropriate value of tfBinaryURLdepends on the operating system, Python version, and GPU support. Find the appropriate value for tfBinaryURL for your system here. For example, if you are installing TensorFlow for Linux, Python 3.4, and CPU-only support, issue the following command to install TensorFlow in the active virtualenv environment:
    

    

    

    
(tensorflow)$ pip3 install --upgrade \
 https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.3.0-cp34-cp34m-linux_x86_64.whl

# select a url to install based on your version requirements
https://www.tensorflow.org/install/install_linux#the_url_of_the_tensorflow_python_package

################################################################################

Do this:

# Python 3.5 GPU support

export tfBinaryURL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.3.0-cp35-cp35m-linux_x86_64.whl

pip3 install --upgrade $tfBinaryURL

# 12. test that tensorflow links with libcudnn

 workon deep learning_3.5
python
>>> import tensorflow

 # 13. Check the Tensorflow version

 python -c "import tensorflow; print(tensorflow.__version__)"

 # 14. Test that Tensorflow is using the GPU

 python -c "import tensorflow as tf; sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))"

 ##### Expected Output #########

 I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.8.0 locally
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:910] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with properties:
name: GeForce GTX 1060 6GB
major: 6 minor: 1 memoryClockRate (GHz) 1.7845
pciBusID 0000:01:00.0
Total memory: 5.93GiB
Free memory: 5.66GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0:   Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1060 6GB, pci bus id: 0000:01:00.0)
Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GTX 1060 6GB, pci bus id: 0000:01:00.0
I tensorflow/core/common_runtime/direct_session.cc:257] Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GTX 1060 6GB, pci bus id: 0000:01:00.0

 # http://stackoverflow.com/questions/43335531/how-to-use-sse4-1-instructions-without-install-tensorflow-from-source

 # 15. Setup Keras
For best performance, set `image_data_format="channels_last"` in your Keras config at ~/.keras/keras.json.

# Or use env vars

export KERAS_BACKEND=tensorflow

# 16. test Keras
git clone http://github.com/rcmalli/keras-squeezenet.git
cd keras-squeezenet
python test.py

Using TensorFlow backend.
Downloading data from https://github.com/rcmalli/keras-squeezenet/releases/download/v1.0/squeezenet_weights_tf_dim_ordering_tf_kernels.h5
4530176/5059384 [=========================>....] - ETA: 0s2017-10-18 20:57:06.845299: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-10-18 20:57:06.845317: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-10-18 20:57:06.845321: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-10-18 20:57:06.845324: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-10-18 20:57:06.845326: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-10-18 20:57:06.952926: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2017-10-18 20:57:06.953146: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties: 
name: GeForce GTX 1060 6GB
major: 6 minor: 1 memoryClockRate (GHz) 1.7845
pciBusID 0000:01:00.0
Total memory: 5.93GiB
Free memory: 5.79GiB
2017-10-18 20:57:06.953159: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0 
2017-10-18 20:57:06.953162: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0:   Y 
2017-10-18 20:57:06.953167: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1060 6GB, pci bus id: 0000:01:00.0)

(deep_learning_3.5) depappas@berlin:~/keras-squeezenet$ 


 # 17. Install TFLearn
# http://tflearn.org/installation/
# TFLearn Installation
# To install TFLearn, the easiest way is to run one of the following options.
# For the bleeding edge version:

pip install git+https://github.com/tflearn/tflearn.git

# For the latest stable version:

pip install tflearn

#You can also install from source by running this command (from source folder):


python setup.py install



Posted by at

Tuesday, April 18, 2017

cryptographic-hash-verification: md5sum sha256sum

Posted by at

Wednesday, February 22, 2017

Drawing deep learning networks

Posted by at

Deep learning: convolutional nets: filter param sizing

In this page the filter param calculations are not explained:

http://neuralnetworksanddeeplearning.com/chap6.html#final_conv

This page explains the filter param calculations:

http://datascience.stackexchange.com/questions/8794/understanding-convolutional-pooling-sizes-deep-learning

up vote0down votefavorite
1
I'm dumb but still trying to understand the code provided from this e-book on deep learning, but it doesn't explain where the n_in=40*4*4 comes from. 40 is from the 40 previous feature maps, but what about the 4*4?
enter image description here
>>> net = Network([
        ConvPoolLayer(image_shape=(mini_batch_size, 1, 28, 28), 
                      filter_shape=(20, 1, 5, 5), 
                      poolsize=(2, 2), 
                      activation_fn=ReLU),
        ConvPoolLayer(image_shape=(mini_batch_size, 20, 12, 12), 
                      filter_shape=(40, 20, 5, 5), 
                      poolsize=(2, 2), 
                      activation_fn=ReLU),
        FullyConnectedLayer(
            n_in=40*4*4, n_out=1000, activation_fn=ReLU, p_dropout=0.5),
        FullyConnectedLayer(
            n_in=1000, n_out=1000, activation_fn=ReLU, p_dropout=0.5),
        SoftmaxLayer(n_in=1000, n_out=10, p_dropout=0.5)], 
        mini_batch_size)
>>> net.SGD(expanded_training_data, 40, mini_batch_size, 0.03, 
            validation_data, test_data)
For instance, what if I do a similar analysis in 1D as shown below, which should that n_in term be?
>>> net = Network([
        ConvPoolLayer(image_shape=(mini_batch_size, 1, 81, 1), 
                      filter_shape=(20, 1, 5, 1), 
                      poolsize=(2, 1), 
                      activation_fn=ReLU),
        ConvPoolLayer(image_shape=(mini_batch_size, 20, 12, 1), 
                      filter_shape=(40, 20, 5, 1), 
                      poolsize=(2, 1), 
                      activation_fn=ReLU),
        FullyConnectedLayer(
            n_in=40*???, n_out=1000, activation_fn=ReLU, p_dropout=0.5),
        FullyConnectedLayer(
            n_in=1000, n_out=1000, activation_fn=ReLU, p_dropout=0.5),
        SoftmaxLayer(n_in=1000, n_out=10, p_dropout=0.5)], 
        mini_batch_size)
>>> net.SGD(expanded_training_data, 40, mini_batch_size, 0.03, 
            validation_data, test_data)
Thanks!
 
shareimprove this question
up vote0down voteaccepted
In the given example from the e-book, the number 4 comes from (12-5+1)/2, where 12 is the input image size (12*12) of the second constitutional layer; 5 is the filter size (5*5) used in that layer; and 2 is the poolsize.
This is similar to how you get the number 12 from the first constitutional layer: 12=(28-5+1)/2. It's well explained in your linked chapter.
Regarding your "For instance" code, your 6th line is not correct:
ConvPoolLayer(image_shape=(mini_batch_size, 20, 12, 1), 
The number 12 should be replaced by (81-5+1)/2 which unfortunately is not an integer. You may want to change the filter_shape in the first layer to (6,1) to make it work. In that case, your 6th line should be: 
ConvPoolLayer(image_shape=(mini_batch_size, 20, 38, 1), 
and your 11th line should be: 
n_in=40*17*1, n_out=1000, activation_fn=ReLU, p_dropout=0.5),



Posted by at

Sunday, February 19, 2017

Bayesian Statistics

Posted by at

Tuesday, February 14, 2017

Python Jupyter Tutorials

Posted by at

Mac OSX Anaconda installation

Install Anaconda
https://www.continuum.io/downloads

See the installation instructions on this page
https://github.com/Hvass-Labs/TensorFlow-Tutorials

mkdir conda
cd conda
/anaconda/bin/conda create --name ./tf python=3

cat .bash_profile
...
# added by Anaconda3 4.3.0 installer
export PATH=/anaconda/bin:$PATH

source .bash_profile

ls .condarc
# if this fails then make a .condrc file
/anaconda/bin/conda config --add channels r
ls .condarc

source activate tf

conda install scikit-learn


Posted by at

Tuesday, January 24, 2017

Atom Editor: setup

http://www.marinamele.com/install-and-configure-atom-editor-for-python

Use a Python virtualenv
 apm install linter
 pip install flake8
 pip install flake8-docstrings
 pip install jupyter

 apm install linter-flake8
 apm install hydrogen

start atom after using workon to select the virutalenv

https://atom.io/packages/hydrogen

http://marcio.io/2015/07/supercharging-atom-editor-for-go-development/

apm install go-plus
apm install go-rename
apm install file-icons
apm install linter-golinter 

go get -u github.com/jstemmer/gotags 
go get -u golang.org/x/tools/cmd/...
go get -u github.com/golang/lint/golint 
 
https://rominirani.com/setup-go-development-environment-with-atom-editor-a87a12366fcf#.94k3uye2t 
 
apm install terminal-plus

http://stackoverflow.com/questions/38012317/why-isnt-autocomplete-of-local-packages-working-in-atom-editor
The issue is addressed in the README.md of the go-plus package:
First of all, make sure autocomplete-plus is present on your setup. Go-plus provides autocompletion through gocode tool, so you should ensure it's in PATH and available.
If you can't get autocompletion for the user-defined packages working, while it's there for packages from standard library, it's likely a trivial gocode-related issue. Try running gocode set. Some expected output'd be: propose-builtins false lib-path "" autobuild false force-debug-output "" package-lookup-mode "go"
What you gotta do is switching autobuild to true, by running gocode set autobuild true. Check autocompletion now, it must be working right.
Posted by at

Wednesday, January 11, 2017

New Ubuntu Setup for Deep Learning, Scala, and SPARK

# Install Ubuntu 16.04
sudo apt-get update
sudo apt-get upgrade

 # See this post for a clean Tensorflow 1.0 CUDA 8.0 CUDNN 5 Python 3.5 Ubuntu 16.04 installation
https://programmingmatrix.blogspot.ca/2017/04/clean-tensorflow-10-installation.html







~/.bash_profile : export JAVA_HOME="`/usr/libexec/java_home -v '1.8*'`"

Scala


~/.bash_profile:
export SCALA_HOME=$HOME/scala-2.12.1
export SCALA_BIN=$SCALA_HOME/bin
export PATH=$PATH:$SCALA_BIN

Spark


cd
mv Downloads/spark-2.0.2-bin-hadoop2.7.tar .
tar xvf spark-2.0.2-bin-hadoop2.7.tar 
~/.bash_profile:
export SPARK_HOME=$HOME/spark
export SPARK_BIN=$SPARK_HOME/bin
export PATH=$PATH:$SPARK_BIN
export KERAS_BACKEND=tensorflow

ln -s spark-2.0.2-bin-hadoop2.7 spark

Golang

Atom Editor 


Posted by at

Wednesday, January 4, 2017

Go,C equivalents

Posted by at

Tuesday, January 3, 2017

Go lang: adding a GitHub import

In your go code

import "github.com/valyala/fasthttp"

#Add to your ~/.bash_profile file for OS X and ~/.bashrc for Linux
export GOPATH=$HOME/golib
export PATH="$PATH:$GOPATH"

# OSX
source~/.bash_profile

# Linux
source~/.bashrc

go get -u github.com/valyala/fasthttp



Posted by at
Subscribe to: Posts (Atom)