Github and SSH

ssh-keygen -t ed25519 -f github.com

ssh-add ~/.ssh/github.com

# save the keys to the cloud or somewhere where you will not lose them

#Now make an ssh config file

# https://gist.github.com/rbialek/1012262

nano ~/.ssh/config

#Mac 

Host github.com

User git

        Hostname github.com

        PreferredAuthentications publickey

        IdentityFile /home/user/.ssh/github.com

# Ubuntu

Host github.com

User git

        Hostname github.com

        IdentityFile /home/<user>/.ssh/github.com

# https://garywoodfine.com/use-pbcopy-on-ubuntu/

sudo apt-get install xclip -y

nano ~/.bashrc

# insert at the end of the .bashrc file

alias pbcopy='xclip -selection clipboard'

alias pbpaste='xclip -selection clipboard -o'

eval "$(ssh-agent -s)"

# save the file

source ~/.bashrc

# copy the public key to GitHub

# Mac
pbcopy < ~/.ssh/github.com.pub

# https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh
# Adding a new SSH key to your GitHub account
# To configure your GitHub account to use your new (or existing) # SSH key, you'll also need to add it to your GitHub account.

eval "$(ssh-agent -s)"

# Now check the ssh connection

ssh git@github.com

# Setup two factor authentication

https://docs.github.com/en/github/authenticating-to-github/securing-your-account-with-two-factor-authentication-2fa/configuring-two-factor-authentication

Put an encryption key on Lastpass or some other password service

Use the key to encrypt the keys and recovery codes and move them to a Google Drive folder

Install Java and Scala on the Mac M1

https://stackoverflow.com/questions/66464412/install-scala-to-apple-silicon

https://sdkman.io

https://stackoverflow.com/questions/64788005/java-jdk-for-apple-m1-chip

You can install Java JDK using sdkman (see sdkman install):

vim .sdkman/etc/config

set sdkman_rosetta2_compatbile=false

###### Instal packages

sdk install java 8.0.302-zulusdk install 

sbt sbt compile
sdk install spark

https://mungingdata.com/java/sdkman-multiple-versions-java-sbt-scala/

sdk current 

Using:

java: 8.0.302-zulu

sbt: 1.5.5

scala: 2.13.6

spark: 3.1.2

https://springframework.guru/using-sdkman-to-manage-java-versions/

###### Test Spark

~/.sdkman/candidates/spark/3.1.2/bin/run-example SparkPi 10

Code and Syntax Highlighting

From: https://gist.github.com/ww9/44f08d44327a40d2ab309a349bebec57

 

Code and Syntax Highlighting

Code blocks are part of the Markdown spec, but syntax highlighting isn't. However, many renderers -- like Github's and Markdown Here -- support syntax highlighting. Markdown Here supports highlighting for dozens of languages (and not-really-languages, like diffs and HTTP headers); to see the complete list, and how to write the language names, see the highlight.js demo page.

 https://highlightjs.org/usage/

 

Conda

https://linuxize.com/post/how-to-install-anaconda-on-ubuntu-18-04/

https://stackoverflow.com/questions/54843067/no-module-named-torch

# Try to install PyTorch using pip:

# First create a Conda environment using:

conda create -n env_pytorch python=3.6

#Activate the environment using:

conda activate env_pytorch

#Now install PyTorch using pip:

pip install torchvision 

#Note: This will install both torch and torchvision.

#Now go to Python shell and import using the command:

python3

import torch

import torchvision

How-To Setup NVIDIA Docker and NGC Registry on your Workstation

 https://www.pugetsystems.com/labs/hpc/How-To-Setup-NVIDIA-Docker-and-NGC-Registry-on-your-Workstation---Part-4-Accessing-the-NGC-Registry-1115/

https://www.nvidia.com/en-us/gpu-cloud

https://ngc.nvidia.com/setup/installers/cli

# signup/login

https://ngc.nvidia.com/signin 

 

https://ngc.nvidia.com/setup/api-key

# Generate API Key

# Generate your own API key in order to use the NGC service through the Docker client or through NGC CLI.

https://ngc.nvidia.com/setup 

https://ngc.nvidia.com/setup/installers/cli

AMD64 Linux Install

The NGC CLI binary for Linux is supported on Ubuntu 16.04 and later distributions.

Click Download CLI to download the zip file that contains the binary, then transfer the zip file to a directory where you have permissions and then unzip and execute the binary. You can also download, unzip, and install from the command line by moving to a directory where you have execute permissions and then running the following command:

wget -O ngccli_cat_linux.zip https://ngc.nvidia.com/downloads/ngccli_cat_linux.zip && unzip -o ngccli_cat_linux.zip && chmod u+x ngc

Check the binary's md5 hash to ensure the file wasn't corrupted during download:

md5sum -c ngc.md5

Add your current directory to path:

echo "export PATH=\"\$PATH:$(pwd)\"" >> ~/.bash_profile && source ~/.bash_profile

You must configure NGC CLI for your use so that you can run the commands.

Enter the following command, including your API key when prompted:

ngc config set

Linux Uninstall:

Remove the binary file:

rm `which ngc`

 

https://www.pugetsystems.com/labs/hpc/How-To-Setup-NVIDIA-Docker-and-NGC-Registry-on-your-Workstation---Part-4-Accessing-the-NGC-Registry-1115/

"... Or, create a file (or alias) to make that a single command."

"To make that process a little easier you can put the following into a file, (maybe call it ngc-login)

docker login -u '$oauthtoken' --password-stdin nvcr.io <<< ' your API key here between the quotes '

Then to change the permisions to make it only accessible by you,

chmod 700 ngc-login"

~/Docker/ngc_login.sh
WARNING! Your password will be stored unencrypted in /home/.../.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

$ docker run --runtime=nvidia --rm nvcr.io/nvidia/cuda:9.0-cudnn7-devel-ubuntu16.04 nvidia-smi
Unable to find image 'nvcr.io/nvidia/cuda:9.0-cudnn7-devel-ubuntu16.04' locally
9.0-cudnn7-devel-ubuntu16.04: Pulling from nvidia/cuda
4007a89234b4: Pull complete
c1de0f9cdfc1: Pull complete
c8ee6ca703b8: Pull complete
b39e2761d3d4: Pull complete
6fd4d5fe5247: Pull complete
a2aa5ab1b28d: Pull complete
d115189e3588: Pull complete
c208532c0f67: Pull complete
822dc7c71a4b: Pull complete
05d21eb076f4: Pull complete
b62c22f14976: Pull complete
Digest: sha256:eb0f7d58b15893c7d3d676d23e9a4abaaaa6c09019f24bce29a448a9b8a261d8
Status: Downloaded newer image for nvcr.io/nvidia/cuda:9.0-cudnn7-devel-ubuntu16.04
Tue May  4 03:20:53 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.73.01    Driver Version: 460.73.01    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce GTX 106...  Off  | 00000000:01:00.0 Off |                  N/A |
|  0%   36C    P8     7W / 200W |      6MiB /  6078MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+



# Setup the Docker credential store

https://docs.docker.com/engine/reference/commandline/login/#credentials-store

https://github.com/docker/docker-credential-helpers/releases

# NGC commands

https://docs.nvidia.com/dgx/ngc-registry-cli-user-guide/index.html

 

 

=====================================================

What is the NGC registry?

The NGC registry is the main component of NGC. NGC stands for "NVIDIA GPU Cloud". That consists of an NVIDIA maintained Docker registry and an AWS machine instance (AMI) on Amazon AWS. That AMI has a setup similar to what I have gone through in first 2 posts of this series (User-namespaces are not configured).

It's the NGC Docker registry that contains all of the great container images and you can use those docker images on your own workstation. You don't need to access AWS and you don't need to have an AWS account. However, you do need to sign up for NGC in order to get a registry key to access those docker images. It's a private registry so you need an account with NVIDIA NGC and the registry key to login to it.

---

What's in the NGC Registry?

There are 4 section in the registry,

• Deep Learning
• HPC Apps
• HPC Visualization
• And a few "partner" docker images

Here are the GPU optimized application container images is in the repository as of this writing, (using labels as listed in the registry repository)

Repositories

nvidia

• caffe
• caffe2
• cntk
• cuda
• digits
• mxnet
• pytorch
• tensorflow
• tensorrt
• theano
• torch

hpc

• gamess
• gromacs
• lammps
• namd
• relion

nvidia-hpcvis

• paraview-holodeck
• paraview-index
• paraview-optix

partners

• chainer
• h2oai-driverless
• mapd
• paddlepaddle

 

Open Images

https://storage.googleapis.com/openimages/web/download.html

Alternatives to ImageNet

https://opendata.stackexchange.com/questions/11536/alternatives-to-imagenet

https://code.flickr.net/tag/machine-tags/

https://code.flickr.net/2008/12/15/machine-tag-hierarchies/

https://commons.wikimedia.org/wiki/Main_Page

 

Image similarity

https://code.flickr.net/2017/03/07/introducing-similarity-search-at-flickr/

Ubuntu 20.04 installation with support for Tensorflow 2.4

 https://gist.github.com/derekpappas/772ff1357f17e6b161598138b43054e5

Uninstall Docker from Ubuntu

sudo systemctl unmask docker
sudo systemctl enable docker
sudo apt remove --purge docker docker.io containerd runc
sudo rm -Rf /etc/docker

Install node via NVM and then react-native

 

 https://github.com/nvm-sh/nvm#install--update-script

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash

source ~/.bash_profile

https://www.keycdn.com/blog/node-version-manager

https://medium.com/@waqqas/how-to-setup-development-environment-for-react-native-on-mac-85fb216ba0ff

AI Transformers

https://arxiv.org/pdf/1706.03762.pdf

http://jalammar.github.io/illustrated-transformer/

https://towardsdatascience.com/attention-is-all-you-need-discovering-the-transformer-paper-

https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html

http://nlp.seas.harvard.edu/2018/04/03/attention.html

https://github.com/jadore801120/attention-is-all-you-need-pytorch

https://www.youtube.com/watch?v=S27pHKBEp30&ab_channel=SeattleAppliedDeepLearning

https://gist.github.com/leopd/

https://github.com/huggingface/transformers

https://keras.io/examples/structured_data/movielens_recommendations_transformers/

https://towardsdatascience.com/attn-illustrated-attention-5ec4ad276ee3

https://towardsdatascience.com/illustrated-self-attention-2d627e33b20a

https://towardsdatascience.com/the-definitive-guide-to-bi-directional-attention-flow-d0e96e9e666b

https://towardsdatascience.com/illustrated-guide-to-transformer-cf6969ffa067

Scala and JSON

 

https://stackoverflow.com/questions/40172313/scala-read-and-parse-json

https://www.lihaoyi.com/post/HowtoworkwithJSONinScala.html 

 

https://mungingdata.com/scala/read-write-json/

 

https://antoine-doeraene.medium.com/5-things-that-developers-do-all-the-time-in-scala-64626c15440a

Startup Financing Documents

https://siliconhillslawyer.com/2021/03/08/the-carta-safe-seed-rounds/

https://nvca.org/model-legal-documents/

https://siliconhillslawyer.com/2021/01/04/fixing-convertible-note-and-safe-economics-in-seed-rounds/

https://www.linkedin.com/in/joseancer/