Tuesday, August 27, 2019

Scala ArrowAssoc

https://stackoverflow.com/questions/4980515/scala-maps-operator/4980734

implicit def any2ArrowAssoc[A](x: A): ArrowAssoc[A] = new ArrowAssoc(x)
This will convert any type into an instance of "ArrowAssoc"
class ArrowAssoc[A](x: A) {
    def -> [B](y: B): Tuple2[A, B] = Tuple2(x, y)
}
So when Scala sees 
"a"->1
It says "There is no method named '->' on String. Are there any implicit conversions in scope that can give me a type that has a method named '->'?" Predef.scala is automatically in scope, and offers a conversion to ArrowAssoc, which obviously has the '->' method. Scala then essentially converts the above to
any2ArrowAssoc("a").->(1)
This method returns a Tuple2("a", 1) (often called a Pair). Map has a constructor that thats an array (varargs) of Tuple2s, so we're off to races! No magic in the compiler (besides implicit conversion, which is used extensively and for many different purposes), and no magic in Maps constructor.

Thursday, August 1, 2019

Instructions for the C++ Torch installation modified for the Mac

https://github.com/intel/mkl-dnn/releases
https://github.com/intel/mkl-dnn

https://intel.github.io/mkl-dnn/dev_guide_build.html

Linux/macOS

  • Generate makefile: 
    mkdir -p build && cd build && cmake ..
  • Build the library: 
    make -j
  • Build the documentation: 
    make doc
  • Install the library, headers, and documentation: 
    make install
git clone https://github.com/intel/mkl-dnn.git
cd mkl-dnn/
mkdir -p build && cd build && cmake ..
make -j
make doc
make install

Validate the Build

Run unit tests:
ctest


// add this env var to your .bash_profile file

export Torch_DIR=$USER/libtorch

// HACK ALERT
// https://github.com/pytorch/pytorch/issues/14727#issuecomment-516575929

wget https://github.com/intel/mkl-dnn/releases/download/v0.20-rc/mklml_mac_2019.0.5.20190502.tgz
mv ~/Downloads//mklml_mac_2019.0.5.20190502.tar .
tar xvf mklml_mac_2019.0.5.20190502.tar


wget https://download.pytorch.org/libtorch/nightly/cpu/libtorch-macos-latest.zip
unzip libtorch-macos-latest.zip 
cd build/
source ~/.bash_profile
cd ..
mv first_example.cc first_example.cpp
cd build

cmake -DCMAKE_PREFIX_PATH=~/libtorch ..

~/torch/build
./first_example 
 0.9762  0.1762  0.9328
 0.5004  0.3410  0.1637
 0.3285  0.0761  0.6350
[ Variable[CPUFloatType]{3,3} ]
 0.9762  0.1762  0.9328
 0.5004  0.3410  0.1637
 0.3285  0.0761  0.6350

[ Variable[CPUFloatType]{3,3} ]



https://github.com/zdevito/ATen/tree/master/aten/src



Friday, April 19, 2019

VP Trees

https://en.wikipedia.org/wiki/Triangle_inequality


https://en.wikipedia.org/wiki/Triangle_inequality



VP trees: A data structure for finding stuff fast

http://ceur-ws.org/Vol-256/submission_12.pdf

https://nullwords.wordpress.com/2013/03/13/the-bk-tree-a-data-structure-for-spell-checking/

http://stevehanov.ca/blog/?id=130

https://fribbels.github.io/vptree/writeup

https://towardsdatascience.com/symspell-vs-bk-tree-100x-faster-fuzzy-string-search-spell-checking-c4f10d80a078

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.36.7401&rep=rep1&type=pdf



2. Metric Spaces and Similarity Queries
queries.
In this section, we briefly give the definitions for metric distance functions and different types of similarity
A metric distance function d(x,y) for a metric space is defined as follows:
i) d(x,y) = d(y,x)ii) 0 < d(x,y) < , x≠ yiii) d(x,x) = 0iv) d(x,y) ≤ d(x,z) + d(z,y) (triangle inequality)
The above conditions are the only ones we can assume when designing an index structure based on distances between objects in a metric space. Note that, for a metric space, no geometric information can be utilized unlike the case for a Euclidean space. Thus, we only have a set of objects from a metric space, and a distance function d() that can be used to compute the distance between any two objects.
Similarity based queries can be posed in a number of ways. The most common one asks for all data objects that are within some specified distance from a given query object. These queries require retrieval of near neighborsof the query object. The formal definition for this type of queries is as follows:
Near Neighbor Query: From a given set of data objects X = {X1, X2, ..., Xn} from a metric space with a metric distance function d(), retrieve all data objects that are within distance of a given query point Y. The resulting set will be { X| X∈ X and d(Xi,Y)≤ }. Here, is generally referred to as the similarity measure, or the tolerance factor.
Some variations of the near neighbor query are also possible. The nearest neighbor query asks for the closest object to a given query object. Similarly, k closest objects may be requested as well.

http://blog.notdot.net/2007/4/Damn-Cool-Algorithms-Part-1-BK-Trees

https://karczmarczuk.users.greyc.fr/TEACH/TAL/Doc/BK_search/shasha_best.pdf

Burkhard and Keller in 1973, in their paper "Some approaches to best match file searching" pdf.



Wednesday, March 27, 2019

How to install OpenCV 3.4.0 on Ubuntu 16.04

How to install OpenCV 3.4.0 on Ubuntu 16.04

Install OpenCV 3.4.0 on Ubuntu 16.04 for all versions of python (2.7, 3.5 and 3.6)

OpenCV is a most popular free and open-source computer vision library among students, researchers, and developers alike. It has  C, C++, Python and Java interfaces and supports Windows, Linux, Mac OS, iOS, and Android. At the time of writing of this blog, the latest version of OpenCV is 3.4.0. This tutorial is designed to help you install OpenCV 3.4.0 on Ubuntu 16.04. This tutorial works for versions 2.7, 3.5 and 3.6 of python. If you want to know how to install python 3.6 on Ubuntu, check out our other tutorial Install Python 3.6 on Ubuntu.
You can install opencv by using official prebuilt pip package or build from source.

To install from official pip package:

Use following commands in a new active empty virtual environment.
To install Opencv Only:
pip install opencv-python
To add contrib package in opencv:
pip install opencv-contrib-python

To install opencv by building from source:

OpenCV can be used in python as C++ python binding. I am going to explain how to build OpenCV from source and generate python binding.
The steps needed to be taken in order to install OpenCV 3.4.0 are:
  1. Update and Upgrade and Cleanup (Required for error-free installation)
  2. Install Dependencies
  3. Download and Build OpenCV

Update and Upgrade and Cleanup (Required for error-free installation)

sudo apt-get update
sudo apt-get upgrade
Now that we have the updates ready, we are ready to install dependencies.

Install Dependencies

sudo apt-get install build-essential 
sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
The following command is needed to process images:
sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
To process videos:
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libxvidcore-dev libx264-dev
For GUI:
sudo apt-get install libgtk-3-dev
For optimization:
sudo apt-get install libatlas-base-dev gfortran pylint
To build OpenCV binding for both python 2 and 3.
sudo apt-get install python2.7-dev python3.5-dev
We are now ready to download and build OpenCV on our Ubuntu machine.

Download and Build OpenCV

To Download OpenCV 3.4.0, go to terminal and type:
wget https://github.com/opencv/opencv/archive/3.4.0.zip -O opencv-3.4.0.zip
To Download OpenCV Contrib 3.4.0 (Contrib has very useful algorithms which is a must for anyone working on Computer Vision), go to terminal and type:
wget https://github.com/opencv/opencv_contrib/archive/3.4.0.zip -O opencv_contrib-3.4.0.zip
We need to unzip to extract the zip files. Install unzip if not installed using the command:
sudo apt-get install unzip
Now extract OpenCV and OpenCV Contrib:
unzip opencv-3.4.0.zip
unzip opencv_contrib-3.4.0.zip
Make a directory named build inside OpenCV-3.4.0:
cd  opencv-3.4.0
mkdir build
cd build
Now we are going to configure cmake:
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local -DOPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-3.4.0/modules -DOPENCV_ENABLE_NONFREE=True ..
The command will take some time to execute. Wait for few seconds.
If the configuration is done without error, we will use make command to build.
make -j4 (where 4 is number of cores)
This command will take a few minutes to complete. On my PC, it took about 20 minutes.
Install in the location /usr/local using command
sudo make install
Reinitialize static libs using the following command
sudo ldconfig
Check if the OpenCV file exists
  • python2
“/usr/local/lib/python2.7/dist-packages/cv2.so”
  • python3.5
“/usr/local/lib/python3.5/dist-packages/cv2.cpython-35m-x86_64-linux-gnu.so”
  • python3.6
“/usr/local/lib/python3.6/dist-packages/cv2.cpython-36m-x86_64-linux-gnu.so”
To use OpenCV 3.4.0 in virtual environment like virtualenv, conda or any other, copy related .so file in your virtual environment python directory
“lib/site-packages/”
Finally, check OpenCV version. Go to the terminal and typepython
On the python terminal, type
import cv2
cv2.__version__
If you ran into any problem while installing OpenCV or have any query related to OpenCV then feel free to comment below. I will try my best to solve it.

Thursday, February 21, 2019

git rebasing

git fetch origin
git rebase -i origin/master
git push --force-with-lease

Monday, February 18, 2019

Ubuntu tips

Hold down the control key to see the Ubuntu Unity keyboard shortcuts

Install Unity tweak tool to customize your Unity desktop env
sudo apt-get install unity-tweak-tool gnome-tweak-tool
unity-tweak-tool

Set the location and visibility of the dock




cslc compilation

On Ubuntu 
 1. install Boost
Download boost
https://www.boost.org/doc/libs/1_62_0/more/getting_started/unix-variants.html

https://codeyarns.com/2017/01/24/how-to-build-boost-on-linux/


 
 undefined reference to `intToString[abi:cxx11](int)'

 https://stackoverflow.com/questions/39826917/boost-build-fails-c11-feature-checks-when-using-custom-gcc-4-x-or-5-x?noredirect=1&lq=1

https://docs.computecanada.ca/wiki/GCC_C%2B%2B_Dual_ABI


https://stackoverflow.com/questions/39826917/boost-build-fails-c11-feature-checks-when-using-custom-gcc-4-x-or-5-x?noredirect=1&lq=1


$ echo "using gcc : 5.4 : /usr/bin/g++-5 : -std=c++11 ;" > ./tools/build/src/user-config.jam



Let's assume for the sake of this example that:
  • You want to build Boost with GCC 5.4
  • The g++ 5.4 binary is at /some/where/g++-5.4
  • You've downloaded the Boost sources and unpacked them into /path/to/sources/of/boost-1.62.0/
  • (Perhaps) you want to install Boost to /dest/path
  • You have N cores (so you want to parallelize the build N-ways)
Now:./b2 install --prefix=--prefix=/opt/local
  1. cd /path/to/sources/of/boost-1.62.0/
  2. Booststrap the build system by running ./bootstrap.sh
  3. echo "using gcc : 5.4 : /usr/bin/g++-5 : -std=c++11 ;" > ./tools/build/src/user-config.jam
  4. ./b2 --prefix=--prefix=/opt/local --toolset=gcc-5 -j 4 define=_GLIBCXX_USE_CXX11_ABI=0
  5. ./b2 --prefix=--prefix=/opt/local --toolset=gcc-5 -j 4 (N being the number of cores on your system)
  6. ./b2 install --prefix=/opt/local
  7.  
  8.  
  9.  ./b2 --prefix=--prefix=/opt/local --toolset=gcc-5 -j 4 define=_GLIBCXX_USE_CXX11_ABI=0

wget https://www-eu.apache.org/dist//xerces/c/KEYS
gpg --import < KEYS
wget https://www-eu.apache.org/dist//xerces/c/3/sources/xerces-c-3.2.2.tar.gz

// Compare the following sha256 to https://www-eu.apache.org/dist//xerces/c/3/sources/xerces-c-3.2.2.tar.gz

sha256sum xerces-c-3.2.2.tar.gz
dd6191f8aa256d3b4686b64b0544eea2b450d98b4254996ffdfe630e0c610413  xerces-c-3.2.2.tar.gz

cd xerces...

./configure
make
sudo make install
$ ls /usr/local/lib/
libxerces-c-3.2.so*  libxerces-c.la*  pkgconfig/  python3.5/
libxerces-c.a         libxerces-c.so@  python2.7/



ELK Stack

Tuesday, February 12, 2019

Thursday, December 13, 2018

From: https://jekyllrb.com/docs/usage/

Command Line Usage
The Jekyll gem makes a jekyll executable available to you in your terminal.
You can use this command in a number of ways:
jekyll new - Creates a new Jekyll site with default gem-based theme
jekyll new --blank - Creates a new blank Jekyll site scaffold
jekyll build or jekyll b - Performs a one off build your site to ./_site (by default)
jekyll serve or jekyll s - Builds your site any time a source file changes and serves it locally
jekyll doctor - Outputs any deprecation or configuration issues
jekyll new-theme - Creates a new Jekyll theme scaffold
jekyll clean - Removes the generated site and metadata file
jekyll help - Shows help, optionally for a given subcommand, e.g. jekyll help build
Typically you’ll use jekyll serve while developing locally and jekyll build when you need to generate the site for production.
To change Jekyll’s default build behavior have a look through the configuration options.

Configure Atom for Jekyll

https://atom.io/packages/jekyll

  • The Build Command will most likely need changing due to some weirdness with $PATH in Atom.
    • Go into the packages settings (Settings -> Packages -> Jekyll).
    • Change Build Command to /path/to/jekyll, build.
    • You can get your /path/to/jekyll by using which jekyll in your terminal.
Test on the Mac with shift-option-T
Test on a Windows machine with shift-alt-T

Wednesday, November 28, 2018

Github pages, Atom, and Jekyll

https://help.github.com/articles/setting-up-your-github-pages-site-locally-with-jekyll/

https://atom.io/packages/jekyll

https://jekyll.github.io/jekyll-seo-tag/

Start Atom

Install the necessary packages



Start and stop the server with shift-option-t on MacOSX



If you run into the above error then do the following:

https://github.com/jekyll/jekyll/issues/4268

Add this to the _config.yml