Monday, September 16, 2019

Scala fold left-augmented examples

https://commitlogs.com/2016/09/10/scala-fold-foldleft-and-foldright/

Code copied from the above link.
Print statements added to show the intermediate results.

val inputList: List[Int] = List(1, 3, 5)
inputList.foldLeft(0) { (acc, i) => acc + i }

def op(acc: Int, i: Int): Int = acc + i

op(acc = 0, i = 1) // 1
op(acc = 1, i = 3) // 4
op(acc = 4, i = 5) // 9

op(op(op(0, 1), 3), 5)
op(op(0 + 1, 3), 5)
op(op(1, 3), 5)
op(1 + 3, 5)
op(4, 5)
(4 + 5)
9
_________________________________________________________________________________

def len(list: List[Any]): Int = list.foldLeft(0) { (count, _) => count + 1 }

op(0, 1) // 1
op(1, 3) // 2
op(2, 5) // 3

op(op(op(0, 1), 3), 5)
op(op(1, 3), 5)
op(2, 5)
3

_________________________________________________________________________________

def len(list: List[Any]): Int = list.foldLeft(0) { (count, i) => println(s"""c: $count i: $i"""); count + 1 }

op(0, 1) // 1
op(1, 3) // 2
op(2, 5) // 3

op(op(op(0, 1), 3), 5)
op(op(1, 3), 5)
op(2, 5)
3

_________________________________________________________________________________

scala> def len(list: List[Any]): Int = list.foldLeft(0) { (count, i) => println(s"""c: $count i: $i"""); count + 1 }
len: (list: List[Any])Int

scala> val inputList: List[Int] = List(1, 3, 5)
inputList: List[Int] = List(1, 3, 5)

scala> len(inputList)
c: 0 i: 1
c: 1 i: 3
c: 2 i: 5
res0: Int = 3

_________________________________________________________________________________


scala> def last[A](list: List[A]): A = list.foldLeft[A](list.head) { (x, cur) =>  println(s"""x: $x cur: $cur"""); cur }
last: [A](list: List[A])A

scala> last(inputList)
x: 1 cur: 1
x: 1 cur: 3
x: 3 cur: 5

res1: Int = 5

_________________________________________________________________________________


scala>  def average(list: List[Double]): Double = list match {
     |   case head :: tail => tail.foldLeft((head, 1.0)) { (avg, cur) =>
     |     ((avg._1 * avg._2 + cur)/(avg._2 + 1.0), avg._2 + 1.0)
     |   }._1
     |   case Nil => Double.NaN
     | }
average: (list: List[Double])Double

scala> average(inputList.map(_.toDouble))
res4: Double = 3.0




_________________________________________________________________________________


scala>  def average(list: List[Double]): Double = list match {
     |   case head :: tail => tail.foldLeft((head, 1.0)) { (avg, cur) =>
     |     println(s"""sum: ${avg._1 * avg._2 + cur} count: ${avg._2 + 1.0}""");
     |     ((avg._1 * avg._2 + cur)/(avg._2 + 1.0), avg._2 + 1.0)
     |   }._1
     |   case Nil => Double.NaN
     | }
average: (list: List[Double])Double

scala> 

scala> average(inputList.map(_.toDouble))
sum: 4.0 count: 2.0
sum: 9.0 count: 3.0
res7: Double = 3.0

_________________________________________________________________________________

scala> def reverse[A](list: List[A]): List[A] =
     |   list.foldLeft(List[A]()) { (r,c) =>  println(s"""r: ${r} c: ${c} """); c :: r }
reverse: [A](list: List[A])List[A]

scala> val l = List(1,3,5,7,9,11,13,15,17)
l: List[Int] = List(1, 3, 5, 7, 9, 11, 13, 15, 17)

scala> reverse(l)
r: List() c: 1 
r: List(1) c: 3 
r: List(3, 1) c: 5 
r: List(5, 3, 1) c: 7 
r: List(7, 5, 3, 1) c: 9 
r: List(9, 7, 5, 3, 1) c: 11 
r: List(11, 9, 7, 5, 3, 1) c: 13 
r: List(13, 11, 9, 7, 5, 3, 1) c: 15 
r: List(15, 13, 11, 9, 7, 5, 3, 1) c: 17 
res10: List[Int] = List(17, 15, 13, 11, 9, 7, 5, 3, 1)

_________________________________________________________________________________

scala> def reverse[A](list: List[A]): List[A] =
     |   list.foldLeft(List[A]()) { (r,c) =>  println(s"""r: ${r} c: ${c} """); c :: r }
reverse: [A](list: List[A])List[A]

scala> val l = List(1,3,5,7,9,11,13,15,17)
l: List[Int] = List(1, 3, 5, 7, 9, 11, 13, 15, 17)

scala> reverse(l)
r: List() c: 1 
r: List(1) c: 3 
r: List(3, 1) c: 5 
r: List(5, 3, 1) c: 7 
r: List(7, 5, 3, 1) c: 9 
r: List(9, 7, 5, 3, 1) c: 11 
r: List(11, 9, 7, 5, 3, 1) c: 13 
r: List(13, 11, 9, 7, 5, 3, 1) c: 15 
r: List(15, 13, 11, 9, 7, 5, 3, 1) c: 17 
res10: List[Int] = List(17, 15, 13, 11, 9, 7, 5, 3, 1)
















Thursday, September 5, 2019

Debugging SBT build files

1. How to see dependency tree in sbt?
-Inspect the SBT dependency tree as described in the documentation:
sbt:foo> inspect tree clean

2. Sonatype repos
https://oss.sonatype.org/content/repositories/public/

3. Resolvers
resolvers ++= Seq(
Resolver.sonatypeRepo("releases"),
Resolver.sonatypeRepo("snapshots")
)

3. Why is SBT unable to resolve my dependency?

4.

5. sbt: Dependency Management

6. When the libdeps are not being imported by a resolver check that the resolver is looking for the correct version at the remote repo. Check the Scala compiler and package version number.

val circeVersion = "0.8.0"
resolvers ++= Seq(
  Resolver.sonatypeRepo("releases"),  Resolver.sonatypeRepo("snapshots")
)

libraryDependencies ++= Seq(
  "io.circe" %% "circe-core",  "io.circe" %% "circe-generic",  "io.circe" %% "circe-parser").map(_ % circeVersion)

7. Check the package Scala compiler and package version numbers in the Ivy cache.

 → ls ~/.ivy2/cache/io.circe/circe-core_2.12/jars/
circe-core_2.12-0.8.0.jar

8. Search at the remote repo for the package to see the version numbers that are available.
https://oss.sonatype.org/#nexus-search;quick~circe

Tuesday, August 27, 2019

Scala ArrowAssoc

https://stackoverflow.com/questions/4980515/scala-maps-operator/4980734

implicit def any2ArrowAssoc[A](x: A): ArrowAssoc[A] = new ArrowAssoc(x)
This will convert any type into an instance of "ArrowAssoc"
class ArrowAssoc[A](x: A) {
    def -> [B](y: B): Tuple2[A, B] = Tuple2(x, y)
}
So when Scala sees 
"a"->1
It says "There is no method named '->' on String. Are there any implicit conversions in scope that can give me a type that has a method named '->'?" Predef.scala is automatically in scope, and offers a conversion to ArrowAssoc, which obviously has the '->' method. Scala then essentially converts the above to
any2ArrowAssoc("a").->(1)
This method returns a Tuple2("a", 1) (often called a Pair). Map has a constructor that thats an array (varargs) of Tuple2s, so we're off to races! No magic in the compiler (besides implicit conversion, which is used extensively and for many different purposes), and no magic in Maps constructor.

Thursday, August 1, 2019

Instructions for the C++ Torch installation modified for the Mac

https://github.com/intel/mkl-dnn/releases
https://github.com/intel/mkl-dnn

https://intel.github.io/mkl-dnn/dev_guide_build.html

Linux/macOS

  • Generate makefile: 
    mkdir -p build && cd build && cmake ..
  • Build the library: 
    make -j
  • Build the documentation: 
    make doc
  • Install the library, headers, and documentation: 
    make install
git clone https://github.com/intel/mkl-dnn.git
cd mkl-dnn/
mkdir -p build && cd build && cmake ..
make -j
make doc
make install

Validate the Build

Run unit tests:
ctest


// add this env var to your .bash_profile file

export Torch_DIR=$USER/libtorch

// HACK ALERT
// https://github.com/pytorch/pytorch/issues/14727#issuecomment-516575929

wget https://github.com/intel/mkl-dnn/releases/download/v0.20-rc/mklml_mac_2019.0.5.20190502.tgz
mv ~/Downloads//mklml_mac_2019.0.5.20190502.tar .
tar xvf mklml_mac_2019.0.5.20190502.tar


wget https://download.pytorch.org/libtorch/nightly/cpu/libtorch-macos-latest.zip
unzip libtorch-macos-latest.zip 
cd build/
source ~/.bash_profile
cd ..
mv first_example.cc first_example.cpp
cd build

cmake -DCMAKE_PREFIX_PATH=~/libtorch ..

~/torch/build
○ → ./first_example 
 0.9762  0.1762  0.9328
 0.5004  0.3410  0.1637
 0.3285  0.0761  0.6350
[ Variable[CPUFloatType]{3,3} ]
 0.9762  0.1762  0.9328
 0.5004  0.3410  0.1637
 0.3285  0.0761  0.6350

[ Variable[CPUFloatType]{3,3} ]



https://github.com/zdevito/ATen/tree/master/aten/src



Friday, April 19, 2019

VP Trees

https://en.wikipedia.org/wiki/Triangle_inequality


https://en.wikipedia.org/wiki/Triangle_inequality



VP trees: A data structure for finding stuff fast

http://ceur-ws.org/Vol-256/submission_12.pdf

https://nullwords.wordpress.com/2013/03/13/the-bk-tree-a-data-structure-for-spell-checking/

http://stevehanov.ca/blog/?id=130

https://fribbels.github.io/vptree/writeup

https://towardsdatascience.com/symspell-vs-bk-tree-100x-faster-fuzzy-string-search-spell-checking-c4f10d80a078

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.36.7401&rep=rep1&type=pdf



2. Metric Spaces and Similarity Queries
queries.
In this section, we briefly give the definitions for metric distance functions and different types of similarity
A metric distance function d(x,y) for a metric space is defined as follows:
i) d(x,y) = d(y,x)ii) 0 < d(x,y) < ∞, x≠ yiii) d(x,x) = 0iv) d(x,y) ≤ d(x,z) + d(z,y) (triangle inequality)
The above conditions are the only ones we can assume when designing an index structure based on distances between objects in a metric space. Note that, for a metric space, no geometric information can be utilized unlike the case for a Euclidean space. Thus, we only have a set of objects from a metric space, and a distance function d() that can be used to compute the distance between any two objects.
Similarity based queries can be posed in a number of ways. The most common one asks for all data objects that are within some specified distance from a given query object. These queries require retrieval of near neighborsof the query object. The formal definition for this type of queries is as follows:
Near Neighbor Query: From a given set of data objects X = {X1, X2, ..., Xn} from a metric space with a metric distance function d(), retrieve all data objects that are within distance r of a given query point Y. The resulting set will be { Xi | Xi ∈ X and d(Xi,Y)≤ r }. Here, r is generally referred to as the similarity measure, or the tolerance factor.
Some variations of the near neighbor query are also possible. The nearest neighbor query asks for the closest object to a given query object. Similarly, k closest objects may be requested as well.

http://blog.notdot.net/2007/4/Damn-Cool-Algorithms-Part-1-BK-Trees

https://karczmarczuk.users.greyc.fr/TEACH/TAL/Doc/BK_search/shasha_best.pdf

Burkhard and Keller in 1973, in their paper "Some approaches to best match file searching" pdf.