OSX remote login

firewall is blocking the IP and port number 22.

https://bluishcoder.co.nz/articles/mac-ssh.html

The Apple Mac OS X operating system has SSH installed by default but the SSH daemon is not enabled. This means you can't login remotely or do remote copies until you enable it.

To enable it, go to 'System Preferences'. Under 'Internet & Networking' there is a 'Sharing' icon. Run that. In the list that appears, check the 'Remote Login' option.

This starts the SSH daemon immediately and you can remotely login using your username. The 'Sharing' window shows at the bottom the name and IP address to use. You can also find this out using 'whoami' and 'ifconfig' from the Terminal application.

https://stackoverflow.com/questions/6313929/how-do-i-open-port-22-in-os-x-10-6-7

I think your port is probably open, but you don't have anything that listens on it.


The Apple Mac OS X operating system has SSH installed by default but the SSH daemon is not enabled. This means you can’t login remotely or do remote copies until you enable it.

To enable it, go to ‘System Preferences’. Under ‘Internet & Networking’ there is a ‘Sharing’ icon. Run that. In the list that appears, check the ‘Remote Login’ option.

This starts the SSH daemon immediately and you can remotely login using your username. The ‘Sharing’ window shows at the bottom the name and IP address to use. You can also find this out using ‘whoami’ and ‘ifconfig’ from the Terminal application.

OSX: SSH for github

The problem is that on OSX the github key needs to be added to the ~/.ssh/config file

https://help.github.com/articles/error-permission-denied-publickey/

$ ssh -T git@github.com

Permission denied (publickey).

$ ssh-add -l -E md5

The agent has no identities.

https://help.github.com/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent/

If you're using macOS Sierra 10.12.2 or later, you will need to modify your ~/.ssh/config file to automatically load keys into the ssh-agent and store passphrases in your keychain.

Host *
 AddKeysToAgent yes
 UseKeychain yes
 IdentityFile ~/.ssh/

$ cd ~/.ssh

$ vi config

$ ssh -T git@github.com

Hi ! You've successfully authenticated, but GitHub does not provide shell access.

build.sbt DSL

http://www.scala-sbt.org/release/docs/Task-Graph.html

What’s the point of the build.sbt DSL? 

The build.sbt DSL is a domain-specific language used construct a DAG of settings and tasks. The setting expressions encode settings, tasks and the dependencies among them.

SBT commands

http://www.scala-sbt.org/release/docs/Running.html

Here are some of the most common sbt commands. For a more complete list, see Command Line Reference.

clean	Deletes all generated files (in the target directory).
compile	Compiles the main sources (in src/main/scala andsrc/main/java directories).
test	Compiles and runs all tests.
console	Starts the Scala interpreter with a classpath including the compiled sources and all dependencies. To return to sbt, type :quit, Ctrl+D (Unix), or Ctrl+Z (Windows).
run *	Runs the main class for the project in the same virtual machine as sbt.
package	Creates a jar file containing the files in src/main/resources and the classes compiled from src/main/scala andsrc/main/java.
help	Displays detailed help for the specified command. If no command is provided, displays brief descriptions of all commands.
reload	Reloads the build definition (build.sbt, project/*.scala,project/*.sbt files). Needed if you change the build definition.

http://www.scala-sbt.org/release/docs/Basic-Def.html

In sbt shell, you can type the name of any task to execute that task. This is why typing compile runs the compile task. compile is a task key.

$ sbt
> tasks

This is a list of tasks defined for the current project.

It does not list the scopes the tasks are defined in; use the 'inspect' command for that.

Tasks produce values.  Use the 'show' command to run the task and print the resulting value.

  clean            Deletes files produced by the build, such as generated sources, compiled classes, and task caches.

  compile          Compiles sources.

  console          Starts the Scala interpreter with the project classes on the classpath.

  consoleProject   Starts the Scala interpreter with the sbt and the build definition on the classpath and useful imports.

  consoleQuick     Starts the Scala interpreter with the project dependencies on the classpath.

  copyResources    Copies resources to the output directory.

  doc              Generates API documentation.

  package          Produces the main artifact, such as a binary jar.  This is typically an alias for the task that actually does the packaging.

  packageBin       Produces a main artifact, such as a binary jar.

  packageDoc       Produces a documentation artifact, such as a jar containing API documentation.

  packageSrc       Produces a source artifact, such as a jar containing sources and resources.

  publish          Publishes artifacts to a repository.

  publishLocal     Publishes artifacts to the local Ivy repository.

  publishM2        Publishes artifacts to the local Maven repository.

  run              Runs a main class, passing along arguments provided on the command line.

  runMain          Runs the main class selected by the first argument, passing the remaining arguments to the main method.

  test             Executes all tests.

  testOnly         Executes the tests provided as arguments or all tests if no arguments are provided.

  testQuick        Executes the tests that either failed before, were not run or whose transitive dependencies changed, among those provided as arguments.

  update           Resolves and optionally retrieves dependencies, producing a report.

More tasks may be viewed by increasing verbosity.  See 'help tasks'.

SBT templates

http://www.scala-sbt.org/0.13/docs/sbt-new-and-Templates.html

Create a new sbt build directory tree using a template.

sbt new sbt/scala-seed.g8

SPARK SimpleApp

https://blog.knoldus.com/2014/06/04/a-simple-application-in-spark-and-scala/

sbt package

$SPARK_HOME/bin/spark-submit   --class "SimpleApp"   --master local[4]   target/scala-2.11/simple-project_2.11-1.0.jar

SBT tips and tricks from Stackoverflow

https://stackoverflow.com/questions/27710887/kafkautils-class-not-found-in-spark-streaming

SPARK: running the examples from https://spark.apache.org

https://spark.apache.org/docs/0.9.0/

./bin/run-example org.apache.spark.examples.SparkPi

./bin/run-example org.apache.spark.examples.streaming.KafkaWordCount

Usage: KafkaWordCount

SSH Shortcuts

http://www.phcomp.co.uk/Tutorials/Unix-And-Linux/ssh-check-server-fingerprint.html

https://help.github.com/articles/github-s-ssh-key-fingerprints/

To compare your local key's finger print against the github fingerprint:

ssh-keygen -l -E md5 -f ~/.ssh/joe_rsa

4096 MD5:7a:95:35:2e:e3:47:43:2a:4c:4d:69:51:9d:bb:83:12 joe.smith@any.com (RSA)

z = keras.layers.add([x, y]) NameError: name 'keras' is not defined

https://keras.io/getting-started/functional-api-guide/#more-examples

from keras.layers import Conv2D, MaxPooling2D, Input
#import keras

input_img = Input(shape=(3, 256, 256))

tower_1 = Conv2D(64, (1, 1), padding='same', activation='relu')(input_img)
tower_1 = Conv2D(64, (3, 3), padding='same', activation='relu')(tower_1)

tower_2 = Conv2D(64, (1, 1), padding='same', activation='relu')(input_img)
tower_2 = Conv2D(64, (5, 5), padding='same', activation='relu')(tower_2)

tower_3 = MaxPooling2D((3, 3), strides=(1, 1), padding='same')(input_img)
tower_3 = Conv2D(64, (1, 1), padding='same', activation='relu')(tower_3)

output = keras.layers.concatenate([tower_1, tower_2, tower_3], axis=1)

$ python concat.py Using TensorFlow backend.

Traceback (most recent call last):

  File "concat.py", line 15, in

    output = keras.layers.concatenate([tower_1, tower_2, tower_3], axis=1)

NameError: name 'keras' is not defined

#######################################

>>> keras

Traceback (most recent call last):

  File "", line 1, in

NameError: name 'keras' is not defined

>>> import keras

>>> keras

>>> 

#######################################

# Fix

from keras.layers import Conv2D, MaxPooling2D, Input
import keras # <<<<< Fix

input_img = Input(shape=(3, 256, 256))

tower_1 = Conv2D(64, (1, 1), padding='same', activation='relu')(input_img)
tower_1 = Conv2D(64, (3, 3), padding='same', activation='relu')(tower_1)

tower_2 = Conv2D(64, (1, 1), padding='same', activation='relu')(input_img)
tower_2 = Conv2D(64, (5, 5), padding='same', activation='relu')(tower_2)

tower_3 = MaxPooling2D((3, 3), strides=(1, 1), padding='same')(input_img)
tower_3 = Conv2D(64, (1, 1), padding='same', activation='relu')(tower_3)

output = keras.layers.concatenate([tower_1, tower_2, tower_3], axis=1)

Saving a model that does not import the backend as "K" breaks model loading. #5088

 https://github.com/fchollet/keras/issues/5088

airalcorn2 commented on Jan 19 • edited

The following code produces this error: NameError: name 'backend' is not defined. from keras import backend from keras.layers import Dense, Input, Lambda from keras.models import load_model, Model x = Input((100, ), dtype = "float32") x_i = Lambda(lambda x: x + backend.epsilon())(x) o = Dense(10, activation = "softmax")(x_i) model = Model(input = [x], output = o) model.compile("sgd", loss = "categorical_crossentropy") model.save("toy_model.h5") model = load_model("toy_model.h5")

 airalcorn2 changed the title from Saving a model that does not import the Keras backend as "K" breaks model loading. to Saving a model that does not import the backend as "K" breaks model loading. on Jan 19

Contributor

joelthchao commented on Jan 19

Quick solution: change backend import name, keras doesn't know your variable in lambda layer from keras import backend as K from keras.layers import Dense, Input, Lambda from keras.models import load_model, Model x = Input((100, ), dtype = "float32") x_i = Lambda(lambda x: x + K.epsilon())(x) o = Dense(10, activation = "softmax")(x_i) model = Model(input = [x], output = o) model.compile("sgd", loss = "categorical_crossentropy") model.save("toy_model.h5") model = load_model("toy_model.h5")

airalcorn2 commented on Jan 19 • edited

@joelthchao - thanks for trying to help, but I had already realized that changing the import name for backend to K would make the error go away (hence, my title). The problem is that Keras allows you to save a model using the actual name of the backend module but is unable to load such a model.

Contributor

bstriner commented on Jan 19

No quick fix but worth looking into. There are similar issues if you try to use some custom imports in your lambda. Another workaround is to do the import within the Lambda. def mylambda(x): from keras import backend import mymodule ... #use backend and mymodule y = Lambda(mylambda)(h)

"Merge" versus "merge", what is the difference? #3921

https://github.com/fchollet/keras/issues/3921

farizrahman4u commented on Sep 29, 2016 • edited

Merge is a layer. Merge takes layers as input Merge is usually used with Sequential models merge is a function. merge takes tensors as input. merge is a wrapper around Merge. merge is used in Functional API Using Merge: left = Sequential() left.add(...) left.add(...) right = Sequential() right.ad(...) right.add(...) model = Sequential() model.add(Merge([left, right])) model.add(...) using merge: a = Input((10,)) b = Dense(10)(a) c = Dense(10)(a) d = merge([b, c]) model = Model(a, d) Can you please confirm that the Keras 1.2.2 code from keras.engine import merge m = merge([init, x], mode='sum') is equivalent to this Keras 2.0.2 code: from keras.layers import add m = add([init, x])