Saturday, March 19, 2016

using-docker-to-efficiently-create-multiple-tomcat-instances/

Wednesday, March 16, 2016

Atomic

Kubernetes

Resetting the Ambari password from the command line

boot2docker for Mac OSX and Windows Docker containers

SequenceIOdocker-ambari

https://github.com/sequenceiq/docker-ambari

cd docker-ambari
sudo make
amb-settings
amb-start-cluster 3
  219  sudo route add -net 172.17.0.0/16 docker-machine



Another Vagrant, VB, Ansible HDP setup: from; Henning Kropp

http://henning.kropponline.de/2015/01/11/hdp-ansible-playbook-example/

https://github.com/hkropp/vagrant-hdp

vagrant ssh

[vagrant@one ~]$ cd /vagrant

vagrant@one vagrant]$ ls
ansible  bin  blueprint.json  blueprints  conf  hostmapping.json  inc  install_blueprint.sh  install.sh  manifests  master_blueprint.json  puppet  README.md  Vagrantfile

 [vagrant@one vagrant]$ ./install_blueprint.sh

In a web browser: http://localhost:8080
username: admin
password: admin

Change the password from the command. See other post.
https://www.blogger.com/blogger.g?blogID=2189538179618563464#editor/target=post;postID=3339040800301665217;onPublishedMenu=allposts;onClosedMenu=allposts;postNum=0;src=postname

Go to the Ambari dashboard
YARN / one.hdp CRIT for 2 minutes 1 Connection failed to http://one.hdp:8188

Clck on ops in the top menu bar and watch the installation progress.


Hortonworks Cloudbreak

SequnceIO Hadoop on Docker introduction

Hadoop on Docker introduction
http://blog.sequenceiq.com/blog/2014/04/04/hadoop-docker-introduction/

Create Hadoop clusters in the cloud using a CLI
http://blog.sequenceiq.com/blog/2014/08/07/clodubreak-shell/

Real-time monitoring of Hadoop clusters
http://blog.sequenceiq.com/blog/2014/10/07/hadoop-monitoring/

Apache Ambari + Spring Shell = Ambari Shell

http://blog.sequenceiq.com/blog/2014/05/26/ambari-shell/

sequenceiq hadoop docker containers

Docker installation on Linux

Install Docker
 https://docs.docker.com/linux/

$ curl -fsSL https://get.docker.com/ | sh
$ curl -fsSL https://get.docker.com/gpg | sudo apt-key add - 
$ docker run hello-world 
docker: Cannot connect to the Docker daemon. Is the docker daemon running on this host?.
See 'docker run --help'. 

Fix for this problem:
http://stackoverflow.com/questions/25404597/cannot-connect-to-the-docker-daemon-is-docker-d-running-on-this-host

After running the command below log out and log back in.
sudo usermod -aG docker 

Rerun the Docker hello world command. The command should work now.
docker run hello-world

Rerun the Docker hello world command. The command should work now.
docker run hello-world
 
Install docker-machine
https://docs.docker.com/machine/

sudo curl -L https://github.com/docker/machine/releases/download/v0.6.0/docker-machine-`uname -s`-`uname -m` > docker-machine
sudo cp docker-machine /usr/local/bin/docker-machine
 
sudo chmod +x /usr/local/bin/docker-machine
 
which docker-machine

Install the docker-machine bash completion scripts

https://github.com/docker/machine/tree/master/contrib/completion/bash

git clone git@github.com:docker/machine.git
sudo cp machine/contrib/completion/bash/* /etc/bash_completion.d
 
 
https://docs.docker.com/machine/get-started/ 
(Virtualbox should already be installed) 
docker-machine create --driver virtualbox default
docker-machine ls
docker-machine env default 
eval "$(docker-machine env default)" 
docker run busybox echo hello world  
docker-machine ip default 
docker run -d -p 8000:80 nginx
curl $(docker-machine ip default):8000 
 
https://docs.docker.com/machine/migrate-to-machine/ 
 
boot2docker docker-machine docker-machine description
init create Creates a new docker host.
up start Starts a stopped machine.
ssh ssh Runs a command or interactive ssh session on the machine.
save - Not applicable.
down stop Stops a running machine.
poweroff stop Stops a running machine.
reset restart Restarts a running machine.
config inspect Prints machine configuration details.
status ls Lists all machines and their status.
info inspect Displays a machine’s details.
ip ip Displays the machine’s ip address.
shellinit env Displays shell commands needed to configure your shell to interact with a machine
delete rm Removes a machine.
download - Not applicable.
 


 
 

Tuesday, March 15, 2016

Random notes: Hadoop on Docker introduction for OSX

Random notes: Building the build environment with Ansible and Docker


Building the build environment with Ansible and Docker
Marton Sereg09 May 2014
http://blog.sequenceiq.com/blog/2014/05/09/building-the-build-environment-with-ansible-and-docker/



Random notes: Vagrant, Virtualbox, and ansible

https://www.vagrantup.com/docs/

http://docs.ansible.com/ansible/playbooks_intro.html

http://www.ansibletutorials.com/

Ansible courses/tutorials
https://codereviewvideos.com/course/ansible-tutorial/video/looping-in-ansible-with-items

https://metauser.net/2014/07/trying-out-ansible/

https://serversforhackers.com/an-ansible-tutorial
Github: https://github.com/undeadops/ansible-demo

[DEPRECATION WARNING]: Instead of sudo/sudo_user, use become/become_user and
make sure become_method is 'sudo' (default). This feature will be removed in a
future release. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.

http://docs.ansible.com/ansible/become.html

http://stackoverflow.com/questions/29966201/ansible-1-9-1-become-and-sudo-issue

Ansible conditionals
http://stackoverflow.com/questions/32648604/ansible-set-tags-per-item-in-a-loop/32649675

Examples:
https://github.com/henrysher/fedora-infra-ansible/commit/9006a3683242ea9ac92721ea6671f3c59b8e607a

http://docs.ansible.com/ansible/playbooks_loops.html

http://blog.scottlowe.org/2014/09/12/a-quick-introduction-to-vagrant/

http://blog.scottlowe.org/2014/10/22/multi-machine-vagrant-with-yaml/

Monday, March 14, 2016

Random notes: Building a Hadoop test cluster

WARNING: UNDER CONSTRUCTION

We want to install a Hadoop cluster on either Vagrant or a set of remote Linux servers, copy ssh keys to machines to boot strap Ansible on the servers, install Ansible modules, and Ambari Blueprint. This is a parallel bookkeeping problem. The different con files need the same info -the host names and IP addresses.

https://www.vagrantup.com/docs/multi-machine/index.html

https://gist.github.com/ryanwi/9923791

https://sukhjinderkainth.wordpress.com/2015/08/19/create-a-multi-machine-vagrant-config-file/


# -*- mode: ruby -*-
# vi: set ft=ruby :
# inspired from https://gist.github.com/dlutzy/2469037
# inspired from https://gist.github.com/ryanwi/9923791

boxes = [
  { :name => :hmaster, ip: '192.168.33.10', ssh_port: 2223 },
  { :name => :hslave0, ip: '192.168.33.11', ssh_port: 2224 },
  { :name => :hslave1, ip: '192.168.33.12', ssh_port: 2225 },
  { :name => :hslave2, ip: '192.168.33.13', ssh_port: 2226 },
  { :name => :hslave3, ip: '192.168.33.14', ssh_port: 2227 },
  { :name => :hslave4, ip: '192.168.33.15', ssh_port: 2228 },
]

CUSTOM_CONFIG = {
                  "BOX_NAME"  =>  "ubuntu/trusty64",
                  "BOX_URL"   =>  "https://vagrantcloud.com/ubuntu/boxes/trusty64",
                  "HEADLESS"  =>  false
                }

Vagrant.configure("2") do |config|

  # headless?  uncomment this to have the VM's window available
  config.vm.provider :virtualbox do |vb|
    vb.gui = CUSTOM_CONFIG['HEADLESS']
  end

  # Disable default ssh in order to manually assign to a known value
  # https://github.com/mitchellh/vagrant/issues/3232
  config.vm.network :forwarded_port, guest: 22, host: 2222, id: "ssh", disabled: true

  boxes.each do |opts|
    config.vm.define opts[:name] do |boxconfig|
      boxconfig.vm.box        = CUSTOM_CONFIG['BOX_NAME']
      boxconfig.vm.box_url    = CUSTOM_CONFIG['BOX_URL']
      boxconfig.vm.hostname   = "%s.vagrant" % opts[:name].to_s
      boxconfig.vm.network      "private_network", ip: opts[:ip]
      boxconfig.vm.network    :forwarded_port, guest: 22, host: opts[:ssh_port]
    end
  end

  # provisioning with ansible
  # config.vm.provision :ansible do |ansible|
  #   ansible.playbook = "./provisioning/site.yml"
  # end

end

 $ vagrant status
Current machine states:

hmaster                   running (virtualbox)
hslave0                   running (virtualbox)
hslave1                   running (virtualbox)
hslave2                   running (virtualbox)
hslave3                   running (virtualbox)
hslave4                   running (virtualbox)

This environment represents multiple VMs. The VMs are all listed
above with their current state. For more information about a specific
VM, run `vagrant status NAME`.


Now lets add an include file with the server vars to the above Vagrantfile
http://stackoverflow.com/questions/16708917/how-do-i-include-variables-in-my-vagrantfile

Ambari Ansible installation.
http://dataminded.be/blog/how-install-hortonworks-hdp-22-ubuntu-1204

Hortonworks blueprints
https://medium.com/@b23llc/managing-distributed-data-products-with-ansible-and-ambari-44c7175555d8#.ytz5ceqsn

https://cwiki.apache.org/confluence/display/AMBARI/Blueprints#Blueprints-Step0:PrepareAmbariServerandAgents

Blueprint Details
Manually Prepare Ambari Server and Agents
  1. Perform your Ambari Server install and setup.
    yum install ambari-server
    ambari-server setup
  2. After setup completes, start your Ambari Server.
    ambari-server start
  3. Install Ambari Agents on all of the hosts you plan to include in your cluster.
    yum install ambari-agent
  4. Set the Ambari Server on the Ambari Agents.
    vi /etc/ambari-agent/conf/ambari-agent.ini
  5. Set hostname= to the Fully Qualified Domain Name for the Ambari Server. Save and exit.
    hostname=c6401.ambari.apache.org
  6. Start the Agents to initiate registration to Server.
    ambari-agent start
  7. Confirm the Agent hosts are registered with the Server.
    http://your.ambari.server:8080/api/v1/hosts

Generate the ssh key for the Hadoop servers
ssh-keygen

Copy the id_rsa.pub key to the Vagrant folder
cp id_rsa.pub /Vagrant

Script
cat /Vagrant/id_rsa.pub >> `/.ssh/authorized_keys

Copy the ssh id_rsa.pub key to the Vagrant machines
  ssh root@MachineB 'bash -s' < local_script.…


Tuesday, March 8, 2016

digitalocean.com/community/tutorials/how-to-install-and-configure-ansible-on-an-ubuntu-12-04-vps

https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-ansible-on-an-ubuntu-12-04-vps

Random notes: SSH tricks for Vagrant and Ambari

Follow the instructions on the following link-see the note below on logging in as root:
https://ambari.apache.org/1.2.2/installing-hadoop-using-ambari/content/ambari-chap1-5-2.html
https://ambari.apache.org/1.2.2/installing-hadoop-using-ambari/content/ambari-chap1-5-2.html

To login as root on the virtual machine:
vagrant up
vagrant ssh
sudo - root
(password vagrant-change it!!!!)

From another terminal in the Vagrant directory that you launched the VM test the connection to the vagrant VM. Vagrant installs the .ssh/id_rsa.pub to the VM
ssh vagrant@192.168.33.100


https://serversforhackers.com/ssh-tricks/

http://www.cyberciti.biz/faq/linux-unix-applesox-ssh-password-on-command-line/

http://askubuntu.com/questions/46424/adding-ssh-keys-to-authorized-keys

An alternative way to install your public key in the remote machine's
authorized_keys:
cat ~/.ssh/id_rsa.pub | ssh root@HOST "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized
 
chmod 700 ~/.ssh

chmod 600 ~/.ssh/authorized_keys 
 
Restart sshd on the remote host
 
sudo /etc/init.d/sshd restart
 
Stopping sshd:                                             [  OK  ]
Starting sshd:                                             [  OK  ]
 
Now test the passwordless ssh login to root@host 

http://www.cyberciti.biz/tips/linux-multiple-ssh-key-based-authentication.html

https://gist.github.com/jexchan/2351996/

http://www.debian-administration.org/article/152/Password-less_logins_with_OpenSSH

https://www.vagrantup.com/docs/vagrantfile/ssh_settings.html

https://www.racf.bnl.gov/docs/authentication/ssh/sshkeygenunix

Monday, March 7, 2016

Random notes: Installing Ansible in a virtual Python environment on Mac OS X and running Ansible in a VirtualBox and Vagrant env

Download and install Vagrant

http://docs.python-guide.org/en/latest/dev/virtualenvs/

mkdir ansible_ve
cd ansible_ve
virtualenv venv
source venv/bin/activate
sudo easy_install ansible

When you are done working in the virtual environment for the moment, you can deactivate it:

deactivate

UNDER CONSTRUCTION

Thursday, March 3, 2016

Random notes: Linux Mint installation + virtual box, vagrant, Ambari Blueprints, Ansible...


https://www.linuxmint.com/download.php
Use md5

https://www.reddit.com/r/sysadmin/comments/2fmkvq/vagrat_saltstack_ansible_docker_chef_puppet/
A reddit about some of the tools in the rest of this post.

https://www.upguard.com/articles/docker-vs-vagrant

https://www.virtualbox.org/wiki/Linux_Downloads

Use the instructions for: Debian-based Linux distributions
Use sha256sum

Test the virtual box setup by running virtual box from the Linux applications menu
-if you get the vt-x not enabled in the BIOS message
--restart the computer, go into the bios and change the virtual support configuration from disabled to enabled.

SSH
https://serversforhackers.com/ssh-tricks/
scroll down...

 ssh -p 2222 vagrant@vagrant-centos65.vagrantup.com
Last login: Tue Mar  8 17:44:27 2016 from 10.0.2.2


ping 10.0.2.15
PING 10.0.2.15 (10.0.2.15) 56(84) bytes of data.
^C
--- 10.0.2.15 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 3999ms


http://docs-v1.vagrantup.com/v1/docs/host_only_networking.html
http://superuser.com/questions/570261/cant-ping-to-vagrant-box
https://www.vagrantup.com/docs/getting-started/networking.html 
http://stackoverflow.com/questions/23074412/how-to-set-host-key-checking-false-in-ansible-inventory-file

http://superuser.com/questions/570261/cant-ping-to-vagrant-box



Download and verify the sha256 sum.
Then click on the link and Mint will install this using the gedi package installer.

https://programmaticponderings.wordpress.com/2014/12/14/installing-puppet-master-and-agents-on-multiple-vm-using-vagrant-and-virtualbox/

http://www.vagrantbox.es

Vagrantbox.es

Vagrant is an amazing tool for managing virtual machines via a simple to use command line interface. With a simple vagrant up you can be working in a clean environment based on a standard template.
These standard templates are called base boxes, and this website is simply a list of boxes people have been nice enough to make publicly available.
http://superuser.com/questions/671191/how-to-ssh-between-a-cluster-of-vagrant-guest-vms?rq=1

http://leonidmirsky.com/ansible/hadoop/devops/2013/11/19/creating-hadoop-test-environment-with-ansible-and-vagrant.html
Try out a virtual hadoop config.

https://dantehranian.wordpress.com/2015/01/20/ansible-vs-puppet-hands-on-with-ansible/

https://www.ansible.com

Tutorials
https://github.com/leucos/ansible-tuto/tree/master/step-01#inventory
https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-ansible-on-an-ubuntu-12-04-vps

https://www.ansible.com/videos
Watch the quick start video

Playbook use YAML
http://yaml.org
http://docs.ansible.com/ansible/YAMLSyntax.html

Simple ansible command to ping all servers
ansible -m ping -u vagrant all

ansible-playbook .yaml -e "username:vagrant"

command: does not use the shell
shell: uses the shell

http://docs.ansible.com/ansible/playbooks_intro.html

http://jinja.pocoo.org/docs/dev/

http://ryandlane.com/blog/2014/08/04/moving-away-from-puppet-saltstack-or-ansible/

https://galaxy.ansible.com/list#/roles?page=1&page_size=10&autocomplete=hadoop


Wednesday, March 2, 2016

How to create a live bootable USB on a Mac to install Linux on a PC



From http://www.webupd8.org/2009/04/4-ways-to-create-bootable-live-usb.html

Using Diskutil (Mac OS X only)
  1. Download the desired .img file
  2. Open a Terminal (under Utilities)
  3. Run diskutil list to get the current list of devices 
  4. Insert your flash media
  5. Run diskutil list again and determine the device node assigned to your flash media (e.g. /dev/disk2) 
  6. Run diskutil unmountDisk /dev/diskN 
  7. Execute sudo dd if=/path/to/downloaded.img of=/dev/diskN bs=1m 
  8. Run diskutil eject /dev/diskN and remove your flash media when the command completes 
  9. Insert the USB into the PC
  10. Enter the BIOS setup in your PC
  11. Set the USB to boot first
  12. Reboot the computer
  13. Install Linux


Friday, September 11, 2015

Newbie Scala build intro

1. Install Java 8
If you forget to do this you will probably see all sorts of unexplained bugs.

2. Download the Intellij IDE
https://www.jetbrains.com/idea/download/

or the scala-ide based on Eclipse
http://scala-ide.org

Check out the plugins for Intellij
https://www.jetbrains.com/idea/plugins/

Install the JetBrains official Scala plugin

3. Create a directory structure (optionally first create your revision control repo structure and cd to it e.g. git...) using this handy script which has some optional sections commented out.
http://alvinalexander.com/sbtmkdirs

http://grosdim.blogspot.com/2013/01/quick-sbt-tutorial.html

http://www.scala-sbt.org/0.13/tutorial/index.html

4. Optionally install sbteclipse so you can generate an eclipse project if you use scala-ide
https://github.com/typesafehub/sbteclipse/
https://github.com/typesafehub/sbteclipse/wiki/Installing-sbteclipse
emacs ~/.sbt/0.13/plugins/plugins.sbt
(~/.sbt/0.13/plugins probably does not exist and either use emacs to make it or mkdir ~/.sbt/0.13/plugins and then use your favorite editor)

add
addSbtPlugin("com.typesafe.sbteclipse" % "sbteclipse-plugin" % "2.5.0")
"After adding the sbteclipse plugin like this, you should either start sbt or, if it was already started, reload the current session by executing the reload command. If everything worked, you should have the new command eclipse available. Now read on about how to use sbteclipse."

$ sbt
Use of ~/.sbtconfig is deprecated, please migrate global settings to /usr/local/etc/sbtopts
[info] Loading global plugins from /Users/depappas/.sbt/0.13/plugins
[info] Updating {file:/Users/depappas/.sbt/0.13/plugins/}global-plugins...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] downloading https://repo.typesafe.com/typesafe/ivy-releases/com.typesafe.sbteclipse/sbteclipse-plugin/scala_2.10/sbt_0.13/2.5.0/jars/sbteclipse-plugin.jar ...
[info] [SUCCESSFUL ] com.typesafe.sbteclipse#sbteclipse-plugin;2.5.0!sbteclipse-plugin.jar (1808ms)
[info] downloading https://repo.typesafe.com/typesafe/ivy-releases/com.typesafe.sbteclipse/sbteclipse-core/scala_2.10/sbt_0.13/2.5.0/jars/sbteclipse-core.jar ...
[info] [SUCCESSFUL ] com.typesafe.sbteclipse#sbteclipse-core;2.5.0!sbteclipse-core.jar (2512ms)
[info] Done updating.
[info] Set current project to html_product_website_generator (in build file:/Users/depappas/scala_test/html_product_website_generator/)
> eclipse
[info] About to create Eclipse project files for your project(s).
[info] Updating {file:/Users/depappas/scala_test/html_product_website_generator/}html_product_website_generator...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Successfully created Eclipse project files for project(s):
[info] html_product_website_generator
> 


5. Configure scala-ide
http://scala-ide.org/docs/user/gettingstarted.html


5. Open up the set project in Intellij
File->Open
select the project

Or in Eclipse

Once you have installed and generated the Eclipse project files using one of the above plug-ins, start Eclipse. Use File → Import → General/Existing Project into Workspace. Select the directory containing your project as root directory, select the project and hit Finish. And Voila.

6. If you want to use scala.xml and get compile errors try this:

Add this line to the build.sbt file

libraryDependencies += "org.scala-lang.modules" % "scala-xml_2.11" % "1.0.5"                                                                                    
$ sbt
> update
> eclipse
Go to Eclipse and refresh the project
The compile errors in Eclipse should be gone.
If you get this error:
  UNRESOLVED DEPENDENCIES         ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.scala-lang#scala-library;2.11.7 : not found

Then find the right scala.xml version:


Lets find the right version of the scala.xml to add to the build.sb file.


Now lets add it to the build.sbt file.



Lets check in the cache dir:  ~/.ivy2/cache 
for the missing libs





Thursday, September 10, 2015

Upgrading to the Java 8 SDK on a Mac

Things seem to be getting simpler...

Download and install the dmg from Oracle

  • http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html


Read the post installation docs

  • http://docs.oracle.com/javase/8/docs/technotes/guides/install/mac_jdk.html#CHDBADCG


Update the system to use Java 8
$ /usr/libexec/java_home -v 1.8.0_60 --exec javac -version
javac 1.8.0_60

$ which java

/usr/bin/java

Friday, July 31, 2015

Using emacs to paste multiple lines into a rectangle-this does not seem to work in Eclipse block mode

Lets say you have several lines that you want to paste into a rectangle so that the lines are inserted into each block.

case ACCESSORIES           : { }    
case AVAILABILITY          : { } 
case BRANDLOGO             : { } 
case BRANDNAME             : { } 
case BRANDPARTNUMBER       : { } 
case BRANDPRODUCTPAGE      : { } 
case BRANDSUGGESTEDPRICE   : { } 
case BRANDSUPPORTPHONE     : { } 
case BRANDWEBSITE          : { } 


Copy the code to paste into the empty braces, 

Availability schema = SchemaRegistry.getSchemaByClass(Availability.class);
String schemaName = schema.getName();

return schemaName;

Now you want to paste in the following code into each set of braces.
This paste operation does not seem to work in Eclipse. However, in Emacs use insert rectangle (cantle x-r-t) to paste the lines into each set of braces. Use control-space and the arrow keys to select the rectangle inside of the braces. Then use control-Y or command-v on the Mac to paste the contents into the mini buffer and hit return. the multiple lines of text are pasted into each set of braces



case ACCESSORIES           : {
Accessories schema = SchemaRegistry.getSchemaByClass(Accessories.class);
String schemaName = schema.getName();
return schemaName;
 }    
case AVAILABILITY          : {
Accessories schema = SchemaRegistry.getSchemaByClass(Accessories.class);
String schemaName = schema.getName();
return schemaName;
 } 
case BRANDLOGO             : {
Accessories schema = SchemaRegistry.getSchemaByClass(Accessories.class);
String schemaName = schema.getName();
return schemaName;
 } 
case BRANDNAME             : {
Accessories schema = SchemaRegistry.getSchemaByClass(Accessories.class);
String schemaName = schema.getName();
return schemaName;
 } 
case BRANDPARTNUMBER       : {
Accessories schema = SchemaRegistry.getSchemaByClass(Accessories.class);
String schemaName = schema.getName();
return schemaName;
 } 
case BRANDPRODUCTPAGE      : {
Accessories schema = SchemaRegistry.getSchemaByClass(Accessories.class);
String schemaName = schema.getName();
return schemaName;
 } 
} 

Saturday, February 1, 2014

Python3 set operations

how-to-subtraction-two-list-in-python
Python has a built-in datatype for an unordered collection of (hashable) things, called a set. If you convert both lists to sets, the comparison will be unordered.
set(x) == set(y)
Documentation on set

EDIT: @mdwhatcott points out that you want to check for duplicates. set ignores these, so you need a similar data structure that also keeps track of the number of items in each list. This is called a multiset; the best approximation in the standard library is a collections.Counter:
>>> import collections
>>> compare = lambda x, y: collections.Counter(x) == collections.Counter(y)
>>> 
>>> compare([1,2,3], [1,2,3,3])
False
>>> compare([1,2,3], [1,2,3])
True
>>> compare([1,2,3,3], [1,2,2,3])
False
>>> 

best-way-to-check-if-a-list-is-empty
 
if not a:
  print "List is empty"
Using the implicit booleanness of the empty list is quite pythonic.
 

Python3 command line help

use help(dict) on the Python command line and replace dict with the troublesome object to see a list of its helper methods etc., e.g. items(...) D.items() -> list of D's (key, value) pairs, as 2-tuples –


>>> help(list)

Help on class list in module builtins:

class list(object)
 |  list() -> new empty list
 |  list(iterable) -> new list initialized from iterable's items
 | 
 |  Methods defined here:
 | 
 |  __add__(...)
 |      x.__add__(y) <==> x+y
 | 
 |  __contains__(...)
 |      x.__contains__(y) <==> y in x
 | 
 |  __delitem__(...)
 |      x.__delitem__(y) <==> del x[y]
 | 
 |  __eq__(...)
 |      x.__eq__(y) <==> x==y
 | 
 |  __ge__(...)
 |      x.__ge__(y) <==> x>=y
 | 
 |  __getattribute__(...)
 |      x.__getattribute__('name') <==> x.name
 | 
 |  __getitem__(...)
 |      x.__getitem__(y) <==> x[y]
 | 
 |  __gt__(...)
 |      x.__gt__(y) <==> x>y
 | 
 |  __iadd__(...)
 |      x.__iadd__(y) <==> x+=y
 | 
 |  __imul__(...)
 |      x.__imul__(y) <==> x*=y
 | 
 |  __init__(...)
 |      x.__init__(...) initializes x; see help(type(x)) for signature
 | 
 |  __iter__(...)
 |      x.__iter__() <==> iter(x)
 | 
 |  __le__(...)
 |      x.__le__(y) <==> x<=y
 | 
 |  __len__(...)
 |      x.__len__() <==> len(x)
 | 
:

Python3 list operations

how-to-subtraction-two-list-in-python

>>> List1=[3,5,6]
>>> List2=[3,7,2]
>>> [x1 - x2 for (x1, x2) in zip(List1, List2)]
[0, -2, 4]

Python sets
>>> a = [1,2,3,4]
>>> b = [3,4,5,6]
>>> set(a)
{1, 2, 3, 4}
>>> (set(b)
{3, 4, 5, 6}
>>> sa= set(a)
>>> sb= set(b)
>>> sa-sb
{1, 2}
>>> sb-sa
{5, 6}
>>> sa.intersection(sb)
{3, 4}
>>> sa.intersection(sb)
{3, 4}
>>> 3 in sa
True
>>> 10 in sa
False
>>> len(sa)
4
>>> sa.issubset(sb)
False
>>> sa.union(sb)
{1, 2, 3, 4, 5, 6}
>>> sa.difference(sb)
{1, 2}
>>> sb.difference(sa)
{5, 6}
>>> set(a) - set(b)
{1, 2}
>>> set(b) - set(a)
{5, 6}
>>> set(a) - set(a)
set()
>>> sa.symmetric_difference(sb)
{1, 2, 5, 6}
>>> sb.symmetric_difference(sa)
{1, 2, 5, 6}

differences-between-isinstance-and-type-in-python

python: using dictionaries instead of case statements

Switch Statements in Python

response, data = somefunc()
if response == "this":
    do_this_with(data)
elif response == "that":
    do_that_with(data)
elif response == "huh":
    duh(data)
# lots more elifs.
else:
    prevent_horrible_crash(data)


Replace with 

maps-as-switch.py

response_map = {"this": do_this_with,
                "that": do_that_with,
                "huh": duh}
response_map.get(response, prevent_horrible_crash)(data)