Cleaning/Updating k8petstore dev tooling and build fixing, moving webserver to

examples/k8petstore/README.md

@@ -69,19 +69,31 @@ For a list of those images, see the `build-and-push` shell script - it builds an
 
 modify the dockerhub user name in it accordingly.
 
-## Get started with the WEBAPP
+## Hacking, extending, and locally testing on the k8petstore
 
 The web app is written in Go, and borrowed from the original Guestbook example by brendan burns.
 
+K8petstore is built to be expanded, and aims to attract developers interested in building and maintaining a polyglot, non-trivial kubernetes app as a community.
+
+It can be a simple way to get started with kuberentes or golang application development.
+
+Thus we've tried to make it easy to hack on, even without kubernetes.  Just run the containers and glue them together using docker IP addresses !
+
 We have extended it to do some error reporting, persisting of JSON petstore transactions (not much different then guestbook entries),
 
 and supporting of additional REST calls, like LLEN, which returns the total # of transactions in the database.
 
-To work on the app, just cd to the `dev` directory, and follow the instructions.  You can easily edit it in your local machine, by installing
+If that is all working, you can finally run `k8petstore.sh` in any Kubernetes cluster, and run the app at scale.
+
+### MAC USERS
 
-redis and go.  Then you can use the `Vagrantfile` in this top level directory to launch a minimal version of the app in pure docker containers.
+To develop against k8petstore, simply run the docker-machine-dev.sh script, which is built for mac users.
 
-If that is all working, you can finally run `k8petstore.sh` in any Kubernetes cluster, and run the app at scale.
+### LINUX USERS
+
+For now, modify the docker-machine-dev.sh script as necessary to use the provider of your choice.  Most linux/docker users are savvy enough to do this easily.
+
+If you need help, just ask on the mailing list.
 
 ## Set up the data generator (optional)
 
@@ -125,15 +137,27 @@ You might want to change it to point to your customized Go image, if you chose t
 
 So, to run this app in Kubernetes, simply run [The all in one k8petstore.sh shell script](k8petstore.sh).
 
-Note that at the top of the script there are a few self explanatory parameters to set, among which the Public IPs parameter is where you can checkout the web ui (at $PUBLIC_IP:3000), which will show a plot and read outs of transaction throughput.
+## Should we use PublicIP, NodePort, Cloud loadbalancers ?
 
-In the mean time, because the public IP will be deprecated in Kubernetes v1, we provide other 2 scripts k8petstore-loadbalancer.sh and k8petstore-nodeport.sh. As the names suggest, they rely on LoadBalancer and NodePort respectively. More details can be found [here](../../docs/user-guide/services.md#external-services).
+The original k8petstore used PUBLIC_IP fields to bind the web app to an IP.
+
+However... because the public IP was deprecated in Kubernetes v1, we provide other 2 scripts k8petstore-loadbalancer.sh and k8petstore-nodeport.sh. As the names suggest, they rely on LoadBalancer and NodePort respectively. More details can be found [here](../../docs/user-guide/services.md#external-services).
+
+We will continue to try to update k8petstore to use the idiomatic networking tools that kubernetes supports, if we fall behind, please create an issue !
 
 ## Future
 
-In the future, we plan to add cassandra support.  Redis is a fabulous in memory data store, but it is not meant for truly available and resilient storage.
+Future development ideas include, adding a persistent k/v store like cassandra/hbase/..., using kafka/activeMQ for the data sink (with redis as a consumer).
+
+Additionally, adding analytics and meaningful streaming queries to the richly patterned data would also be interesting.
+
+We are open to other ways of expanding the coverage and realism of the k8petstore application.
 
-Thus we plan to add another tier of queueing, which empties the REDIS transactions into a cassandra store which persists.
+Reach out with ideas, pull requests, and so on!
+
+The end goal is to support polyglot, real world, data-intensive application on kuberenetes which can be used both to learn how to maintain kubernetes applications
+
+as well as for scale and functionality testing.
 
 ## Questions
 
@@ -141,7 +165,6 @@ For questions on running this app, you can ask on [Slack](../../docs/troubleshoo
 
 For questions about bigpetstore, and how the data is generated, ask on the apache bigtop mailing list.
 
-
 <!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
 [![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/examples/k8petstore/README.md?pixel)]()
 <!-- END MUNGE: GENERATED_ANALYTICS -->

examples/k8petstore/Vagrantfile

-# -*- mode: ruby -*-
-# vi: set ft=ruby :
-
-require 'fileutils'
-
-#$fes = 1
-#$rslavess = 1 
-
-Vagrant.configure("2") do |config|
-
-    config.vm.define "rmaster" do |rm|     
-        rm.vm.provider "docker" do |d|
-            d.vagrant_vagrantfile = "./dev/hosts/Vagrantfile"  
-            d.build_dir = "redis-master"
-            d.name = "rmaster"
-            d.create_args = ["--privileged=true", "-m", "1g"]
-            #d.ports = [ "6379:6379" ]
-            d.remains_running = true
-        end
-    end
-
-    config.vm.define "frontend" do |fe|
-        fe.vm.provider "docker" do |d|
-            d.vagrant_vagrantfile = "./dev/hosts/Vagrantfile"
-            d.build_dir = "web-server"
-            d.name = "web-server"
-            d.create_args = ["--privileged=true"]
-            d.remains_running = true
-            d.create_args = d.create_args << "--link" << "rmaster:rmaster"
-            d.ports = ["3000:3000"]
-            d.env = {"REDISMASTER_SERVICE_HOST"=>"rmaster","REDISMASTER_SERVICE_PORT"=>"6379"} 
-        end
-    end
- 
-    ### Todo , add data generator.
-
-end

examples/k8petstore/build-push-containers.sh

@@ -27,3 +27,6 @@ docker push jayunit100/k8-petstore-redis:$version
 docker push jayunit100/k8-petstore-redis-master:$version
 docker push jayunit100/k8-petstore-redis-slave:$version
 docker push jayunit100/k8-petstore-web-server:$version
+
+### Now, start the application.
+

examples/k8petstore/dev/README

-### Local development 
-
-1) Install Go
-
-2) Install Redis
-
-Now start a local redis instance 
-
-```sh
-redis-server
-```
-
-And run the app 
-
-```sh
-export GOPATH=~/Development/k8hacking/k8petstore/web-server/
-cd $GOPATH/src/main/
-## Now, you're in the local dir to run the app.  Go get its depenedencies.
-go get
-go run PetStoreBook.go
-```
-
-Once the app works the way you want it to, test it in the vagrant recipe below.  This will guarantee that you're local environment isn't doing something that breaks the containers at the  versioning level.
-
-### Testing 
-
-This folder can be used by anyone interested in building and developing the k8petstore application.
-
-This is for dev and test.
-
-`vagrant up` gets you a cluster with the app's core components running.
-
-You can rename Vagrantfile_atomic to Vagrantfile if you want to try to test in atomic instead.
-
-** Now you can run the code on the kubernetes cluster with reasonable assurance that any problems you run into are not bugs in the code itself :) *

examples/k8petstore/dev/Vagrantfile

-# -*- mode: ruby -*-
-# vi: set ft=ruby :
-
-require 'fileutils'
-
-#$fes = 1
-#$rslavess = 1 
-
-Vagrant.configure("2") do |config|
-
-    config.vm.define "rmaster" do |rm|     
-        rm.vm.provider "docker" do |d|
-            d.vagrant_vagrantfile = "./hosts/Vagrantfile"  
-            d.build_dir = "../redis-master"
-            d.name = "rmaster"
-            d.create_args = ["--privileged=true"]
-            #d.ports = [ "6379:6379" ]
-            d.remains_running = true
-        end
-    end
-   
-   puts "sleep 20 to make sure container is up..."
-   sleep(20)    
-   puts "resume"
-
-    config.vm.define "frontend" do |fe|
-        fe.vm.provider "docker" do |d|
-            d.vagrant_vagrantfile = "./hosts/Vagrantfile"
-            d.build_dir = "../web-server"
-            d.name = "web-server"
-            d.create_args = ["--privileged=true"]
-            d.remains_running = true
-            d.create_args = d.create_args << "--link" << "rmaster:rmaster"
-            d.ports = ["3000:3000"]
-            d.env = {"REDISMASTER_SERVICE_HOST"=>"rmaster","REDISMASTER_SERVICE_PORT"=>"6379"} 
-        end
-    end
-
-   
- 
-    ### Todo , add data generator.
-
-
-end

examples/k8petstore/dev/hosts/Vagrantfile

-VAGRANTFILE_API_VERSION = "2"
-
-Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
-  config.vm.box = "jayunit100/centos7"
-  config.vm.provision "docker"
-  config.vm.provision "shell", inline: "ps aux | grep 'sshd:' | awk '{print $2}' | xargs kill"
-  config.vm.provision "shell", inline: "yum install -y git && service firewalld stop && service docker restart"  
-  config.vm.provision "shell", inline: "docker ps -a | awk '{print $1}' | xargs --no-run-if-empty docker rm -f || ls"
-  config.vm.network :forwarded_port, guest: 3000, host: 3000
-
-end

examples/k8petstore/dev/test.sh

-#!/bin/bash
-
-# Copyright 2015 The Kubernetes Authors All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-## First set up the host VM.  That ensures
-## we avoid vagrant race conditions.
-set -x 
-
-cd hosts/ 
-echo "note: the VM must be running before you try this"
-echo "if not already running, cd to hosts and run vagrant up"
-vagrant provision
-#echo "removing containers"
-#vagrant ssh -c "sudo docker rm -f $(docker ps -a -q)"
-cd ..
-
-## Now spin up the docker containers
-## these will run in the ^ host vm above.
-
-vagrant up
-
-## Finally, curl the length, it should be 3 .
-
-x=`curl localhost:3000/llen`
-
-for i in `seq 1 100` do
-    if [ x$x == "x3" ]; then 
-       echo " passed $3 "
-       exit 0
-    else
-       echo " FAIL" 
-    fi
-done
-
-exit 1 # if we get here the test obviously failed.

examples/k8petstore/docker-machine-dev.sh

+#!/bin/bash
+
+# Copyright 2015 The Kubernetes Authors All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+!/bin/bash
+
+function setup_vm() {
+    ### Provider = vbox.  You can use another one if you want... But untested.
+    PROVIDER=virtualbox
+
+    ### Create a VM specific to this app...
+    if docker-machine ls |grep -q k8petstore ; then 
+        echo "VM already exists, moving on..."
+    else
+        docker-machine create --driver $PROVIDER k8petstore
+    fi
+}
+
+function setup_docker() {
+
+    ## Set the docker server, and then clean all containers... 
+    eval "$(docker-machine env k8petstore)"
+    docker rm -f `docker ps -a -q`
+
+    ### Now capture the IP.
+    MACHINE_IP="`docker-machine ip k8petstore`"
+
+}
+
+function build_containers() {
+
+    version="`date +"%m-%d-%Y-%s"`"
+    pushd ../redis
+    docker build -t jayunit100/k8-petstore-redis:$version ./
+    popd
+
+    pushd ../redis-master
+    docker build -t jayunit100/k8-petstore-redis-master:$version ./    
+    popd
+    
+    pushd ../redis-slave
+    docker build -t jayunit100/k8-petstore-redis-slave:$version ./
+    popd
+    
+    pushd ../web-server
+    docker build -t jayunit100/k8-petstore-web-server:$version ./
+    popd
+}
+
+function runk8petstore() {
+
+    ### Finally, run the application.
+    ### This app is gauranteed to be a clean run using all the source.
+    ### You can use it to iteratively test/deploy k8petstore and make new changes.
+
+    ### TODO, add slaves.
+
+    echo "Running k8petstore now..."
+    docker run -d -p 6379:6379 jayunit100/k8-petstore-redis-master:$version
+    docker run -d -e REDISMASTER_SERVICE_HOST=$MACHINE_IP -e REDISMASTER_SERVICE_PORT=6379 -p 3000:3000 jayunit100/k8-petstore-web-server:$version
+    
+}
+
+setup_vm
+
+setup_docker
+
+build_containers
+
+runk8petstore

examples/k8petstore/web-server/Dockerfile

-FROM google/golang:latest
+FROM golang:latest
 
 # Add source to gopath.  This is defacto required for go apps.
-ADD ./src /gopath/src/
+ADD ./src /gopath/src/k8petstore
+RUN mkdir /gopath/bin/
 ADD ./static /tmp/static
 ADD ./test.sh /opt/test.sh
 RUN chmod 777 /opt/test.sh
-# $GOPATH/[src/a/b/c]
-# go build a/b/c
-# go run main
 
 # So that we can easily run and install
-WORKDIR /gopath/src/
+WORKDIR /gopath/src
 
 # Install the code (the executables are in the main dir)  This will get the deps also.
-RUN go get main
-#RUN go build main
+RUN export GOPATH=/gopath/ && go get k8petstore
+RUN export GOPATH=/gopath/ && go install k8petstore
+
 
 # Expected that you will override this in production kubernetes.
 ENV STATIC_FILES /tmp/static
-CMD /gopath/bin/main
+CMD /gopath/bin/k8petstore

hack/lib/golang.sh

@@ -70,7 +70,7 @@ kube::golang::test_targets() {
     cmd/genconversion
     cmd/gendeepcopy
     cmd/genswaggertypedocs
-    examples/k8petstore/web-server
+    examples/k8petstore/web-server/src
     github.com/onsi/ginkgo/ginkgo
     test/e2e/e2e.test
   )