Running Your Services On Docker An experience report
Running Your Services On Docker An experience report www.synerzip.com Webinar Series 2015 Who Am I? Robert Bastian Director, Platform and Architecture at Drillinginfo 20+ years industry experience in Telcos, Gaming and Energy I love APIs and services! Agile and DevOps advocate
www.synerzip.com Webinar Series 2015 2 Why Docker? My World Needed To Change 5+ individual teams building micro services in Java and Scala Frictionless deployment of micro-services using Chef & AWS 25+ separate micro-services deployed in the previous 18 months Each service is typically deployed to a single AWS virtual machine Each service is deployed 6x - dev, test, staging (2x) and production (2x)
25+ micro-services became nearly 150 AWS virtual machines www.synerzip.com Webinar Series 2015 3 Why Docker? COST! The AWS bill is too damn high! Decline in the global price of oil causing churn in our business 6 AWS virtual machines per service isnt sustainable with our budget AWS monthly bill started to gain visibility from sr. management and the board www.synerzip.com
Webinar Series 2015 4 Why Docker? WASTE! We werent using the compute and memory resources purchased from AMZN! Nearly all micro-services were at 1% CPU utilization Nearly all micro-services were only using 40% of memory (JVM) 150+ virtual machines essentially sitting idle www.synerzip.com Webinar Series 2015
5 Why Docker? LOCK IN! How would we leave AMZN if we wanted to? Could we use Drillinginfo ITs Openstack platform? What about alternate IaaS providers like Rackspace or Azure? What about Container as a Service (CaaS) providers like Joyent, Tutum or Profitbricks? What about using Amazons Container Service? www.synerzip.com Webinar Series 2015
6 My World Needs To Change - Problem Statement How can we deploy fewer virtual machines while increasing the density and utilization of services per machine without locking us into a specific IaaS provider? www.synerzip.com Webinar Series 2015 7
How Docker Solves All The Problems www.synerzip.com Webinar Series 2015 8 Docker Containers - Shipping Matrix From Hell www.synerzip.com Webinar Series 2015
9 Docker Containers - Standard Shipping Container www.synerzip.com Webinar Series 2015 10 Whats Inside Doesnt Matter www.synerzip.com
Webinar Series 2015 11 Why Docker Is Important - Before Containers Very inefficient use of memory and CPU resources www.synerzip.com Webinar Series 2015 12
Why Docker Is Important - After Containers Isolated services in fewer VMs... www.synerzip.com and use VMs more efficiently. Webinar Series 2015
13 Why Is Docker Important? Docker container technology provides our micro-services platform: Increased density of isolated micro-services per virtual machine (9:1!) Containerized micro-services are portable across machines and providers Containerized micro-services are much faster than virtual machines www.synerzip.com Webinar Series 2015 14
Containers Alone Arent Enough www.synerzip.com Webinar Series 2015 15 But Containers Arent Enough! Running containerized micro-services in production requires much more than just Docker. It requires a Platform that can do the following: Building and pushing Docker images to an image repository Pulling images, provisioning and scheduling containers
Discovering and binding to services running as containers Containers discovering and binding to other containers Operating and managing services in containers www.synerzip.com Webinar Series 2015 16 Drillinginfo Docker Platform: Build & Store Images Problem: Detect changes at Github and build a new Docker
image www.synerzip.com Webinar Series 2015 Problem: Where do we store our Docker images? 17
Drillinginfo Docker Platform: Jenkins & Dockerhub Problem: How do we build images? Jenkins automates the image builds. We started building our images with Ubuntu 14.04 (1GB) We settled on Alpine, a minimal linux distribution (5MB) Typical micro-services now ~ 390MB Problem: Where do we put them? Dockerhub. Tried Docker Trusted Registry and Core OS Enterprise Registry Settled on using Dockerhub Use latest and sem-ver tags on our images www.synerzip.com Webinar Series 2015 18
Drillinginfo Docker Platform: Provisioning, Scheduling Problem: Which host do the containers run on? Problem: How are containers started and configured? Dockerhub
www.synerzip.com Webinar Series 2015 19 Drillinginfo Docker Platform - Chef Problem: How do we determine which host to run a container on and how do we configure and start the container? We solve scheduling and provisioning with Chef. Chef schedules containers on specific hosts using Chef roles Chef provisions and configures containers using Chef recipes and environments
Each micro-service has an associated Chef recipe that converts Chef attributes into container environment variables www.synerzip.com Webinar Series 2015 20 Drillinginfo Docker Platform: Service Directory DI Web Applications www.synerzip.com
Problem: How can web applications discover and bind to containers? Webinar Series 2015 DI Docker Containers 21 Drillinginfo Docker Platform - Consul
Problem: How do our browser applications locate service containers? We use Hashicorps Consul as our service directory. Containers automatically register themselves with Consul when started. The Docker daemon emits real-time lifecycle events for container start We use a utility container called Registrator to automate the registration of micro-service containers with Consul Containers are registered with a health check that Consul polls to determine the health of the container www.synerzip.com Webinar Series 2015 22
Drillinginfo Docker Platform: Service Discovery Problem: How can web applications discover and bind to containers? www.synerzip.com Webinar Series 2015 23 Drillinginfo Docker Platform - Consul Template
Problem: How do our browser applications use services deployed in containers? We use Hashicorps Consul Template for service discovery and Varnish for load balancing. Consul Template detects containers in Consul and updates Varnish configuration Consul Template participates in the Consul cluster using Consul Client Consul Template automatically adds healthy containers and removes sick containers from the Varnish load balancer by updating Varnish configuration Browser applications use Varnish routes to reach services running in containers www.synerzip.com Webinar Series 2015 24
Drillinginfo Docker Platform: Container Dependencies Problem: How can containers discover and bind to other containers? www.synerzip.com Webinar Series 2015 25
Drillinginfo Docker Platform - Service Proxy Problem: How can containers find their containerized dependencies on the same host and different hosts? We use Consul, Nginx and Consul Template to implement a Service Proxy for inter and intra-host container communication. We built a utility container called Service Proxy that uses Consuls service directory to locate a container's ip address and port Service Proxy then uses Consul Template to create an nginx.conf with load balanced routes for each service container Docker Links work for intra-host dependencies but with a gotcha www.synerzip.com Webinar Series 2015
26 Drillinginfo Docker Platform: Operations & Monitoring Problem: How do we detect failed or failing containers? www.synerzip.com Webinar Series 2015 27
Drillinginfo Docker Platform - Operations & Monitoring Problem: How do we monitor containers and notify and escalate when containerized services arent healthy? We use Uptime and VictorOps monitor our containerized services. A utility container monitors Docker container lifecycle events and automatically registers a service check with Uptime when a container starts Uptime service interruptions to VictorOps for on-call scheduling, paging and escalation www.synerzip.com Webinar Series 2015
28 Drillinginfo Docker Platform: Operations & Monitoring Problem: How do we monitor the resource usage of hosts and containers? www.synerzip.com Webinar Series 2015
29 Drillinginfo Docker Platform - Operations & Monitoring Problem: How do we monitor our Docker hosts resource usage? We use Datadog to monitor the Docker host utilization and the services metrics. Datadog helps us visualize the resource usage on a host Datadog helps us understand how our services are performing Datadog helps us understand how to pack containers onto hosts by exposing the current utilization of CPU and memory resources on the host www.synerzip.com Webinar Series 2015
30 Drillinginfo Docker Platform - Overview UPTIME www.synerzip.com Webinar Series 2015 31 Drillinginfo Docker Platform - Wrap Up The Docker container technology and the Drillinginfo Docker Platform provide our micro-services infrastructure the following benefits:
Reduced cost for IaaS hosting Reduced waste of virtual machine resources Standardized deployment mechanism for micro-services Standardized service directory, service discovery Standardized metrics dashboards, monitoring and alerting www.synerzip.com Webinar Series 2015 32 Drillinginfo Docker Platform - Future Chef has gotten us where we are today but not where we want to be. Container orchestration
Host provisioning and pooling www.synerzip.com Webinar Series 2015 33 Drillinginfo Docker Platform - Orchestration Docker Compose will replace Chef roles defining the micro-services deployed on our platform and which Docker host they run on. The Docker Compose YAML file: Defines which containerized micro-services run on which host Define the environment variables for each container
I believe that IaaS providers will standardize on Docker Compose for container orchestration. www.synerzip.com Webinar Series 2015 34 Drillinginfo Docker Platform - Provisioning & Pooling Docker Machine will replace Chef for provisioning virtual machines with Docker. Docker Machine automates the provisioning of Docker hosts Docker Swarm will replace Chef for scheduling containers on a host. Swarm combines Docker Machines into a single pool of compute and memory
resources Swarm provides container scheduling and supports plug-in schedulers Docker Compose will define all the containers that run on the Swarm www.synerzip.com Webinar Series 2015 35 Running Your Services On Docker: Thank You! Questions? www.synerzip.com
Webinar Series 2015 36 Contact Info Please feel free to contact me with any additional questions or comments! Email: [email protected] LinkedIn: rbastian Twitter: @rbastian www.synerzip.com Webinar Series 2015
37 Running Your Services On Docker - Links https://www.docker.com/ https://www.varnish-cache.org/ https://hub.docker.com/ https://www.nginx.com/ https://jenkins-ci.org/
Webinar Series 2015 38 www.synerzip.com Ashish Shanker [email protected] 469.374.0500 www.synerzip.com Webinar Series 2015 39
3 9 Synerzip in a Nutshell Software product development partner for small/mid-sized technology companies Exclusive focus on small/mid-sized technology companies, typically venture-backed companies in growth phase By definition, all Synerzip work is the IP of its respective clients Deep experience in full SDLC design, dev, QA/testing, deployment Dedicated team of high caliber software professionals for each client
Seamlessly extends clients local team offering full transparency Stable teams with very low turn-over NOT just staff augmentation, but provide full management support Actually reduces risk of development/delivery Experienced team uses appropriate level of engineering discipline Practices Agile development responsive yet disciplined Reduces cost dual-site team, 50% cost advantage Offers long-term flexibility allows (facilitates) taking offshore team captive aka
BOT option www.synerzip.com Webinar Series 2015 40 Synerzip Clients www.synerzip.com Webinar Series 2015
41 Next Webinar Role of the Architect in Agile Complimentary Webinar: Thursday, November 12, 2015 @ Noon CST Presented by: Chris Edwards, P.Eng Software Manager, IHS Inc. www.synerzip.com Webinar Series 2015
42 Connect with Synerzip @Synerzip linkedin.com/company/synerzip facebook.com/Synerzip Ashish Shanker [email protected] 469.374.0500 www.synerzip.com Webinar Series 2015
43 Running Your Services On Docker: Thank You! Questions? www.synerzip.com Webinar Series 2015 44
Understand seasonality (Harvest Festival) Bake bread Shadow puppets Computing Programming A and B Y3 - Unit 3.5 Controlling on screen models, Y3 - Unit 3..6 Repeat forever Iteration Y4 - Unit 4.5 Controlling on screen models Y4 - Unit 4.6...
Different sizes will require different design elements. Proportion - (measured by height of a human being) Balance - equal distribution of emphasis. Mass - bulk and weight. 2. What is the setting of the performance? Period, location, etc.
The thrust is inversely proportional to the sag; halving the sag doubles the thrust. This raises an interesting question of economy through. OPTIMAL SAG : A large sag increases the cable length, but reduces the tensile force & allows a...
Work In Progress: Post July 2016 payroll closedown, any data changes / employee change data not actioned by the current employer / payroll provider to be forwarded to St Helens & Knowsley NHS Trust - Single Lead Employer for action...
William R. Watts Foundation, Inc. Zale Foundation. Gateway recognizes the following Corporations, Foundations and Organizations that have contributed to the success of Gateway in 2013- 2014. We apologize for any omissions. A Warm Thanks To Our Donors
Times Monotype Sorts Symbol Arial 02 measurement.reliability Basic Elements of a Graph Multiple Plot, More than One Set of Data Multiple Plot, Double-Y Axis (Used when data represent different units of measurement) Bar Graph (Used for Plotting Discontinuous Data) Frequency...
The capital charge comes as a protection against unexpected losses: Capital = EV - P(p) P% P(p) Economic Capital EL FV EV Default Only/ "Fair Value" CR+ Internal CPV KMV CMs Public Credit rating of counter-party Credit rating of underlying...
Ready to download the document? Go ahead and hit continue!