Data Loading...
Thomas Uphill, John Arundel, Neependra Khare, Hideto Saito, Hui-Chuan Chloe Lee, Ke-Jou Carol Hsu
DevOps: Puppet, Docker, and Kubernetes Learning Path Get hands-on recipes to automate and manage Linux containers with the Docker 1.6 environment and jump-start your Puppet development
FOR SALE IN INDIA ONLY
DevOps: Puppet, Docker, and Kubernetes Get hands-on recipes to automate and manage Linux containers with the Docker 1.6 environment and jump-start your Puppet development
A course in three modules
BIRMINGHAM - MUMBAI
DevOps: Puppet, Docker, and Kubernetes Copyright © 2017 Packt Publishing
All rights reserved. No part of this course may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this course to ensure the accuracy of the information presented. However, the information contained in this course is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this course. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this course by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Published on: March 2017 Production reference: 1050517
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN: 978-1-78829-761-5 www.packtpub.com
Credits Authors Thomas Uphill
Content Development Editor Juliana Nair
John Arundel Neependra Khare Hideto Saito Hui-Chuan Chloe Lee Ke-Jou Carol Hsu Reviewers Dhruv Ahuja James Fryman Jeroen Hooyberghs Pedro Morgado Scott Collier Julien Duponchelle Allan Espinosa Vishnu Gopal Matt Ma
Graphics Kirk D'Penha Production Coordinator Shantanu N. Zagade
Preface With so many IT management and DevOps tools on the market, both open source and commercial, it's difficult to know where to start. DevOps is incredibly powerful when implemented correctly, here's how to get it done.
What this learning path covers Module 1, Puppet Cookbook (third edition), this module covers all aspects of your puppet infrastructure using simple easy to follow recipes that are independent and can be used to solve real world problems quickly. Puppet Cookbook takes the reader from a basic knowledge of Puppet to a complete and expert understanding of Puppet's latest and most advanced features. With emphasis on real-world implementation, this book delves into various aspects of writing good Puppet code, including using Puppet community style, checking your manifests with puppet-lint and community best practices. It then shows the readers how to set up Puppet for the first time, including instructions on installing Puppet, creating your first manifests, using version control with Puppet and so on. You will also learn to write better manifests, manage resources, files and applications. You'll then be introduced to powerful tools that have grown up around Puppet, including Hiera, Facter, and rspec-puppet. Finally, you will also learn to master common Monitoring, Reporting, and Troubleshooting techniques. Updated with the latest advancements and best practices, this book gives you a clear view on how to "connect the dots" and expands your understanding to successfully use and extend Puppet. Module 2, Docker Cookbook, this module aims to help you get working with Docker by providing you with step-by-step recipes that enable you to effectively deploy Docker in your development, test, and production environments.
i
Preface You will start with verifying and installing Docker on different environments and look into understanding and working with containers and images. Next, you will move on to study the operations related to images. You then proceed to learn about network and data management for containers and how to build an environment for Continuous Integration with the help of services from companies like Shippable and Drone. The book then explores the RESTful APIs provided by Docker to perform different operations like image/container operations before taking a look at the Docker Remote API client. The book ends with a look at logs and troubleshooting Docker to solve issues and bottlenecks. Module 3, Kubernetes Cookbook, Kubernetes is Google's solution to managing a cluster of containers. Kubernetes provides a declarative API to manage clusters while giving us a lot of flexibility. This book will provide you with recipes to better manage containers in different scenarios in production using Kubernetes. We will start by giving you a quick brush up on how Kubernetes works with containers along with an overview of the main Kubernetes features such as Pods, Replication Controllers, and more. Next, we will teach you how to create Kubernetes cluster and how to run programs on Kubernetes. We'll explain features such as High Availability Kubernetes master setup, using Kubernetes with Docker, and orchestration with Kubernetes using AWS. Later, will show you how to use Kubernetes-UI, and how to set up and manage Kubernetes clusters on the cloud and bare metal. Upon completion of this book, you will be able use Kubernetes in production and will have a better understanding of how to manage your containers using Kubernetes.
What you need for this learning path The primary softwares required are as follows:
Puppet 3.7.3
Kubernetes 1.1.3Java
Etcd 2.1.1
Flanneld 0.5.2
Docker 1.7.1
Kubernetes 1.2.2
Etcd 2.3.1
Amazon Web Services
ii
Preface
entOS
7.1/ubuntu
14.04/Amazeon
Linux 2015.09
Debian and Enterprise Linux-based distributions
Who this learning path is for This Learning Path is for developers, system administrators, and DevOps engineers who want to use Puppet, Docker, and Kubernetes in their development, QA, or production environments. This Learning Path assumes experience with Linux administration and requires some experience with command-line usage and basic text file editing.
Reader feedback Feedback from our readers is always welcome. Let us know what you think about this course— what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of. To send us general feedback, simply e-mail [email protected], and mention the course's title in the subject of your message. If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Customer support Now that you are the proud owner of a Packt course, we have a number of things to help you to get the most from your purchase.
Downloading the example code You can download the example code files for this course from your account at http:// www.packtpub.com. If you purchased this course elsewhere, you can visit http://www. packtpub.com/support and register to have the files e-mailed directly to you.
iii
Preface You can download the code files by following these steps: 1. Log in or register to our website using your e-mail address and password. 2. Hover the mouse pointer on the SUPPORT tab at the top. 3. Click on Code Downloads & Errata. 4. Enter the name of the course in the Search box. 5. Select the course for which you're looking to download the code files. 6. Choose from the drop-down menu where you purchased this course from. 7.
Click on Code Download.
You can also download the code files by clicking on the Code Files button on the course's webpage at the Packt Publishing website. This page can be accessed by entering the course's name in the Search box. Please note that you need to be logged in to your Packt account. Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR / 7-Zip for Windows
Zipeg / iZip / UnRarX for Mac
7-Zip / PeaZip for Linux
The code bundle for the course is also hosted on GitHub at https://github.com/ PacktPublishing/DevOps-Puppet-Docker-and-Kubernetes. We also have other code bundles from our rich catalog of books, videos, and courses available at https:// github.com/PacktPublishing/. Check them out!
Errata Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our courses—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this course. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your course, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title. To view the previously submitted errata, go to https://www.packtpub.com/books/ content/support and enter the name of the course in the search field. The required information will appear under the Errata section.
iv
Preface
Piracy Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy. Please contact us at [email protected] with a link to the suspected pirated material. We appreciate your help in protecting our authors and our ability to bring you valuable content.
Questions If you have a problem with any aspect of this course, you can contact us at questions@ packtpub.com, and we will do our best to address the problem.
v
Module 1: Puppet Cookbook Chapter 1: Puppet Language and Style
3
Introduction Adding a resource to a node Using Facter to describe a node Installing a package before starting a service Installing, configuring, and starting a service Using community Puppet style Creating a manifest Checking your manifests with Puppet-lint Using modules Using standard naming conventions Using inline templates Iterating over multiple items Writing powerful conditional statements Using regular expressions in if statements Using selectors and case statements Using the in operator Using regular expression substitutions Using the future parser
4 4 5 6 8 10 13 15 17 22 24 25 28 30 32 35 36 38
Chapter 2: Puppet Infrastructure
43
Introduction Installing Puppet Managing your manifests with Git Creating a decentralized Puppet architecture
44 44 45 51 i
Table of Contents
Writing a papply script Running Puppet from cron Bootstrapping Puppet with bash Creating a centralized Puppet infrastructure Creating certificates with multiple DNS names Running Puppet from passenger Setting up the environment Configuring PuppetDB Configuring Hiera Setting node-specific data with Hiera Storing secret data with hiera-gpg Using MessagePack serialization Automatic syntax checking with Git hooks Pushing code around with Git Managing Environments with Git
Chapter 3: Writing Better Manifests Introduction Using arrays of resources Using resource defaults Using defined types Using tags Using run stages Using roles and profiles Passing parameters to classes Passing parameters from Hiera Writing reusable, cross-platform manifests Getting information about the environment Importing dynamic information Passing arguments to shell commands
Chapter 4: Working with Files and Packages Introduction Making quick edits to config files Editing INI style files with puppetlabs-inifile Using Augeas to reliably edit config files Building config files using snippets Using ERB templates Using array iteration in templates Using EPP templates
ii
54 57 60 63 65 66 69 72 73 76 77 79 80 82 85
89 90 90 91 94 97 100 104 106 108 109 112 114 116
119 120 120 123 126 128 130 132 135
Table of Contents
Using GnuPG to encrypt secrets Installing packages from a third-party repository Comparing package versions
Chapter 5: Users and Virtual Resources Introduction Using virtual resources Managing users with virtual resources Managing users' SSH access Managing users' customization files Using exported resources
Chapter 6: Managing Resources and Files Introduction Distributing cron jobs efficiently Scheduling when resources are applied Using host resources Using exported host resources Using multiple file sources Distributing and merging directory trees Cleaning up old files Auditing resources Temporarily disabling resources
Chapter 7: Managing Applications Introduction Using public modules Managing Apache servers Creating Apache virtual hosts Creating nginx virtual hosts Managing MySQL Creating databases and users
Chapter 8: Internode Coordination Introduction Managing firewalls with iptables Building high-availability services using Heartbeat Managing NFS servers and file shares Using HAProxy to load-balance multiple web servers Managing Docker with Puppet
136 142 145
149 149 150 154 157 161 164
169 170 170 174 177 178 181 184 188 190 191
195 195 196 198 200 204 207 209
213 213 214 220 227 236 242
iii
Table of Contents
Chapter 9: External Tools and the Puppet Ecosystem
247
Introduction Creating custom facts Adding external facts Setting facts as environment variables Generating manifests with the Puppet resource command Generating manifests with other tools Using an external node classifier Creating your own resource types Creating your own providers Creating custom functions Testing your puppet manifests with rspec-puppet Using librarian-puppet Using r10k
248 248 251 254 255 257 261 264 267 269 273 278 280
Chapter 10: Monitoring, Reporting, and Troubleshooting
285
Introduction Noop – the don't change anything option Logging command output Logging debug messages Generating reports Producing automatic HTML documentation Drawing dependency graphs Understanding Puppet errors Inspecting configuration settings
285 286 289 291 293 295 298 303 306
Module 2: Docker Cookbook Chapter 1: Introduction and Installation Introduction Verifying the requirements for Docker installation Installing Docker Pulling an image and running a container Adding a nonroot user to administer Docker Setting up the Docker host with Docker Machine Finding help with the Docker command line
Chapter 2: Working with Docker Containers Introduction Listing/searching for an image Pulling an image iv
311 311 319 320 321 324 325 328
329 330 330 332
Table of Contents
Listing images Starting a container Listing containers Looking at the logs of containers Stopping a container Deleting a container Setting the restart policy on a container Getting privileged access inside a container Exposing a port while starting a container Accessing the host device inside the container Injecting a new process to a running container Returning low-level information about a container Labeling and filtering containers
Chapter 3: Working with Docker Images Introduction Creating an account with Docker Hub Creating an image from the container Publishing an image to the registry Looking at the history of an image Deleting an image Exporting an image Importing an image Building images using Dockerfiles Building an Apache image – a Dockerfile example Accessing Firefox from a container – a Dockerfile example Building a WordPress image – a Dockerfile example Setting up a private index/registry Automated builds – with GitHub and Bitbucket Creating the base image – using supermin Creating the base image – using Debootstrap Visualizing dependencies between layers
Chapter 4: Network and Data Management for Containers Introduction Accessing containers from outside Managing data in containers Linking two or more containers Developing a LAMP application by linking containers Networking of multihost containers with Flannel Assigning IPv6 addresses to containers
334 335 338 339 340 341 343 344 345 346 347 348 350
353 354 355 356 358 360 361 363 364 364 370 373 377 382 386 389 391 392
393 393 398 400 404 406 408 413
v
DevOps: Puppet, Docker, and Kubernetes With so many IT management and DevOps tools on the market, both open source and commercial, it’s difficult to know where to start. DevOps is incredibly powerful when implemented correctly, and here’s how to get it done. This Learning Path covers three broad areas: Puppet, Docker, and Kubernetes.
Things you will learn:
This Learning Path is a large resource of recipes to ease your daily DevOps tasks. We begin with recipes that help you develop a complete and expert understanding of Puppet’s latest and most advanced features. Then we provide recipes that help you efficiently work with the Docker environment.
• Create services with Docker to
• Discover how to build high availability Kubernetes clusters
• Deal with inherent issues with container virtualization and container concepts
enable swift development and deployment of applications
• Make optimum use of Docker in a testing environment
• Create efficient manifests to Finally, we show you how to better manage containers in different scenarios in production using Kubernetes.This course is based on these books Puppet Cookbook, Third Edition, Docker Cookbook and Kubernetes Cookbook.
www.packtpub.com
streamline your deployments
• Automate Puppet master deployment using Git hooks, r10k, and PuppetDB
FOR SALE IN INDIA ONLY