Intelligent Automation with VMware Apply machine learning techniques to VMware virtualization and networking

Ajit Pratap Kundan

www.packt.com

FOR SALE IN INDIA ONLY

Intelligent Automation with VMware Apply machine learning techniques to VMware virtualization and networking

Ajit Pratap Kundan

BIRMINGHAM - MUMBAI

Intelligent Automation with VMware Copyright © 2019 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Commissioning Editor: Amey Varangaonkar Acquisition Editor: Reshma Raman Content Development Editor: Kirk Dsouza Technical Editor: Jovita Alva Copy Editor: Safis Editing Project Coordinator: Namrata Swetta Proofreader: Safis Editing Indexer: Pratik Shirodkar Graphics: Alishon Mendonsa Production Coordinator: Aparna Bhagat First published: March 2019 Production reference: 1290319 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-78980-216-0

www.packtpub.com

mapt.io

Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

Why subscribe? Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals Improve your learning with Skill Plans built especially for you Get a free eBook or video every month Mapt is fully searchable Copy and paste, print, and bookmark content

Packt.com Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details. At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.

Contributors About the author Ajit Pratap Kundan stands at the leading edge of the most innovative cloud technology in the world. He has helped position VMware as a leader in the private cloud area in relation to his federal government customers through an SDDC approach. An innovative techy with 18+ years of industry experience, he has promoted technologies to his government and defense customers. Ajit is a valued writer on cloud technologies and has authored one book, VMware CrossCloud Architecture, published by Packt. He currently resides in Delhi, India, with his wife, Archana, and their two sons, Aaradhya and Akshansh. I would like to give deep thanks and gratitude to my VMware and partner colleagues, along with their customers, for their guidance and suggestions.

About the reviewers James Bowling is a cloud infrastructure architect/engineer, VCIX-DCV/CMA, VCP-DTM, VMware vExpert (x7), Cisco Champion – Datacenter(x3), EMC Elect(x2), DFW VMUG Leader, and virtualization enthusiast located in Dallas, Texas, with over 18 years' experience. His experience ranges from designing, deploying, and maintaining virtualized infrastructures, while utilizing different types of technology, to automation and scaling resources. He also maintains a personal blog focusing on virtualization, vSential (dot) com. He has spoken at the following events: Veeam User Group Veeam Whiteboard Friday VMware User Group (VMUG) VMworld (US/EMEA) Interop – Las Vegas (Virtualization and Data Center Architecture Track Chair) Martin Gavanda has more than 10 years' experience, mainly in connection with service providers offering IaaS solutions based on VMware vSphere products. He was responsible for the design and implementation of IaaS solutions in the Central Europe region. Currently, he is working as an independent cloud architect, focusing on large infrastructure projects and practicing as a VMware instructor. For the past year, he has led more than a dozen on-site VMware workshops. He has created several virtual classes focusing on the VMware vSphere platform, with thousands of students subscribed, and he runs his own blog regarding virtualization and the cloud.

Packt is searching for authors like you If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

Table of Contents Preface

1

Section 1: VMware Approach with ML Technology Chapter 1: Machine Learning Capabilities with vSphere 6.7 Technical requirements ML and VMware ML-based data analysis

Using virtualized GPUs with ML

Modes of GPU usage

Comparing ML workloads to GPU configurations DirectPath I/O Scalability of GPU in a virtual environment Containerized ML applications inside a VM vGPU scheduling and vGPU profile selection

Power user and designer profiles Knowledge and task user profiles Adding vGPU hosts to a cluster with vGPU Manager

ML with NVIDIA GPUs

Pool and farm settings in Horizon Configuring hardware-accelerated graphics

Virtual shared graphics acceleration Configuring vSGA settings in a virtual machine Virtual machine settings for vGPU GRID vPC and GRID vApps capabilities GRID vWS to Quadro vDWS

Summary Further reading Chapter 2: Proactive Measures with vSAN Advanced Analytics Technical requirements Application scalability on vSAN Storage and network assessment Storage design policy

VMware best practices recommendations

Network design policy

VMware best practices recommendations VMware's Customer Experience Improvement Program/vSAN ReadyCare

Intelligent monitoring

General monitoring practices

vSAN Health Check plugin vSAN Observer vRealize Operations Manager monitoring

9 10 10 10 11 13 13 14 15 16 17 18 18 18 19 19 22 23 24 24 27 28 31 32 33 33 34 34 35 35 37 38 39 40 41 41 42 42

Table of Contents Challenges affecting business outcomes Business benefits Technical Issues Technical solution Log Intelligence advantages

HA configuration in stretched clusters Two-node clusters

Witness appliance for the vSAN cluster Configuring the vSAN cluster

vSAN policy design with SPBM

Defining a policy based on business objectives FTT policy with RAID configurations

Summary Further reading Chapter 3: Security with Workspace ONE Intelligence Technical requirements Workspace ONE Intelligence Business objectives of Workspace ONE Intelligence Integrated deep insights App analytics for smart planning Intelligent automation driven by decision engines Design requirements Conceptual designs

Top ten use cases of Workspace ONE Intelligence

Identifying and mitigating mobile OS vulnerabilities Insights into Windows 10 OS updates and patches Predicting Windows 10 Dell battery failures and automating replacement Identifying unsupported OS versions and platforms Tracking OS upgrade progress Monitoring device utilization or usage Increasing compliance across Windows 10 devices Comprehensive mobile app deployment visibility Tracking migration and adoption of productivity applications Adopting internal mobile applications

Workspace ONE Trust Network Workspace ONE AirLift

Workspace ONE platform updates

Expanded Win32 app delivery Simplified macOS adoption Extended security for Microsoft Office 365 (O365) applications VMware Boxer with Intelligent Workflows Extended management for rugged devices

Summary Chapter 4: Proactive Operations with VMware vRealize Suite Technical requirements Unified end-to-end monitoring Intelligent operational analytics

[ ii ]

43 43 44 44 44 45 46 46 47 50 50 52 53 53 55 56 56 57 58 59 60 61 62 68 68 69 69 69 70 70 71 71 72 72 73 74 74 75 75 75 75 76 76 77 78 78 78

Table of Contents

The vRealize Operations Manager architecture Application architecture overview

Capacity planning Critical success factors Kubernetes solution from VMware

Pivotal Container Service and VMware Kubernetes Engine

SDDC journey stages

VMware container-based services

Deploying NSX-T for network virtualization on ESXi and deploying PKS for use in a private cloud Deploying the NSX-T foundation Deploying and running containerized workloads

VMware Cloud on AWS

VMware Cloud on AWS differs from on-premises vSphere VMware Cloud on the AWS implementation plan Implementation plan for VMware Cloud on AWS

Detailed initial steps to configure VMC on AWS Installation, configuration, and operating procedures Hybrid-linked-mode testing functionality Support and troubleshooting

Summary Further reading Chapter 5: Intent-Based Manifest with AppDefense Technical requirements VMware innovation for application security Digital governance and compliance Intelligent government workflows with automation Transforming networking and security Business outcomes of the VMware approach Expanding globally with AppDefense

Application-centric alerting for the SOC

Transforming application security readiness Innovating IT security with developers, security, and the Ops team Least-privilege security for containerized applications Enhanced security with AppDefense

AppDefense and NSX

Detailed implementation and configuration plan

Environment preparation for AppDefense deployment

Summary

79 80 80 81 82 82 83 84 84 85 85 86 87 88 89 90 90 95 97 98 99

101 102 102 103 104 105 105 108 109 110 110 111 112 113 116 117 121

Section 2: ML Use Cases with VMware Solutions Chapter 6: ML-Based Intelligent Log Management Technical requirements Intelligent log management with vRealize Log Insight Log Intelligence value propositions

Log Intelligence key benefits for service providers Audit log examples

[ iii ]

125 125 126 126 129 130

Table of Contents

Cloud operations stages Standardize Service Broker Strategic partner

The Log Insight user interface

Indexing performance, storage, and report export The user experience Events

VMware vReaIize Network Insight Supported data sources

Summary Chapter 7: ML as a Service in the Cloud Technical requirements MLaaS in a private cloud VMware approach for MLaaS

MLaaS using vRealize Automation and vGPU

NVIDIA vGPU configuration on vSphere ESXi Customizing the vRealize Automation blueprint

LBaaS overview

LBaaS design use cases

Challenges with network and security services The NaaS operating model

LBaaS network design using NSX BIG-IP DNS high-level design

Customizing the BIG-IP DNS component

The BIG-IP DNS load-balancing algorithm Global availability Ratio Round robin

The LBaaS LTM design

Configuring BIG-IP LTM objects Designing the LTM load-balancing method Designing the LTM virtual server

Summary Chapter 8: ML-Based Rule Engine with Skyline Technical requirements Proactive support technology – VMware Skyline Collector, viewer, and advisor Release strategy

Overview of Skyline Collector

The requirements for Skyline Collector

Networking requirements Skyline Collector user permissions VMware Skyline Collector admin interface Linking with My VMware account Managing endpoints

[ iv ]

131 131 132 132 133 134 135 137 139 140 142 145 145 146 146 146 147 147 151 151 155 156 159 160 160 161 161 161 162 162 162 163 164 164 165 165 166 167 168 170 170 171 172 174 176 176

Table of Contents

CEIP

Configuring VMware Skyline Collector admin interface Auto-upgrade

Types of information that are collected Product usage data utilization

Summary Chapter 9: DevOps with vRealize Code Stream Technical requirements Application development life cycles CD pipeline CI pipeline Planning

SDLC SCM CI AR Release pipeline automation (CD) CM

HC COM

Feedback

Request fulfillment

Change management Release management Compliance management Incident management Event management Capacity management

Wavefront dashboard

Getting insights by monitoring how people work

Automation with vRealize

Deploying Infrastructure as Code

vRealize Code Stream

Pipeline automation model – the release process for any kind of software vRCS deployment architecture

System architecture Integrating vRCS with an external, standalone vRA

Summary Further reading Chapter 10: Transforming VMware IT Operations Using ML Overview on business and operations challenges

The challenges of not having services owners for the operations team A solution with service owners Responsibilities of the service owner

Transforming VMware technical support operations SDDC services

Service catalog management

Service design, development, and release

[v]

177 178 179 179 180 182 183 184 184 185 185 186 186 186 187 187 187 187 188 188 188 189 189 190 190 191 191 191 193 194 195 197 198 201 201 202 205 206 207 209 210 211 212 212 213 215 215 215

Table of Contents Cloud business management operations Service definition and automation

NSX for vSphere Recommendations with priority

Recommendations with priority 1 Recommendations with priority 2 Recommendations with priority 3

Virtual data centers

IaaS solution using vRealize Suite

Business-level administration and organizational grouping vRA deployment vRA appliance communication Services running as part of the identity service A complete solution with the desired result

Summary

215 216 216 219 219 221 223 224 226 227 230 230 231 233 234

Section 3: Dealing with Big Data, HPC , IoT, and Coud Application Scalability through ML Chapter 11: Network Transformation with IoT Technical requirements IoT VMware Pulse

The queries that arise related to VMware Pulse

Pulse IoT Center infrastructure management blueprint Deploying and configuring the OVA Configuring IoT support Virtual machines in the OVA IoT use cases with VMware Pulse

Powering the connected car (automotive industry) Entertainment, parks, and resorts Smart hospitals (medical) Smart surveillance (higher education) Smart warehouse (retail industry) The internet of trains (transportation and logistics) The financial industry Smart weather forecasting

IoT data center network security

NSX distributed firewall Prerequisites to any automation

Hybrid cloud for scale and distribution

Summary Chapter 12: Virtualizing Big Data on vSphere Technical requirements Big data infrastructure Hadoop as a service

Deploying the BDE appliance Configuring the VMware BDE The BDE plugin

[ vi ]

237 238 238 239 239 241 241 245 245 253 254 254 254 255 255 256 256 256 257 258 258 260 260 263 263 264 264 266 267 268

Table of Contents Configuring distributions on BDE The Hadoop plugin in vRO

Open source software

Considering solutions with CapEx and OpEx Benefits of virtualizing Hadoop

Use case – security and configuration isolation Case study – automating application delivery for a major media provider

Summary Further reading Chapter 13: Cloud Application Scaling Technical requirements Cloud-native applications Automation with containers Container use cases

Challenges with containers

PKS on vSphere

PKS availability zone

PKS/NSX-T logical topologies

Use cases with different configurations PKS and NSX-T Edge Nodes and Edge Cluster PKS and NSX-T communications Storage for K8s cluster node VMs Datastores

Summary Chapter 14: High-Performance Computing Technical requirements Virtualizing HPC applications Multi-tenancy with guaranteed resources

Critical use case – unification High-performance computing cluster performances A standard Hadoop architecture Standard tests Intel tested a variety of HPC benchmarks

Summary Other Books You May Enjoy

270 271 276 277 278 279 279 280 281 283 283 284 285 286 286 287 289 293 293 294 295 296 297 298 299 299 300 301 303 305 308 309 311 312 315

Index

319

[ vii ]

Preface This book presents an introductory perspective on how machine learning (ML) plays an important role in the VMware environment. It offers a basic understanding of how to leverage ML primitives, along with a deeper look into integration with VMware tools that are used for automation purposes today.

Who this book is for This book is intended for those planning, designing, and implementing the virtualization/cloud components of the Software-Defined Data Center foundational infrastructure. It helps users to put intelligence in their automation tasks to get self driving data center. It is assumed that the reader has knowledge of, and some familiarity with, virtualization concepts and related topics, including storage, security, and networking.

What this book covers Chapter 1, Machine Learning Capabilities with vSphere 6.7, covers performance benchmarking

on ML-based applications using GPUs in vSphere environment to support different customer business objectives.

Chapter 2, Proactive Measures with vSAN Advanced Analytics, explains how to improve the

support experience for HCI environments, which will help customers maintain performance by rapidly resolving issues and minimizing downtime by means of proactive telemetry capabilities from vSAN Support Insight advanced analytics. Chapter 3, Security with Workspace ONE Intelligence, describes an innovative approach to

enterprise security for employees, apps, endpoints, and networks with access management, device, and app management, and for trusted analytics frameworks. Chapter 4, Proactive Operations with VMware vRealize Suite, explains how to automate data

centers and public clouds running on vSphere by injecting advanced analytics into its VMware vRealize Suite.

Intelligent Automation with VMware This book presents an introductory perspective on how machine learning plays an important role in a VMware environment. It offers a basic understanding of how to leverage machine learning primitives, along with a deeper look into integration with the VMware tools used for automation today. This book begins by highlighting how VMware addresses business issues related to its workforce, customers, and partners with emerging technologies such as machine learning to create new, intelligence-driven, end user experiences.

You will learn how to apply machine learning techniques incorporated in VMware solutions for data center operations. You will go through management toolsets with a focus on machine learning techniques. At the end of the book, you will learn how the new vSphere Scale-Out edition can be used to ensure that HPC, big data performance, and other requirements can be met (either through development or by fine-tuning guidelines) with mainstream products.

Things you will learn: •

Orchestrate on-demand deployments based on defined policies

•

Reduce rework in a multi-layered scalable manner in any cloud

•

Automate away common problems and make life easier by reducing errors

•

Explore the centralized life cycle management of hybrid clouds

•

Deliver services to end users rather than to virtual machines

•

Use common code so you can run it across any cloud

www.packt.com

FOR SALE IN INDIA ONLY