Data Loading...
Python API Development Fundamentals Develop a full-stack web application with Python and Flask
Jack Chan, Ray Chung, and Jack Huang FOR SALE IN INDIA ONLY
www.packt.com
Python API Development Fundamentals Develop a full-stack web application with Python and Flask
Jack Chan Ray Chung Jack Huang
Python API Development Fundamentals Copyright © 2019 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Authors: Jack Chan, Ray Chung, and Jack Huang Technical Reviewer: Amritansh Managing Editor: Aditya Shah Acquisitions Editors: Kunal Sawant and Anindya Sil Production Editor: Salma Patel Editorial Board: Shubhopriya Banerjee, Bharat Botle, Ewan Buckingham, Megan Carlisle, Mahesh Dhyani, Manasa Kumar, Alex Mazonowicz, Bridget Neale, Dominic Pereira, Shiny Poojary, Abhisekh Rane, Erol Staveley, Ankita Thakur, Nitesh Thakur, and Jonathan Wray. First Published: November 2019 Production Reference: 1211119 ISBN: 978-1-83898-399-4 Published by Packt Publishing Ltd. Livery Place, 35 Livery Street Birmingham B3 2PB, UK
Table of Contents Preface
i
Chapter 1: Your First Step
1
Introduction .................................................................................................... 2 Understanding API ......................................................................................... 2 RESTful API ...................................................................................................... 4 REST Constraints/Principles ................................................................................ 4
HTTP Protocol ................................................................................................. 5 HTTP Methods and CRUD .............................................................................. 5 The JSON Format ............................................................................................ 7 HTTP Status Codes ......................................................................................... 8 Commonly used HTTP Status Codes .................................................................. 8
Open API ......................................................................................................... 8 The Flask Web Framework .......................................................................... 10 Building a Simple Recipe Management Application ................................ 10 Virtual Environment .......................................................................................... 11 Exercise 1: Building Our First Flask Application ............................................. 12 Exercise 2: Managing Recipes with Flask ........................................................ 15
Using curl or httpie to Test All the Endpoints .......................................... 20 Exercise 3: Testing Our API Endpoints with httpie and curl ......................... 21
Postman ........................................................................................................ 27 The Postman GUI ............................................................................................... 27 Sending a GET Request ...................................................................................... 28 Sending a POST Request ................................................................................... 29 Saving a Request ................................................................................................ 29
Activity 1: Sending Requests to Our APIs Using Postman ............................. 30 Exercise 4: Automated Testing Using Postman .............................................. 30 Activity 2: Implement and Test the delete_recipe Function ......................... 32
Summary ....................................................................................................... 33
Chapter 2: Starting to Build Our Project
35
Introduction .................................................................................................. 36 What is Flask-RESTful? ................................................................................. 36 Using Flask-RESTful to Develop Our Recipe-Sharing Platform, "Smilecook" ..................................................... 36
Virtual Environment .................................................................................... 37 Exercise 5: Creating a Development Project in PyCharm ............................. 38
Creating a Recipe Model ............................................................................. 40 Exercise 6: Creating the Recipe Model ............................................................ 41 Resourceful Routing .......................................................................................... 42 Exercise 7: Defining an API Endpoint for the Recipe Model ......................... 43 Exercise 8: Defining the Recipe Resource ....................................................... 45 Exercise 9: Publishing and Unpublishing the Recipes ................................... 46
Configuring Endpoints ................................................................................. 48 Exercise 10: Creating the Main Application File ............................................. 48
Making HTTP Requests to the Flask API using curl and httpie ............... 50 Exercise 11: Testing the Endpoints Using curl and httpie ............................. 50 Exercise 12: Testing the Auto-Incremented Recipe ID .................................. 51 Exercise 13: Getting All the Recipes Back ....................................................... 52 Exercise 14: Testing the Recipe Resources ..................................................... 53 Exercise 15: Negative Testing ........................................................................... 55 Exercise 16: Modifying the Recipes .................................................................. 56 Exercise 17: Getting Back Specific Recipes with a Certain ID ....................... 57
Activity 3: Testing the APIs Using Postman .................................................... 58 Activity 4: Implementing the Delete Recipe Function ................................... 58
Summary ....................................................................................................... 59
Chapter 3: Manipulating a Database with SQLAlchemy
61
Introduction .................................................................................................. 62 Databases ..................................................................................................... 62 Database Management System ....................................................................... 62
SQL ................................................................................................................. 63 ORM ............................................................................................................... 63 Exercise 18: Setting Up a Smilecook Database ............................................... 64
Defining Our Models ................................................................................... 67 Exercise 19: Installing Packages and Defining Models ................................. 69 Exercise 20: Using Flask-Migrate to Build a Database Upgrade Script ........ 74 Exercise 21: Applying Database Insertion ....................................................... 79 Activity 5: Creating a User and a Recipe ......................................................... 81
Password Hashing ....................................................................................... 82 Exercise 22: Implement the User Registration Feature and Hash the User's Password ......................................................................... 82 Exercise 23: Testing the Application in Postman ........................................... 86 Activity 6: Upgrading and Downgrading a Database ..................................... 89
Summary ....................................................................................................... 90
Chapter 4: Authentication Services and Security with JWT
93
Introduction .................................................................................................. 94 JWT ................................................................................................................. 94 Flask-JWT-Extended ..................................................................................... 96 Exercise 24: Implementing a User Login Function ......................................... 97 Exercise 25: Testing the User Login Function .............................................. 101
Exercise 26: Creating the me Endpoint ........................................................ 105
Designing the Methods in the Recipe Model .......................................... 107 Exercise 27: Implementing Access-Controlled Recipe Management Functions ..................................................................... 107 Exercise 28: Testing the Recipe Management Functions ........................... 111
Refresh Tokens ........................................................................................... 114 Exercise 29: Adding a Refresh Token Function ........................................... 115 Exercise 30: Obtaining a New Access Token Using a Refresh Token ........ 117
The User Logout Mechanism .................................................................... 118 Exercise 31: Implementing the Logout Function ........................................ 119 Exercise 32: Testing the Logout Function .................................................... 121 Activity 7: Implementing Access Control on the publish/unpublish Recipe Function .................................................. 123
Summary ..................................................................................................... 123
Chapter 5: Object Serialization with marshmallow
125
Introduction ................................................................................................ 126 Serialization versus Deserialization ........................................................ 126 marshmallow .............................................................................................. 127 A Simple Schema ........................................................................................ 127 Field Validation ................................................................................................ 128 Customizing Deserialization Methods .......................................................... 128
UserSchema Design ................................................................................... 129 Exercise 33: Using marshmallow to Validate the User Data ..................... 130 Exercise 34: Testing the User Endpoint before and after Authentication ............................................................................... 133
RecipeSchema Design ............................................................................... 135 Exercise 35: Implementing RecipeSchema .................................................. 136 Exercise 36: Testing the Recipe API .............................................................. 142
The PATCH Method .................................................................................... 146 Exercise 37: Using the PATCH Method to Update the Recipe .................... 146 Searching for Authors and Unpublished Recipes ....................................... 149 Using the webargs Package to Parse the Request Arguments ................. 150 Exercise 38: Implementing Access Control on Recipes .............................. 150 Exercise 39: Retrieving Recipes from a Specific Author ............................. 154 Activity 8: Serializing the recipe Object Using marshmallow .................... 158
Summary ..................................................................................................... 159
Chapter 6: Email Confirmation
161
Introduction ................................................................................................ 162 Mailgun ....................................................................................................... 163 Exercise 40: Get Started with Using Mailgun ............................................... 163 Exercise 41: Using the Mailgun API to Send Out Emails ............................. 166
User Account Activation Workflow .......................................................... 168 Exercise 42: Generating the Account Activation Token ............................. 169 Exercise 43: Sending Out the User Account Activation Email ................... 170 Activity 9: Testing the Complete User Registration and Activation Workflow ................................................................................ 174 Setting Up Environment Variables ................................................................ 174 Exercise 44: Setting Up Environment Variables in PyCharm ..................... 174
HTML Format Email ................................................................................... 176 Activity 10: Creating the HTML Format User Account Activation Email ... 178
Summary ..................................................................................................... 178
Chapter 7: Working with Images
181
Introduction ................................................................................................ 182 Building the User Avatar Function ........................................................... 182 Exercise 45: Adding the avatar_image Attribute to the User Model ........ 183
Flask-Uploads ............................................................................................. 185 Upload Sets ...................................................................................................... 186 Exercise 46: Implementing the User Avatar Upload Function .................. 186 Exercise 47: Testing the User Avatar Upload Function Using Postman ... 191
Image Resizing and Compression ............................................................ 195 Introduction to Pillow ................................................................................ 195 Exercise 48: Implementing Image Compression in Our Smilecook Application ........................................................................ 196 Exercise 49: Testing the Image Compression Function .............................. 199 Activity 11: Implementing the Recipe Cover Image Upload Function ...... 200 Activity 12: Testing the Image Upload Function .......................................... 200
Summary ..................................................................................................... 201
Chapter 8: Pagination, Searching, and Ordering
203
Introduction ................................................................................................ 204 Pagination ................................................................................................... 204 Paginated APIs ............................................................................................ 205 Exercise 50: Implementing Pagination on the Published Recipes Retrieval Function ............................................................................ 206 Exercise 51: Testing the Pagination Functions ............................................ 211 Activity 13: Implementing Pagination on the User-Specific Recipe Retrieval API ........................................................................................ 215 Activity 14: Testing Pagination on the User-Specific Recipe Retrieval API ........................................................................................ 215
Recipe Searching ........................................................................................ 216 Exercise 52: Implementing the Search Function ......................................... 217 Exercise 53: Testing the Search Function ..................................................... 218
Sorting and Ordering ................................................................................. 219 Exercise 54: Implementing Sorting and Ordering ....................................... 221
Exercise 55: Testing the Sorting and Ordering Feature ............................. 222 Activity 15: Searching for Recipes with Specific Ingredients ..................... 224
Summary ..................................................................................................... 225
Chapter 9: Building More Features
227
Introduction ................................................................................................ 228 Caching ........................................................................................................ 228 Benefit of Caching ........................................................................................... 229
Flask-Caching .............................................................................................. 229 Exercise 56: Implementing Caching Functionality Using Flask-Caching ........................................................................................ 230 Exercise 57: Testing the Caching Function with Postman ......................... 233 Clearing the Cache when Data Updates ...................................................... 235 Activity 16: Getting Cache Data after Updating Recipe Details ................. 235 Exercise 58: Implementing Cache-Clearing Functionality .......................... 236 Exercise 59: Verifying the Cache-Clearing Function ................................... 237
API Rate Limiting ........................................................................................ 240 HTTP Headers and Response Codes ............................................................. 241
Flask-Limiter ............................................................................................... 241 Exercise 60: Implementing API Rate-Limiting Functionality ...................... 242 Exercise 61: Verifying the Rate-Limit Function ............................................ 244 Exercise 62: Adding a Whitelist ..................................................................... 245 Activity 17: Adding Multiple Rate-Limit Restrictions .................................. 247
Summary ..................................................................................................... 247
Chapter 10: Deployment
249
Introduction ................................................................................................ 250 Deployment ................................................................................................ 250 Comparing SaaS, PaaS, and IaaS .............................................................. 251
The Heroku Platform ................................................................................. 252 Configuration Handling in Smilecook ...................................................... 253 Exercise 63: Configuration Handling for the Production and Development Environments .................................................................. 253 Exercise 64: Adding a Staging Configuration Class ..................................... 256
Heroku Application .................................................................................... 256 Exercise 65: Creating a New Application in Heroku ................................... 257
Heroku Add-Ons ......................................................................................... 260 Exercise 66: Installing Heroku Postgres ....................................................... 260
Setting Up Environment Variables for the Heroku App ........................ 263 Exercise 67: Setting Up the App Environment Variables ............................ 264 Deployment Using Heroku Git ...................................................................... 265 What is Git? ...................................................................................................... 266 What is gitignore? ........................................................................................... 266 What is Procfile? .............................................................................................. 266 What is Gunicorn? ........................................................................................... 266 Exercise 68: Setting Up the Git and the Heroku CLI ................................... 267 Exercise 69: Checking the Heroku Postgres Tables in pgAdmin ............... 272
Setting Up Variables in Postman ............................................................. 276 Exercise 70: Setting Up Variables in Postman ............................................. 276 Activity 18: Changing access_token to a Variable in Postman .................. 278
Setting up the Front-end Interface to Work with the Smilecook API ...................................................................................... 279 Summary ..................................................................................................... 282
Appendix
285
Index
349
>
Preface
About This section briefly introduces the authors, the coverage of this book, the technical skills you'll need to get started, and the hardware and software requirements required to complete all of the included activities and exercises.
ii | Preface
About the Book Python is a flexible language that can be used for much more than just script development. By knowing how the Python RESTful APIs work, you can build a powerful backend for web applications and mobile applications using Python. You'll take your first steps by building a simple API and learning how the frontend web interface can communicate with the backend. You'll also learn how to serialize and deserialize objects using the marshmallow library. Then, you'll learn how to authenticate and authorize users using Flask-JWT. Apart from all this, you'll also learn how to enhance your APIs by adding useful features, such as email, image upload, searching, and pagination. You'll wrap up the whole book by deploying the APIs to the cloud. By the end of this book, you'll have the confidence and skill to leverage the power of RESTful APIs and Python to build efficient web applications.
About the Authors Jack Chan started programming at the age of 10. He was an active participant in worldwide programming contests at university. Since graduation, he has been working in the finance and IT industries for more than 10 years, building systems that analyze millions of transactions and positions to spot suspicious activity. He has leveraged the powerful analytical Python libraries to perform data analysis and performance optimization for a trading system that works at a microsecond level. He has an in-depth knowledge of the modern software development life cycle, which uses automated testing, continuous integration, and agile methodologies. Among all programming languages, he found Python to be the most expressive and powerful. He has created courses and taught students all over the world, using Python as the teaching language. Inspiring aspiring developers to take on the software engineering career path has always been Jack's goal. Ray Chung is a developer and an instructor. He loves helping students learn to code and master software development. He is now self-employed and develops web applications, network applications, and chatbots using Python. The first program he sold was a network application that helped clients to configure, maintain and test thousands of multi-vendor network devices. He's experienced with big projects such as a Marathon's online registration system, rental car management systems, and more. He has worked extensively with Google App Engine, PostgreSQL, and advanced system architecture design. He has been a self-taught developer for many years and knows the most efficient ways to learn a new skill.
About the Book | iii Jack Huang is a programmer with more than 7 years of experience in developing web applications in Python, Javascript, and .NET. He is skilled in web frameworks such as Flask, Django, and Vue, as well as in PostgreSQL, DynamoDB, MongoDB, RabbitMQ, Redis, Elasticsearch, RESTful API design, payment processing, system architecture design, database design, and Unix systems. He has written applications for an accessories shop platform, an ERP system, a divination web application, a podcast platform, a job search service, a blog system, a salon reservation system, an e-commerce service, and more. He also has experience in handling large amounts of data and optimizing payment processing. He is an expert web application developer who loves coding and is constantly following the newest technology.
Learning Objectives By the end of this book, you will be able to: • Understand the concept of a RESTful API • Build a RESTful API using Flask and the Flask-Restful extension • Manipulate a database using Flask-SQLAlchemy and Flask-Migrate • Send out plaintext and HTML format emails using the Mailgun API • Implement a pagination function using Flask-SQLAlchemy • Use caching to improve API performance and efficiently obtain the latest information • Deploy an application to Heroku and test it using Postman
Audience This book is ideal for aspiring software developers who have a basic-to-intermediate knowledge of Python programming and who want to develop web applications using Python. Knowledge of how web applications work will be beneficial, but is not essential.
Approach This book takes the learning-by-doing approach to explain concepts to you. You'll build a real-life web application by implementing each concept that you learn in theory. This way, you'll reinforce your new skill.
Python API Development Fundamentals Python is a flexible language that can be used for much more than just script development. By knowing how Python RESTful APIs work, you can build a powerful backend for web applications and mobile applications using Python. You'll take your first steps by building a simple API and learning how the frontend web interface can communicate with the backend. You'll also learn how to serialize and deserialize objects using the marshmallow
library. Then, you'll learn how to authenticate and authorize users using Flask-JWT. You'll also learn how to enhance your APIs by adding useful features, such as email, image upload, searching, and pagination. You'll wrap up the whole book by deploying your APIs to the cloud. By the end of this book, you'll have the confidence and skill to leverage the power of RESTful APIs and Python to build efficient web applications.
Things you will learn: •
Understand the concept of RESTful APIs
•
•
Build a RESTful API using Flask and the Flask-RESTful extension
Implement a pagination function using Flask-SQLAlchemy
•
•
Manipulate databases using Flask-SQLAlchemy and Flask-Migrate
Use caching to improve API performance and efficiently get the latest information
•
•
Send out plaintext and HTML format emails using the Mailgun API
Deploy an application to Heroku and test it using Postman
www.packt.com
FOR SALE IN INDIA ONLY