Hands-on-3: Load balance and reverse proxy

All files of this hands-on can be downloaded from THIS LINK. Due time is 2019-12-16 23:59.

In this series of hands-on, we are going to deploy and modify a simple online sock shop to learn how to improve the application's performance with system methods.

The sock shop is an open-source project maintained by Weaveworks and Container Solutions, which simulates the user-facing part of an e-commerce website that sells socks. The project intends to demonstrate how to develop a website system using the microservice structure. The structure of this system looks like the following graph where arrow means dependency relationships (e.g., frontend depends on all backend microservices):

dependency

1 File Structure

This hands-on contains the following files:

If you are confused by any description above, keep calm and carry on. More detailed information will be described in the following sections.

2 Deploy the website

In this part, we are going to deploy the website in any platform with docker (e.g., docker for windows, virtual machine, real Linux, AWS, etc.) as you prefer and store the deployment steps into a shell script for later usage.

2.1 Deploy components

The components are deployed in Docker containers and connected using DNS mapping. For example, to start the component Frontend, you need to use docker run -d dplsming/sockshop-frontend:0.1 command, which created a container from the corresponding docker image. And for Fontend to find component Cart, it uses name cart instead of IP address like 172.20.1.2.

Therefore, to deploy this application, you first need to create a docker network using docker network to hold later created containers. Then you can use docker run to download images containing corresponding components and run containers from the images. What's more, don't forget to append --add-host option when using docker run to add the mapping from components' domain names to IP addresses into specific containers' host file. --network option is also necessary for docker run command, which locates the created container into the specified network. Last but not least, don't forget to use -p <hostport>:8079 operations when starting the container of Frontend component to generate port mapping in the host machine for your website.

NOTE: By default, the docker network will choose the next available subnet and docker run --network will assign following available IP address in the subnet. Therefore, the deployment steps are non-deterministic. For later scriptwriting convenience, We highly recommend you to allocate fixed IP addresses (check --ip option for docker run and --subnet operation for docker network) for your components.

The following table shows the mapping between a component's name and its corresponding docker image and domain name used by others to locate it.

ComponentDocker ImageDomain name in other components
Frontenddplsming/sockshop-frontend:0.1null
Shippingweaveworksdemos/shipping:0.4.8shipping
Userweaveworksdemos/user:0.4.4user
Ordersweaveworksdemos/orders:0.4.7orders
Cartweaveworksdemos/carts:0.4.8carts
Paymentweaveworksdemos/payment:0.4.3payment
Catalogueweaveworksdemos/catalogue:0.3.5catalogue
ObjectDBmongo:3.4carts-db, user-db, and orders-db for component Cart, User and Orders accordingly
CatalogueDBdplsming/sockshop-cataloguedb:0.1catalogue-db

Note: For component User, environment variable MONGO_HOST of its container should be set to user-db:27017 with command-line option -e.

QUESTION 1: Please try to deploy the online shop application according to the dependency graph and the table listed above. You should hand in an executable shell script that contains all your deployment steps and can be used later to deploy the website conveniently.

2.2 Check correctness

After successfully deploying the sock shop application, you can access the online store with your browser. To do so, please enter the IP address of your host machine (mostly localhost) and port mapped to the frontend component into your browser. You should be able to browse and purchase socks in the online store. What's more, you can also check the correctness of your development using our check script:

In the script, $your_ip_address means the IP address of your host machine inside a docker container. It can be retrieved by checking the gateway of docker network inspect bridge and is usually 172.17.0.1. $your_port is the host port mapped to frontend:8079 as you deployed in the previous section.

The result should be something that looks like:

If there are only 0(0.00%) in the # fails column and no exceptions printed in the log part, that means you have done a correct deployment.

After making sure all things work well, you can continue to load testing parts.

3 Load test

3.1 Test environment and methodology

Since the sock shop application and load test scripts both require many CPU cores and memory resources, laptops are not suitable for later load testing. Therefore, we choose amazon AWS EC2 as our testing platform. Please follow this tutorial to set up your own AWS EC2 instance and redeploy your sock shop onto the remote server.

NOTE: Since our budget is limited, don't forget to "stop (hibernate)" your EC2 instance when you are not working on this hands-on.

NOTE: The access latency of EC2 instance is high. Please be patient and try to only do the final load test on AWS EC2 instance while leaving other experimental optimizations on your own machine.

For load testing, we choose an open-source load testing framework Locust, the same as that we used in previous correctness testing. In each locust process, all users are emulated in the same thread. Therefore, we will run two locust processes for testing, one for background stress emulation (background-stress.py), the other for testing and monitoring. In each process, an emulated user will wait some interval time during two requests to further simulate real user behaviors. More details of each test case can be found in later sections.

The sample output of Locust are listed as below, where some critical metrics are:

Note: There may be some failures in the load testing results due to the server can't handle such high pressure. Don't worry, ignore them.

3.2 Case 1: Static pages accessing

The first test case of our load testing is to simulate some users accessing the static assets, especially the images (case1.py), while other users are purchasing socks as usual (background-stress.py). You can try this test case by using the command:

Where $LOAD means how many background-stress users are simulated and $TIME refers to the load-testing time. We highly recommend not using a $LOAD higher than 150. This is because a higher load makes no sense since the single thread locust process cannot send requests at a very high throughput. $TIME can be set to 30s for experimental tests and is set to 120s in load-test.py to gain a more stable result.

To use the load-test.py, try the following command:

This script will last for about 20 minutes and will test 10 points at different stress. The result will be printed to screen in CSV format containing Load, Throughput, Average Response Time, 95% Response Time, 99% Response Time tuples. The output CSV looks like follows:

Question 2: Please try the command by yourself and draw some curves to show the trend of RT and throughput under different loads.

Question 3: Please describe and explain the phenomenon you find in the curve you have just drawn.

Under this workload, the Express web server in the frontend component serving both dynamic requests and static pages, which slowdowns the response of static response. Luckily, Nginx has much better performance compared to express according to this link. Therefore, use Nginx to dispatch static and dynamic requests may help in this situation.

Question 4: Try to optimize the static page accessing response time using Nginx. Please briefly describe what you have done and draw the curves to show the effect of your optimization.

Hint: To do so, you need to create another docker image from Nginx to divide static asset requests and dynamic service requests apart. The source code and docker files of the frontend are given in the frontend folder. In the source code, the folder public contains all the static assets, which can be moved out and be served by Nginx.

3.3 Case 2: Dynamic service accessing

Note: This test should be done based on your first optimization.

The second test case of our load testing is simulating when many users are purchasing socks together like 11.11. To try this workload, use scripts below where $LOAD and $TIME remains the same meaning as before:

And for drawing the curves, use the script below:

In this case, all components of the application are all suffering from high loads, among which the frontend suffers the most. This imbalance is due to all requested are routed by the frontend while other components only serve requests in their responsibility.

Instead of re-implementing the frontend component or optimizing Express, load-balancing can be applied to improve the performance under this situation. To do so, you need to create multiple frontend components and route user requests to them using Nginx.

Question 5: Try to optimize the performance of the application under this workload with load-balancing. Please briefly describe your modification. You also need to re-run load-test.sh and generate figures to show how well your optimization works.

HINT: The frontend component manages all user sessions, so some environment variables may need to be set since frontend containers don't know sessions maintained by others. Check server.js and config.js for more details.

Question 6: Try your optimization under the situation where all frontend containers are enforced with a CPU limitation using --cpus 0.5 when created. Is your optimization more effective or less effective? Why?

4 Hands in

To hand in, please compress the following files into a compressed file with a title like 516037910xxx.zip.

After that, please upload your compressed file to our public FTP (ftp://public.sjtu.edu.cn/upload/handson3, username: followmoon, password: public) before 2019-12-16 23:59.

Note: Please don't add images with too large file size into your document due to the size limit of the public FTP.