Setting Up Virtual Machine(VM) Environment

VM environment set up documentation turned tutorial.

This work article is pretty comprehensive but you can view all the steps from Flask app to deployment documented at thegoodeconomist.com.

Access VM command line through Putty. Once in Putty command line terminal for thegoodeconomist04 let’s rename the hostname with the following commands. I’m calling it “the goodeconomist04” but you can call it whatever you want. This will help in the future if you have more than one machine or even helping distinguish between your personal computer.

sudo hostnamectl set-hostname thegoodeconomist04

Install nano editor. The “-y” just adds yes to all parameter in the install command. This program is how we’ll edit files on your machine.

There are some files we can edit directly from WinSCP. The files in our user folder. But then there are files in the /etc/ folder of the VM that require sudo privileges. I’m sure there is a way to set up WinSCP to do this but I haven’t figured it out. If someone has a good way or easy instruction to follow please let me know!

sudo yum install nano -y

Next, we will need to extend disk space.

sudo yum -y install cloud-utils-growpart gdisk
lsblk
sudo growpart /dev/xvda 2
lsblk
df -h
sudo xfs_growfs /
df -h

I was not able to load miniconda and my application on the VM without extending the disk space. This might be because the VM that I selected was so small. I’m not really sure but I couldn’t move forward without this.

This is for a folder structure that I’m using. Honestly, I’m not even sure it’s the best but I started using it and it works. Now all my path mappings are based on this.

Once in the ec2-user folder make an environments directory.

mkdir environments

Once in the environment’s folder:

mkdir flaskApp1

This is the application name that I use.

Next, move your application to flaskApp1 folder. I have a folder structure a little further down if you want to peak to see what it should look like.

Download miniconda

I download to my personal computer then move it to the VM. I struggled a bit when I tried to download it directly to the VM. This is just an extra step but it works.

Miniconda - Conda documentation

Install miniconda by navigating to the folder in your terminal and typing:

bash Miniconda3-latest-Linux-x86_64.sh

You will need to press enter many times to scroll through their terms and conditions (T & C’s as I’ve heard them called before). At the end you get to a screen that looks like below. You’ll have to type “yes” to accept and then tell the installer where you want to store miniconda.

I just press enter to allow miniconda to build in same folder the install package was moved into.

It should not take too long. For some reason this has taken me various lengths of time. When it takes too long something happens. It might be something I’m doing. I’ve had to kill this machine and restart the process. Rarely does trying to fix problems that start here ever get better.

If you see this then it’s a good sign and you should close the shell and restart.

After its installed you need to activate your environment and install the packages.

Close and reopen the Putty shell.

This is what my folder structure looks like. Yours should probably similar. Don’t worry about the “__pycache__” folders those are created by python.

Create your application environment as you usually would. Mine is very simple and just needs flask so I am just naming it “flaskApp1” but you can name it anything and installing python 3.9.

conda create -n flaskApp1 python=3.9 flask

Once the environment is built activate it.

conda activate flaskApp1

Run your application. It should run fine in the terminal.

python run.py

The screenshot below shows me trying to access the website through the web by typing in the IP address followed by “:5000” which means port 5000. It doesn’t work yet but it will.

At this point we cannot see the application for two reasons:
a. We need to adjust the run.py file
b. Deactivate server firewall

The run.py file needs the parameter:

app.run(host=’0.0.0.0’)

The screen shot below shows my run.py file as well as the folder its in on the VM through WinSCP.

Disable firewall

This could have been done earlier but at this point when you can’t see the app in port 5000 you need to disable to firewall to start seeing what you produced.

sudo firewall-cmd --state  
sudo systemctl stop firewalld
sudo systemctl disable firewalld

Now try re running your application and check out the ip address port 5000.

This is a good sign, but at this point we are running it off of port 5000 and we want to get it running on a production port so we need to install nginx, gunicorn and create some files. Once this is done we can get it to run a production port and we won’t need to use the terminal to keep the application running. It will just run on its own.

Install nginx

Next, install nginx.

This part is the impetus for this entire blog entry. I spent ten’s of hours troubleshooting the configuration and thinking that something else was the problem. In fact, I just had the configuration incorrect. There is an abundance of content that helped, but still required a lot of struggle on my part. I hope this helps but I wouldn’t be surprised if I still didn’t explain something in a way that you need to understand your specific circumstances.

sudo yum install nginx -y

When I installed nginx this is what the folder looks like, except for the “proxy_params” (highlighted):

Create nginx conf file

We need to create a .conf file in the /etc/nginx/conf.d folder. The way my connections are set up I can nano into a new file by using these commands.

Some instructions mention a sites-enabled folder. Not needed and never encountered this in my download or process. Also, there is no need to make any changes to any of the files in the /etc/nginx/conf.d folder except for adding the proxy_params file. Some instructions discuss making alterations to the nginx.conf file but this was not needed to get things to work. The .conf files in the /etc/nginx/conf.d folder do all the redirecting.

sudo nano /etc/nginx/conf.d/52.71.108.45.conf

The copy this text into your new file in the nano editor you just opened.

Things to note:
a.server name needs to get replaced with your ip address. Once you get a domain you’ll enter in the domain here. You can put more than one domain as well. I’ll show in this as we move along in this documentation.
b.the path I have in “location/static” will be different for you but this structure should make sense. It just needs to access your static files in your flask app.
c.“location/” just copy this exactly. Later on in this documentation you’ll change http://localhost:8000; to another port because that is where nginx will direct traffic if you choose to have another app on this same VM.

server {
        listen       80;
        listen       [::]:80;
        server_name  52.71.108.45;
location /static {
  alias /home/ec2-user/environments/flaskApp1/appFolder/static;
 }
location / {
  proxy_pass http://localhost:8000;
  include /etc/nginx/proxy_params;
  proxy_redirect off;
 }
}

Create nginx proxy_params file

Then we need to create a proxy_params file in the /etc/nginx folder

Copy this code into that file:

proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;

Your file and directory should look like this:

Restart nginx. It may already be running. Get comfortable with these commands. I’ve seen people use reload. That doesn’t seem to work for me. I stop and then start again. I’ve noticed now that I may not need to start and restart nginx for changes to the flask app, but as I was troubleshooting the nginx config files I did this often to verify if a change worked or not.

Here are the commands that I use:

sudo systemctl stop nginx
sudo systemctl start nginx

Once it’s running you should be able to type in the IP address and see a page that looks like this. Indicating that nginx is running but your app isn’t running, at least on port 8000 the default port for flask or something…

Install gunicorn

pip install gunicorn

To run gunicorn you’ll need to pass the number of workers on your machine. I’m not really sure what this is but it is related to the number of cores on your machine. One of the tutorials I’ve used instructed that documentation says the number of workers is two times the number of cores on your machine plus one.

In order to see the number of cores on the machine:

nproc --all

Now that we have this information we need to build a service file to run the application in the background. To do this nano into a new file that is called the application name .service.

The copy the code below into the flaskApp1.service file. Make adjustments where necessary but if you used the same names as in this documentation copy this exactly. The description can be different and improved.

[Unit]
Description=Gunicorn instance to serve flaskApp1 from flaskApp1 conda enviornment
After=network.target
[Service]
User=ec2-user
WorkingDirectory=/home/ec2-user/environments/flaskApp1
Environment="PATH=/home/ec2-user/miniconda3/envs/flaskApp1/bin"
ExecStart=/home/ec2-user/miniconda3/envs/flaskApp1/bin/gunicorn -w 3 run:app
[Install]
WantedBy=multi-user.target

Now let’s see if it works by using a command that runs the service file which uses gunicron to ultimately run flaskApp1.

sudo systemctl start flaskApp1

At this point even with nginx and flaskApp1.service running when I type the IP address into a internet browser I still get 502 Bad Gateway error.

What I need to do is change the SELINUX permissions. I’ve not seen a lot of content around doing this so I’m not sure it’s the best and most secure but it works. I’d welcome a better explanation or alternative.

Set SELINUX permission

One more thing to do is set the SELINUX permission.

sudo setenforce Permissive

Making this change will now give us the site on port 8000 which can be accessed by entering the IP address in any browser.