Create Custom Python Package

Pip install package into your venv to use in your python scripts

Goal of this article

The goal of this article is to share the steps used to create a python package that you can install into your venv environment using pip. At the end of the article, I use the custom-made modules from the package in terminal using python.

Why am I building this package?

I wanted to do this so that I could pass SQLAlchemy objects from the same models.py file easily across different applications.

As I was building the website using Flask Blueprints I found it easiest to build a module to share config objects as well.

In a separate article I will show I how I use these custom-made packages, for my config and models, in the Flask Blueprints application.

There are a few resources to do this but the one I found most useful is this medium article.

The Github repository for this article is found at webApp02-packages.

Folder Structure

Here is the folder structure for this package. This graphic shows all the folders and files you’ll need to create except the .egg-info folder. That gets created by python when you pip install.

C:\Users\Captian2020\Documents\webApp02-packages2
|
\ — -nr_packages
   | setup.py
   |
   + — -nr_packages.egg-info #← This gets created after pip install
   |
   + — -nr_config
   | | config.py
   | \- __init__.py
   |
   \ — -nr_models
     | models.py
     \- __init__.py

Here is some context as to what each folder, or tier of folder, does in my process:

Tier 1: Root folder (webApp02-packages) — this can be any name because it never really gets referenced in any point in the pip/venv process, except as the address of where things will be stored. Hence, after installing it through pip you probably can’t change this folder name.

Tier 2: Package folder (nr_packages) — name this something that you can install in pip. It seems “_” gets converted to “-” once installed.

Tier 3: Module folders (nr_config, nr_models) — the names of these folders will be what you import at the beginning of your scripts when you write:

import [MODULE NAME]

Or

from [MODULE NAME] import [OBJECT]

Create wsh_config module

The config module is going to house a class that refers to a json file that has all the application configuration parameters as well as keep sensitive data out of my python script files.

config.py file

import os
import json
if os.environ.get(‘COMPUTERNAME’)==’CAPTAIN2020':
    with open(r”C:\Users\captian2020\Documents\config_files\config_webApp02.json”) as config_file:
       config = json.load(config_file)
class ConfigDev:
    DEBUG = True
    PORT=’5000'
    SECRET_KEY = config.get(‘SECRET_KEY’)
    SQL_URI = config.get(‘SQL_URI’)
class ConfigProd:
    DEBUG = False
    PORT=’80'
    SECRET_KEY = config.get(‘SECRET_KEY’)
    SQL_URI = config.get(‘SQL_URI’)

You’ll notice I have a config_webApp02.json file that is stored in my C:\ at the top of this script.

The config.py file references a json file that has the sensitive information. You’ll need to map a json file of your own. Below is an example of what mine looks like. You can store it anywhere on your computer but you’ll need to reference it, as I did above in the config.py file.

config_webApp02.json

{
“SECRET_KEY”: “f8zx6WoDCJOxXQznGXCg”,
"SQL_URI": "sqlite:///D:\\databases\\webApp02\\webApp02.db"
}

Then you’ll need a __init__.py file in the module folder to let python know what objects are available. I have read __init__.py can be blank and it will just read all the python objects. However, that has not worked for me for building the package in this article.

__init__.py

from .config import ConfigDev, ConfigProd

That’s it for the nr_models module.

Create wsh_model module

The purpose of this module is to house the models.py file which contains your SQLAlchemy table classes, session object, and flask_login objects. Below shows part of my models.py file. You can see the rest in the Github repository.

Models.py

from flask import Flask
from wsh_config import ConfigDev
from sqlalchemy import create_engine, inspect
from sqlalchemy import Column, Integer, String, Text, Float, DateTime, ForeignKey, \
    Date
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker, Session, relationship
from datetime import datetime
from nr_config import ConfigDev, ConfigProd
from flask_login import UserMixin, LoginManager
config = ConfigDev()
Base = declarative_base()
engine = create_engine(config.SQL_URI, echo = True, connect_args={“check_same_thread”: False})
Session = sessionmaker(bind = engine)
sess = Session()
login_manager= LoginManager()
login_manager.login_view = ‘users.login’
login_manager.login_message_category = ‘info’
@login_manager.user_loader
def load_user(any_name_for_id_obj):
    return sess.query(Users).filter_by(id = any_name_for_id_obj).first()
class Users(Base, UserMixin):
    __tablename__ = ‘users’
    id = Column(Integer, primary_key = True)
    email = Column(Text, unique = True, nullable = False)
    password = Column(Text, nullable = False)
    time_stamp_utc = Column(DateTime, nullable = False, default = datetime.utcnow)
def __repr__(self):
    return f’Users(id: {self.id}, email: {self.email})’

#Build db
if 'users' in inspect(engine).get_table_names():
    print('db already exists')
else:
    Base.metadata.create_all(engine)
    print('NEW db created.')

I added Flask_login in the models.py file because it was the easiest way to get this to work and I don’t see anything wrong with it. In the other article, where I am using this package in Flask Blueprints, you’ll see that I import Login_manager into my __init__.py file inside the create_app() method.

The other thing you’ll notice here is that I am using SQLAlchemy and not Flask-sqlalchemy. This is by design because I believe it will be easier to share the model objects this way. In this article I use SQLAlchemy and not Flask-sqlalchemy in my Flask API and scheduler applications.

__init__.py

from .models import login_manager, sess, Users

That’s it for the wsh_models module.

Setup.py

from setuptools import setup
setup (
    name=”nr_packages”,
    version = “0.1”,
    author=”NickRodriguez”,
    author_email=”nick@dashanddata.com”,
    description = “some description”,
    packages=[‘nr_config’,’nr_models’],
    python_requires=”>=3.6",
)

The two things that I had to beware of when it comes to the setup.py file:

1. name: this is the name of the tier2 folder (nr_packages).

2. packages: these are the names of the tier3 folders (wsh_config, wsh_models). If you only have one its still a list and you just have one string item in the list.

I’ve seen other more detailed setup.py files. This one seems to work fine. I lifted the framework from another medium article I referenced at beginning of this article.

Install the packages to the venv/pip environment

Step 1: Activate your environment. In my case,

C:\Users\captian2020\Documents\_environments>venv_webApp02\Scripts\activate

Step 2: From the terminal navigate to the tier 1/root folder. In my case this is ‘webApp02-packages’. In my case it looks like:

(venv_webApp02) C:\Users\captian2020\Documents\webApp02-packages>

Step 3: From terminal, once you are in the environment and root folder for your packages do:

(venv_webApp02) C:\Users\captian2020\Documents\webApp02-packages>pip install -e nr-packages
#Note: Even though I’ve named the package ‘nr_packages’ the package becomes called 'nr-packages'.

Or, from the “nr_packages” folder:

(venv_webApp02) C:\Users\captian2020\Documents\webApp02-packages\nr_packages>pip install -e .

The models.py file needs SQLAlchemy to run so you’ll need to pip install sqlalchemy.

Verify the installation

Besides an error that would probably be evident in your terminal there are two ways to make sure it installed correctly

1. Pip list — the package should show up under this command. Since I can see it at the bottom of the list I know it has been added successfully.

(venv_webApp02) C:\Users\captian2020\Documents\webApp02-packages>pip list
Package Version Location
 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
greenlet 1.1.3
pip 21.2.4
setuptools 58.1.0
SQLAlchemy 1.4.40
nr-packages 0.1 c:\users\captian2020\documents\webApp02-packages\nr_packages
WARNING: You are using pip version 21.2.4; however, version 22.2.2 is available.
You should consider upgrading via the ‘C:\Users\captian2020\Documents\_environments\venv_webApp02\Scripts\python.exe -m pip install — upgrade pip’ command.

2. Run python from terminal and check one of the objects. In the snippet below I am checking that I can get the SQL_URI path attribute. Since it returns the right path I know if works.

(venv_webApp02) C:\Users\captian2020\Documents>python
Python 3.10.2 (tags/v3.10.2:a58ebcc, Jan 17 2022, 14:12:15) [MSC v.1929 64 bit (AMD64)] on win32
Type “help”, “copyright”, “credits” or “license” for more information.
>>> from nr_config import ConfigDev
>>> config_obj_test = ConfigDev()
>>> config_obj_test.SQL_URI
‘sqlite:///D:\\databases\\webApp02\\webApp02.db’
>>> exit()

Good luck!