Tag: python


A Singleton Configuration Class in Python

Posted in Python

permalink

Overview

In this post we cover a strategy for managing configurations for programs using a Singleton pattern to create a static Config class.

This allows the user to create an instance of the Config class, pointing it to a specific config file, which it loads into memory.

The Config class provides several static methods for accessing configuration options from the config file. Here's an example of its usage:

Config('/path/to/config.json')

if Config.get_foo() == "bar":
    do_stuff()

The …



Tags:    python    programming    patterns    design patterns    registry    computer science   


Using Mock API Servers

Posted in Python

permalink

Summary

In a prior post, we covered how to write a mock API server that stored a thread as a class attribute and used it to run the server in the background by starting a thread.

However, we neglected to cover how to actually use the mock API server. So here we include some examples of how you can use the mock API server to write better tests for components that require interacting with APIs.

The MockAPIServer Class

Let's start with a recap of the mock API …



Tags:    http    server    python    mock    mocking    api    flask    web server   


Creating Mock API Servers

Posted in Python

permalink

Overview

In this post we discuss a way of mocking an API server during tests. This technique will let you create a fake API server that can respond to API calls however you want.

The technique is twofold:

  • First, we create a mock API handler that extends BaseHTTPRequestHandler, which is the built-in HTTP server class in Python. We can extend the server class to control how it responds to requests - to …



Tags:    http    server    python    mock    mocking    api    flask    web server   


Python: From Args to Kwargs

Posted in Python

permalink

Overview

In this short blog post, we talk about how and when you can take a method signature that defines input positional arguments by name, like this:

def foo(arg1, arg2, arg3):
    pass

and write code that will return a dictionary containing a keyword arguments-like structure:

>>> foo('red', 'blue', 'green')
{
    'arg1': 'red',
    'arg2': 'blue',
    'arg3': 'green'
}

We will cover an example of writing a decorator that utilizes input arguments from both the decorator and from the …



Tags:    python    programming    arguments    functions    methods    parameters   


Confuse-A-Constructor: When Class A's Constructor Returns Objects of Type B

Posted in Python

permalink

Confuse-A-Constructor

Today, we are going to confuse a constructor.

What is the constructor?

One of the first concepts encountered in object-oriented programming is that of the constructor, the method that is …




Python Patterns: The Registry

Posted in Python

permalink

Overview

This post is a summary of a useful Python programming pattern called the Registry pattern.

The Registry pattern is a way of keeping track of all subclasses of a given class.

More details about this pattern are available at https://github.com/faif/python-patterns.

What is the Registry Pattern …



Tags:    python    programming    patterns    design patterns    registry    computer science   


Incorporating Terraform Commands into Makefiles

Posted in Terraform

permalink

Summary

This blog post covers a useful pattern for incorporating terraform commands into a Makefile.

This is useful for cases where terraform is being used to manage infrastructure. In the end you will be able to run a command like

make plan-infra
make deploy-infra

and and have this call the …



Tags:    terraform    makefile    make    python   


Automatically Generating Up-To-Date requirements.txt for Python Projects

Posted in Python

permalink

Summary

In this post, we cover a pattern for automatically generating a requirements.txt file that has the latest compatible versions of required software, and that specifies the full and exact version of each package to make the Python environment reproducible.

This will turn a requirements input file (called requirements.txt.in for example) that looks like

numpy

into a requirements file that specifies the exact version of numpy and all dependencies, like

numpy==1.18 …


Tags:    python    pip    version control    make    makefile   


Mocking AWS in Unit Tests

Posted in Python

permalink

Overview

This post covers a technique for mocking AWS in unit tests so that you can test functionality that normally requires API calls and handling responses, by mocking those responses instead of making actual API calls.

A Simple Example: Mocking API Responses

The Genuine AWS Call

Let's start with an example of an AWS API call. Here's how our program will be structured: start with a driver lister.py that creates an AWS secrets manager client and defines a function to list …



Tags:    python    pytest    tests    aws    mock    mocking   


Deconvoluting Convolutional Neural Networks

Posted in Machine Learning

permalink


Approximating Pi (Happy Pi Day)

Posted in Mathematics

permalink

Favorite Pi Approximations

What's your favorite \(\pi\) approximation?

Some of my favorite approximations of \(\pi\) come from Ramanujan-Sato series. These are mathematical series that generalize from a remarkable formula for \(\pi\) given by Srinivasa Ramanujan, an Indian mathematician:

$$ \pi^{-1} = \dfrac{\sqrt{8}}{99^2} \sum_{k \geq 0} \dfrac{ (4k)! }{ \left( 4^k k! \right …


Tags:    pi    continued fractions    number theory    mathematics    python    irrational numbers   


Five Letter Words: Part 5: The Try Trie Tree

Posted in Computer Science

permalink

Tags:    python    computer science    graphs    algorithms    art of computer programming    knuth    five letter words    tries    trees   


Five Letter Words: Part 4: Revisiting Diff by One

Posted in Computer Science

permalink

About the Five-Letter Words

In Volume 4, Facsimile 0 of Donald Knuth's Art of Computer Programming, in which Knuth covers graph theory, he introduces a list of five-letter words as part of a data set useful in exploring graph theory and graph algorithms.

The list of words is part of the Stanford Graph Base, a set of data sets that are useful for studying graph theory and networks.

See Five Letter Words on the charlesreid1.com wiki for details.

Different by 1, Revisited

This post …




centillion: a document search engine

Posted in Centillion

permalink

We're excited to announce the public release of centillion, a document search engine.

centillion is a search tool that can be used by any individual or organization to index Github repositories (including the content of markdown files), Google Drive folders (including the content of .docx files), and Disqus comment threads.

centillion is tested using Travis CI.

centillion was originally written for the NIH Data Commons effort (which recently concluded). centillion was built to facilitate information-finding in a project with hundreds of people at dozens of institutions generating a sea of email threads, Google Drive folders, markdown files, websites, and Github …



Tags:    python    centillion    search    search engine    google drive    github    flask   


Any Color You Like, As Long As It's 00ADD8

Posted in Go

permalink

A short post with some thoughts on how writing Go code has helped me learn to stop worrying and love auto-formatting tools.

Go code is terse. Not Python-terse, but terse. And unlike Java, you don't find yourself constantly resorting to the security blanket of objects - something that Python (mercifully) can go either way on.

I used Java when I taught computer science at South Seattle College, and remember telling students once that one day, students taking CSC 142/143 would be using Go instead of Java. These days, I'm not as certain of that, but given that Go's strengths are …



Tags:    go    golang    rosalind    bioinformatics    black    python    gofmt   


A Few of My Favorite PEPs

Posted in Python

permalink

Table of Contents



What's your favorite PEP?

PEPs, or Python Enhancement Proposals, are documents in which features, additions, or general ideas are proposed as additions to the core Python language.

As a Python user, we believe it's important to ask questions like this.

Picking a "favorite PEP" is not just about having a ready and clever answer to a question you might expect in a technical interview; the PEP documents really are important, and really do …



Tags:    python    pep    computer science    programming   


Context Managers in Python

Posted in Python

permalink

A Predicament

Recently we spent some time contributing to dib-lab/eelpond (renamed to elvers), an executable Snakemake workflow for running the eelpond mRNAseq workflow.

In the process of tracking down a confusing bug in the Snakemake workflow, we used Snakemake's ability to print a directed acyclic graph (hereafter referred to as a dag) representing its task graph. Snakemake prints the dot …



Tags:    context managers    testing    python    programming   


Building Snakemake Command Line Wrappers for Kubernetes Workflows

Posted in Snakemake

permalink

Tags:    python    bioinformatics    workflows    pipelines    snakemake    travis    kubernetes    minikube   


Building Snakemake Command Line Wrappers for Workflows

Posted in Snakemake

permalink

NOTE: These ideas are implemented in the repository charlesreid1/2019-snakemake-cli.

Basic Idea: Wrapping Snakemake API Calls

2018-snakemake-cli

This blog post covers the implementation of an idea that was originally explored in a blog post from Titus Brown, Pydoit, snakemake, and workflows-as-applications.

That blog post implemented a basic command line wrapper around the Snakemake API to demonstrate how a Snakemake workflow could be …



Tags:    python    bioinformatics    workflows    pipelines    snakemake    travis   


First Post of the Fall, Part 2: Flaskadillo

Posted in Python

permalink

Flask + ILLO = Flaskadillo

On October 15, 2018, I had the opportunity to offer an in-lab learning opportunity (ILLO) at the Lab for Data Intensive Biology. The ILLO focused on Flask, a useful Python library for creating and running web servers. This library is useful because it has a very low learning curve, but also has the complexity to handle complicated, real-world projects.

As a part of this in-lab learning opportunity, I created repository with five simple Flask examples to highlight five useful capabilities of Flask.

The repository is called flaskadillo and it is available on git.charlesreid1.com or on …



Tags:    Github    Software    Python    Flask   


Current Projects

Posted in General

permalink

A list of various ongoing projects:

The Git College of Surgery:

Python + APIs:

  • building an API that calls APIs so you can API while you API (a webhook that calls a hook - see captain hook)
  • testing APIs with Python + requests (currently top secret, coming soon.)

Python + Command line:

  • command line utilities with python …


Tags:    Git    Github    Software    Python   


D3 Calendar Visualizations

Posted in Javascript

permalink

Table of Contents



Starting example

Let's begin with a D3 example. Mike Bostock provided a Calendar View block illustrating how to draw a very interesting visualization of large amounts of data over time:

You might recognize this type of graph from Github, whose activity graph shows the same visualization.

The data shown in this example consists of several years of stock market data. It is a simple but very large data set, with each data …



Tags:    javascript    d3    computer science    python    colors   


4x4 Rubik's Cube: Part 4: Sequence Order

Posted in Rubiks Cube

permalink

This is Part 4 of a 4-part blog post on the mathematics of the 4x4 Rubik's Cube, its relation to algorithms, and some curious properties of Rubik's Cubes.

See Part 1 of this blog post here: Part 1: Representations

See Part 2 of this blog post here: Part 2: Permutations

See Part 3 of this blog post here: Part 3: Factoring Permutations

You are currently reading Part 4 of this blog post: Part 4: Sequence Order

Table of Contents



Tags:    rubiks cube    combinatorics    permutations    python    puzzles    art of computer programming    knuth   


4x4 Rubik's Cube: Part 3: Factoring Permutations

Posted in Rubiks Cube

permalink

This is Part 3 of a 4-part blog post on the mathematics of the 4x4 Rubik's Cube, its relation to algorithms, and some curious properties of Rubik's Cubes.

See Part 1 of this blog post here: Part 1: Representations

See Part 2 of this blog post here: Part 2: Permutations

You are currently reading Part 3 of this blog post: Part 3: Factoring Permutations

See Part 4 of this blog post here: Part 4: Sequence Order

Table of Contents



Tags:    rubiks cube    combinatorics    permutations    python    puzzles    art of computer programming    knuth   


4x4 Rubik's Cube: Part 2: Permutations

Posted in Rubiks Cube

permalink

This is Part 2 of a 4-part blog post on the mathematics of the 4x4 Rubik's Cube, its relation to algorithms, and some curious properties of Rubik's Cubes.

See Part 1 of this blog post here: Part 1: Representations

You are currently reading Part 2 of this blog post: Part 2: Permutations

See Part 3 of this blog post here: Part 3: Factoring Permutations

See Part 4 of this blog post here: Part 4: Sequence Order

Table of Contents



Tags:    rubiks cube    combinatorics    permutations    python    puzzles    art of computer programming    knuth   


4x4 Rubik's Cube: Part 1: Representations

Posted in Rubiks Cube

permalink

This is Part 1 of a 4-part blog post on the mathematics of the 4x4 Rubik's Cube, its relation to algorithms, and some curious properties of Rubik's Cubes.

You are currently reading Part 1 of this blog post: Part 1: Representations

See Part 2 of this blog post here: Part 2: Permutations

See Part 3 of this blog post here: Part 3: Factoring Permutations

See Part 4 of this blog post here: Part 4: Sequence Order

Table of Contents



Tags:    rubiks cube    mathematics    combinatorics    permutations    python    puzzles   


Let's Generate Permutations!

Posted in Computer Science

permalink

Generating Permutations

In today's post we're going to discuss the generation of permutations.

Often, in combinatorics problems, we are interested in how many different instances or configurations of a particular thing we can have (what we'll call "enumeration" or "counting"). However, that is different from wanting to actually see all of those configurations. Indeed, if we are counting something with an astronomical number of configurations, we don't want to try to list all of them.

However, as usual, Donald Knuth, who covers the topic of permutation generation in Volume 4A of his classic work, The Art of Computer Programming, uncovers …




Five Letter Words: Part 3: Letter Coverage and Dynamic Programming

Posted in Computer Science

permalink

NOTE: The code covered in this post uses Python 3. The scripts can be converted to Python 2 with minimal effort, but the author would encourage any user of Python 2 to "put on your big kid pants" and make the switch to Python 3. Let's all make this painful, drawn-out switch from Python 2 to Python 3 a thing of the past, shall we?

Table of Contents

Introduction

The letter/word coverage problem, as presented by Donald Knuth in Volume 4, Facicle 0 of his masterpiece Art of …




Five Letter Words: Part 2: More Five-Word Algorithms

Posted in Computer Science

permalink

NOTE: The code covered in this post uses Python 3. The scripts can be converted to Python 2 with minimal effort, but the author would encourage any user of Python 2 to "put on your big kid pants" and make the switch to Python 3. Let's all make this painful, drawn-out switch from Python 2 to Python 3 a thing of the past, shall we?

Table of Contents




Five Letter Words: Part 1: Getting Familiar With The List

Posted in Computer Science

permalink

NOTE: The code covered in this post uses Python 3. The scripts can be converted to Python 2 with minimal effort, but the author would encourage any user of Python 2 to "put on your big kid pants" and make the switch to Python 3. Let's all make this painful, drawn-out switch from Python 2 to Python 3 a thing of the past, shall we?

Table of Contents

About the Five-Letter …




Undergraduate Research Project: Wireless Sensor Networks for Internet of Things Applications (Part 2: The Technologies)

Posted in Wireless

permalink

Undergraduate Research Project (UGR): The Technologies

In this post we'll cover some of the technologies that were used in our South Seatte College undergraduate research project. The project involved an ensemble of different technologies to complete each component of the data analysis pipeline. Some components were planned for, but other components were implemented due to "surprise" challenges that cropped up during the course of the project, while …



Tags:    wireless    security    undergraduate research project    stunnel    SSH    aircrack    mongodb    python    jupyter    linux    raspberry pi   


Undergraduate Research Project: Wireless Sensor Networks for Internet of Things Applications (Part 1: The Project)

Posted in Wireless

permalink

Table of Contents:

Overview of the Undergraduate Research (UGR) Project

South Seattle UGR Project

For the past year, in addition to my duties as a computer science and math instructor at South Seattle College, I have served as a research mentor for an NSF-funded undergraduate research project involving (off-and-on) five different South Seattle students - all of whom have expressed interest in transferring to the University of Washington's computer science …



Tags:    wireless    security    undergraduate research project    stunnel    SSH    aircrack    mongodb    python    jupyter    linux    raspberry pi   


Python vs. Perl: N Queens Problem

Posted in Computer Science

permalink

Table of Contents

Background

Revisiting the N queens problem, this time implementing the solution in Python.

Verb-oriented solution, functional, and based on Perl solution

More fair comparison - both are interpreted languages, not compiled languages

Compare Python and Perl, ease of implementation, speed, flexibility

N Queens Problem

As a recap from the …



Tags:    python    perl    java    algorithms    recursion    n-queens