2021-10-22 21:25:42+00:00

Google App Engine (GAE) applications structured with multiple services (e.g., individual YAML configurations for service-a, service-b, etc.) run in completely isolated sandboxes. In these setups, importing shared utilities—such as user authentication checks, custom exception handlers, or telemetry logging helpers—is problematic. If a developer copies shared files into each service folder, updates become difficult to synchronize. A clean approach organizes common helpers into a central directory and imports them dynamically during execution.

By modifying Python's search path dynamically or copying includes during the build phase, we ensure that microservices share code cleanly.


1. Modifying Python Search Paths at Runtime

When running local test suites or running GAE stubs, developers can import modules by modifying sys.path before executing imports. We write a bootloader file to include the shared folder:

# bootloader.py
import sys
import os

def load_shared_libraries():
    # Navigate up to find the central shared directory
    current_dir = os.path.dirname(os.path.abspath(__file__))
    shared_path = os.path.abspath(os.path.join(current_dir, '../shared_includes'))
    
    if shared_path not in sys.path:
        sys.path.insert(0, shared_path)
        print(f"Injected shared path to sys.path: {shared_path}")

# Load paths before importing core modules
load_shared_libraries()
import db_connector
import auth_middleware

2. Automating Code Bundling in Deploy Scripts

App Engine deployment commands require that all imported source files reside inside the directory containing the app.yaml file. We write a release script that copies global includes prior to launching deployment commands, ensuring a clean and self-contained deployment context:

# deploy_service.py
import shutil
import os
import subprocess

def prepare_and_deploy(service_name):
    source_includes = "../shared_includes"
    dest_includes = f"./{service_name}/shared_includes"
    
    # Remove old copies if they exist
    if os.path.exists(dest_includes):
        shutil.rmtree(dest_includes)
        
    print(f"Copying global includes to {service_name}...")
    shutil.copytree(source_includes, dest_includes)
    
    # Run deployment
    try:
        cmd = ["gcloud", "app", "deploy", f"{service_name}/app.yaml", "--quiet"]
        subprocess.run(cmd, check=True)
    finally:
        # Clean up copied files to prevent dirtying git repositories
        print("Cleaning up build folders...")
        shutil.rmtree(dest_includes)

if __name__ == "__main__":
    prepare_and_deploy("analytics-service")

3. Enhancing Local Development Iterations

Using symbolic links during local development allows developers to edit shared code files and see results immediately across all running services without manually re-triggering file copying, maintaining fast execution loops.