GCP Professional Cloud Architect: Google Cloud App Engine Global Includes: Managing Common Imports

2021-10-22 21:25:42+00:00

Google App Engine (GAE) applications structured with multiple services (e.g., individual YAML configurations for service-a, service-b, etc.) run in completely isolated sandboxes. In these setups, importing shared utilities—such as user authentication checks, custom exception handlers, or telemetry logging helpers—is problematic. If a developer copies shared files into each service folder, updates become difficult to synchronize. A clean approach organizes common helpers into a central directory and imports them dynamically during execution.

By modifying Python's search path dynamically or copying includes during the build phase, we ensure that microservices share code cleanly.


1. Modifying Python Search Paths at Runtime

When running local test suites or running GAE stubs, developers can import modules by modifying sys.path before executing imports. We write a bootloader file to include the shared folder:

# bootloader.py
import sys
import os

def load_shared_libraries():
    # Navigate up to find the central shared directory
    current_dir = os.path.dirname(os.path.abspath(__file__))
    shared_path = os.path.abspath(os.path.join(current_dir, '../shared_includes'))
    
    if shared_path not in sys.path:
        sys.path.insert(0, shared_path)
        print(f"Injected shared path to sys.path: {shared_path}")

# Load paths before importing core modules
load_shared_libraries()
import db_connector
import auth_middleware

2. Automating Code Bundling in Deploy Scripts

App Engine deployment commands require that all imported source files reside inside the directory containing the app.yaml file. We write a release script that copies global includes prior to launching deployment commands, ensuring a clean and self-contained deployment context:

# deploy_service.py
import shutil
import os
import subprocess

def prepare_and_deploy(service_name):
    source_includes = "../shared_includes"
    dest_includes = f"./{service_name}/shared_includes"
    
    # Remove old copies if they exist
    if os.path.exists(dest_includes):
        shutil.rmtree(dest_includes)
        
    print(f"Copying global includes to {service_name}...")
    shutil.copytree(source_includes, dest_includes)
    
    # Run deployment
    try:
        cmd = ["gcloud", "app", "deploy", f"{service_name}/app.yaml", "--quiet"]
        subprocess.run(cmd, check=True)
    finally:
        # Clean up copied files to prevent dirtying git repositories
        print("Cleaning up build folders...")
        shutil.rmtree(dest_includes)

if __name__ == "__main__":
    prepare_and_deploy("analytics-service")

3. Enhancing Local Development Iterations

Using symbolic links during local development allows developers to edit shared code files and see results immediately across all running services without manually re-triggering file copying, maintaining fast execution loops.

Memory Profiling & Leak Detection in Long-Running Python Daemons

Use tracemalloc snapshots and objgraph reference graphs to eliminate cyclic references and C-extension memory growth on constrained 1GB RAM edge devices.


About the Author — Hire Aldian Fazrihady (Cloud Engineer, GCP Professional Cloud Architect, GCP Professional Data Engineer, Backend, AI & Full Stack Engineer)

Aldian Fazrihady is a Top-Rated Senior Cloud Engineer, GCP Professional Cloud Architect (Google Cloud Certified), GCP Professional Data Engineer (Google Cloud Certified), Backend Engineer (Python, Django, FastAPI, Go, PostgreSQL), AI Application Engineer (LLMs, RAG, MCP, Google ADK, LangGraph), and Full Stack Engineer (React, Next.js, TypeScript) with 15+ years of production software engineering experience.

Hire Aldian Fazrihady on Upwork | Book 1-Day Programming Service ($100 USD / Rp 1.000.000) | View Live AI Portfolio | Send a Direct Message

Related Engineering Articles

Engineering Role Pillar Guides & Category Hubs