2021-06-05 20:17:08+00:00

When maintaining a multi-service architecture on Google App Engine (GAE), microservices are typically structured into independent directories under a single repository. A common anti-pattern in these setups is when a microservice imports helper modules directly from a peer microservice's directory (e.g., service_a importing from ../service_b/utils.py) instead of using a shared utility library. While standard linters cannot detect these boundary breaches, we can write custom static analysis rules to enforce architectural constraints.

By leveraging Python's Abstract Syntax Tree (AST) and extending Pylint's class checkers, we can programmatically block invalid cross-service imports.


1. Writing a Custom AST Import Checker

We write a custom Python script that utilizes the built-in ast module to inspect import statements and flag imports referencing neighbor directories:

# boundary_linter.py
import ast
import os
import sys

class ImportBoundaryChecker(ast.NodeVisitor):
    def __init__(self, current_file):
        self.current_file = current_file
        # Extract the service directory name (e.g. 'service_a')
        self.service_dir = current_file.split(os.sep)[-2]

    def visit_ImportFrom(self, node):
        # We analyze relative imports and module level names
        if node.level > 1:
            print(f"ERROR: Relative import level {node.level} detected in {self.current_file}")
            sys.exit(1)
        
        if node.module:
            parts = node.module.split('.')
            # Check if importing from another service package directory
            if any(p.startswith('service_') and p != self.service_dir for p in parts):
                print(f"ERROR: Cross-service import of '{node.module}' in {self.current_file}")
                sys.exit(1)
        self.generic_visit(node)

def run_boundary_check(filepath):
    with open(filepath, 'r') as f:
        tree = ast.parse(f.read(), filename=filepath)
    checker = ImportBoundaryChecker(filepath)
    checker.visit(tree)

if __name__ == "__main__":
    for path in sys.argv[1:]:
        if path.endswith('.py'):
            run_boundary_check(path)

2. Extending Pylint with Custom Checker Plugins

To integrate this boundary check into our standard Pylint workflow, we package the custom logic as a Pylint plugin. We register a new checker class that listens for import tokens:

# pylint_boundary_plugin.py
from pylint.checkers import BaseChecker
from pylint.interfaces import IAstroidChecker

class ServiceBoundaryChecker(BaseChecker):
    __implements__ = IAstroidChecker
    name = 'service-boundary-checker'
    
    msgs = {
        'E9901': (
            'Import from peer microservice directory is forbidden',
            'cross-service-import',
            'Decouple microservices by utilizing shared libraries instead.'
        ),
    }

    def visit_importfrom(self, node):
        # Access AST tokens programmatically via Astroid
        module_name = node.modname
        if 'service_' in module_name and not module_name.startswith(self.current_service):
            self.add_message('E9901', node=node)

def register(linter):
    linter.register_checker(ServiceBoundaryChecker(linter))

3. Benefits of Enforcing Import Isolation

Enforcing import isolation at the static analysis stage ensures that service boundaries are maintained. This allows teams to deploy or refactor individual microservices independently without risk of running into unexpected runtime import crashes on App Engine production instances.