Multi-Purpose Container Architecture

This document describes our approach to using a single Docker image for multiple background services in the Tracker GraphQL application.

Overview

Instead of maintaining separate Docker images for each background service, we use a multi-purpose container approach where a single Docker image can run different services based on the command parameter provided at startup.

The container can run the following services:

1. Location History Refresher (refresher): Listens for PostgreSQL notifications to refresh the location_history materialized view.
2. Tracker Report Fetcher (fetcher): Fetches and stores Apple FindMy location reports for trackers.

Implementation

The implementation consists of:

1. A unified Dockerfile that includes all dependencies
2. An entrypoint script that selects which service to run
3. AWS ECS task definitions that specify different commands

All scripts are consolidated in the fetcher/build directory for a cleaner structure.

Directory Structure

fetcher/
└── build/
    ├── Dockerfile
    ├── docker-entrypoint.sh
    ├── location_history_refresher.py
    ├── tracker_report_fetcher.py
    └── requirements.txt

Entrypoint Script

#!/bin/bash
set -e

case "$1" in
  refresher)
    exec python location_history_refresher.py
    ;;
  fetcher)
    shift  # Remove the first argument
    exec python tracker_report_fetcher.py "$@"
    ;;
  help)
    echo "Usage: docker run <image> [service] [options]"
    echo ""
    echo "Services:"
    echo "  refresher                Run the location history refresher"
    echo "  fetcher [options]        Run the tracker report fetcher with options"
    echo ""
    echo "Examples:"
    echo "  docker run <image> refresher"
    echo "  docker run <image> fetcher --interval 300 --batch-size 64"
    ;;
  *)
    echo "Unknown service: $1"
    echo "Run 'docker run <image> help' for usage information"
    exit 1
    ;;
esac

Dockerfile

FROM python:3.13-slim

WORKDIR /app

# Install system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    libpq-dev \
    curl \
    && rm -rf /var/lib/apt/lists/*

# Copy requirements first to leverage Docker cache
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
RUN pip install --no-cache-dir findmy>=0.4.0

# Copy application code
COPY . .

# Make the scripts executable
RUN chmod +x tracker_report_fetcher.py
RUN chmod +x location_history_refresher.py
RUN chmod +x docker-entrypoint.sh

# Create a non-root user and switch to it
RUN groupadd -r tracker && useradd -r -g tracker tracker
RUN chown -R tracker:tracker /app
USER tracker

# Add healthcheck
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
    CMD curl -f http://localhost:8000/health || exit 1

# Set the entrypoint
ENTRYPOINT ["/app/docker-entrypoint.sh"]

# Default command (can be overridden)
CMD ["help"]

AWS ECS Deployment

For AWS ECS deployment, we create separate task definitions for each service, all using the same container image but with different command parameters. We use an Ansible playbook to automate the deployment process.

Ansible Deployment Playbook

We use a parameterized Ansible playbook to deploy either the Tracker Report Fetcher or Location History Refresher service to Amazon ECS. The playbook accepts a service_type parameter to specify which service to deploy.

Prerequisites

• Ansible installed
• AWS CLI configured with appropriate credentials

Usage

# Deploy fetcher service (default)
ansible-playbook playbook-ecs-deploy.yml

# Deploy refresher service
ansible-playbook playbook-ecs-deploy.yml -e "service_type=refresher"

The playbook includes configurable variables for network settings (cluster name, subnet IDs, security group IDs) and service-specific configurations. See fetcher/README_ECS_DEPLOY.md for detailed documentation.

Playbook Structure

The playbook uses the community.aws modules for ECS deployment and automatically:

1. Creates the ECS cluster if it doesn't exist
2. Creates a security group with appropriate rules for the services:
   • No inbound rules (since nothing connects to these services)
   • Outbound rules for DNS (53/UDP and TCP), PostgreSQL (5432/TCP), Anisette service (6969/TCP), and HTTPS (443/TCP)

It uses a dictionary of service configurations to dynamically set the appropriate values based on the service_type parameter (which defaults to "fetcher" if not specified):

service_configs:
  fetcher:
    service_name: tracker-report-fetcher-service
    family: tracker-report-fetcher
    container_name: tracker-report-fetcher
    image: 951665295205.dkr.ecr.eu-west-2.amazonaws.com/tracker/fetcher:latest
    command: [fetcher]
    log_group: /ecs/tracker-report-fetcher
    secret_prefix: tracker-fetcher-parameters
  refresher:
    service_name: location-history-refresher-service
    family: location-history-refresher
    container_name: location-history-refresher
    image: 951665295205.dkr.ecr.eu-west-2.amazonaws.com/tracker/refresher:latest
    command: [refresher]
    log_group: /ecs/location-history-refresher
    secret_prefix: tracker-refresher-parameters

This approach makes it easy to maintain both service configurations in a single file while still allowing for service-specific settings.

Using AWS Secrets Manager

Instead of hardcoding sensitive information like database credentials and API keys in the task definition, we can use AWS Secrets Manager to securely store and manage these parameters.

Creating a Secret Using the AWS Console

1. Navigate to the AWS Secrets Manager console
2. Click "Store a new secret"
3. Select "Other type of secret"
4. Add key-value pairs for each parameter:

DB_HOST: your-rds-endpoint
DB_PORT: 5432
DB_NAME: postgres
DB_USER: postgres
DB_PASSWORD: your-secure-password
ANISETTE_SERVER: https://your-anisette-server
REDIS_HOST: your-redis-endpoint
REDIS_PORT: 6379
REDIS_PASSWORD: your-redis-password

5. Click "Next"
6. Name your secret (e.g., tracker-fetcher-parameters)
7. Add a description and tags if needed
8. Click "Next" and then "Store"

Creating a Secret Using the AWS CLI

You can also create secrets using the AWS CLI, which is useful for automation and CI/CD pipelines:

1. First, create a JSON file with the secret values:

cat > secret-values.json << EOF
{
  "DB_HOST": "your-rds-endpoint",
  "DB_PORT": "5432",
  "DB_NAME": "postgres",
  "DB_USER": "postgres",
  "DB_PASSWORD": "your-secure-password",
  "ANISETTE_SERVER": "https://your-anisette-server",
  "REDIS_HOST": "your-redis-endpoint",
  "REDIS_PORT": "6379",
  "REDIS_PASSWORD": "your-redis-password"
}
EOF

2. Then use the aws secretsmanager create-secret command:

aws secretsmanager create-secret \
    --name tracker-fetcher-parameters \
    --description "Parameters for the tracker fetcher container" \
    --secret-string file://secret-values.json \
    --tags Key=Environment,Value=Production

3. To update an existing secret:

aws secretsmanager update-secret \
    --secret-id tracker-fetcher-parameters \
    --secret-string file://secret-values.json

Referencing the Secret in Task Definitions

Once the secret is created, you can reference it in your task definitions. Note that we're using the full ARN format with specific parameter names to clearly indicate which parameter is being accessed:

{
  "containerDefinitions": [
    {
      "name": "tracker-report-fetcher",
      "image": "951665295205.dkr.ecr.eu-west-2.amazonaws.com/tracker/fetcher:latest",
      "command": [
        "fetcher",
        "--interval",
        "300",
        "--batch-size",
        "64",
        "--max-age",
        "7"
      ],
      "secrets": [
        {
          "name": "DB_HOST",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:DB_HOST::"
        },
        {
          "name": "DB_PORT",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:DB_PORT::"
        },
        {
          "name": "DB_NAME",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:DB_NAME::"
        },
        {
          "name": "DB_USER",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:DB_USER::"
        },
        {
          "name": "DB_PASSWORD",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:DB_PASSWORD::"
        },
        {
          "name": "ANISETTE_SERVER",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:ANISETTE_SERVER::"
        },
        {
          "name": "REDIS_HOST",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:REDIS_HOST::"
        },
        {
          "name": "REDIS_PORT",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:REDIS_PORT::"
        },
        {
          "name": "REDIS_PASSWORD",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:REDIS_PASSWORD::"
        }
      ]
      // ... other container definition properties
    }
  ]
  // ... other task definition properties
}

Required IAM Permissions

The ECS task execution role needs permission to read the secret. Add the following policy to your ecsTaskExecutionRole:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["secretsmanager:GetSecretValue"],
      "Resource": [
        "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters-*"
      ]
    }
  ]
}

Benefits of Using Secrets Manager

1. Security: Sensitive information is not stored in task definitions or environment variables
2. Centralized Management: Update credentials in one place
3. Rotation: Automatically rotate credentials on a schedule
4. Audit: Track who accessed secrets and when
5. Version Control: Roll back to previous versions if needed

Task Definition Examples

Location History Refresher with Environment Variables

{
  "family": "location-history-refresher",
  "containerDefinitions": [
    {
      "name": "location-history-refresher",
      "image": "951665295205.dkr.ecr.eu-west-2.amazonaws.com/tracker/fetcher:latest",
      "command": ["refresher"],
      "essential": true,
      "environment": [
        { "name": "DB_HOST", "value": "your-rds-endpoint" },
        { "name": "DB_PORT", "value": "5432" },
        { "name": "DB_NAME", "value": "postgres" },
        { "name": "DB_USER", "value": "postgres" },
        { "name": "DB_PASSWORD", "value": "postgres" }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/location-history-refresher",
          "awslogs-region": "eu-west-2",
          "awslogs-stream-prefix": "ecs"
        }
      }
    }
  ],
  "executionRoleArn": "arn:aws:iam::951665295205:role/ecsTaskExecutionRole",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "256",
  "memory": "512"
}

Location History Refresher with Secrets

{
  "family": "location-history-refresher",
  "containerDefinitions": [
    {
      "name": "location-history-refresher",
      "image": "951665295205.dkr.ecr.eu-west-2.amazonaws.com/tracker/fetcher:latest",
      "command": ["refresher"],
      "essential": true,
      "secrets": [
        {
          "name": "DB_HOST",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:DB_HOST::"
        },
        {
          "name": "DB_PORT",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:DB_PORT::"
        },
        {
          "name": "DB_NAME",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:DB_NAME::"
        },
        {
          "name": "DB_USER",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:DB_USER::"
        },
        {
          "name": "DB_PASSWORD",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:DB_PASSWORD::"
        }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/location-history-refresher",
          "awslogs-region": "eu-west-2",
          "awslogs-stream-prefix": "ecs"
        }
      }
    }
  ],
  "executionRoleArn": "arn:aws:iam::951665295205:role/ecsTaskExecutionRole",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "256",
  "memory": "512"
}

Tracker Report Fetcher with Environment Variables

{
  "family": "tracker-report-fetcher",
  "containerDefinitions": [
    {
      "name": "tracker-report-fetcher",
      "image": "951665295205.dkr.ecr.eu-west-2.amazonaws.com/tracker/fetcher:latest",
      "command": [
        "fetcher",
        "--interval",
        "300",
        "--batch-size",
        "64",
        "--max-age",
        "7"
      ],
      "essential": true,
      "environment": [
        { "name": "ANISETTE_SERVER", "value": "https://your-anisette-server" },
        { "name": "DB_HOST", "value": "your-rds-endpoint" },
        { "name": "DB_PORT", "value": "5432" },
        { "name": "DB_NAME", "value": "postgres" },
        { "name": "DB_USER", "value": "postgres" },
        { "name": "DB_PASSWORD", "value": "postgres" },
        { "name": "REDIS_HOST", "value": "your-redis-endpoint" },
        { "name": "REDIS_PORT", "value": "6379" },
        { "name": "ACCOUNT_STORE", "value": "/data/account.json" }
      ],
      "mountPoints": [
        {
          "sourceVolume": "data-volume",
          "containerPath": "/data",
          "readOnly": false
        }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/tracker-report-fetcher",
          "awslogs-region": "eu-west-2",
          "awslogs-stream-prefix": "ecs"
        }
      }
    }
  ],
  "volumes": [
    {
      "name": "data-volume",
      "efsVolumeConfiguration": {
        "fileSystemId": "your-efs-id",
        "rootDirectory": "/"
      }
    }
  ],
  "executionRoleArn": "arn:aws:iam::951665295205:role/ecsTaskExecutionRole",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "512",
  "memory": "1024"
}

Tracker Report Fetcher with Secrets

{
  "family": "tracker-report-fetcher",
  "containerDefinitions": [
    {
      "name": "tracker-report-fetcher",
      "image": "951665295205.dkr.ecr.eu-west-2.amazonaws.com/tracker/fetcher:latest",
      "command": [
        "fetcher",
        "--interval",
        "300",
        "--batch-size",
        "64",
        "--max-age",
        "7"
      ],
      "essential": true,
      "secrets": [
        {
          "name": "ANISETTE_SERVER",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:ANISETTE_SERVER::"
        },
        {
          "name": "DB_HOST",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:DB_HOST::"
        },
        {
          "name": "DB_PORT",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:DB_PORT::"
        },
        {
          "name": "DB_NAME",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:DB_NAME::"
        },
        {
          "name": "DB_USER",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:DB_USER::"
        },
        {
          "name": "DB_PASSWORD",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:DB_PASSWORD::"
        },
        {
          "name": "REDIS_HOST",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:REDIS_HOST::"
        },
        {
          "name": "REDIS_PORT",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:REDIS_PORT::"
        },
        {
          "name": "REDIS_PASSWORD",
          "valueFrom": "arn:aws:secretsmanager:eu-west-2:951665295205:secret:tracker-fetcher-parameters:REDIS_PASSWORD::"
        }
      ],
      "environment": [
        { "name": "ACCOUNT_STORE", "value": "/data/account.json" }
      ],
      "mountPoints": [
        {
          "sourceVolume": "data-volume",
          "containerPath": "/data",
          "readOnly": false
        }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/tracker-report-fetcher",
          "awslogs-region": "eu-west-2",
          "awslogs-stream-prefix": "ecs"
        }
      }
    }
  ],
  "volumes": [
    {
      "name": "data-volume",
      "efsVolumeConfiguration": {
        "fileSystemId": "your-efs-id",
        "rootDirectory": "/"
      }
    }
  ],
  "executionRoleArn": "arn:aws:iam::951665295205:role/ecsTaskExecutionRole",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "512",
  "memory": "1024"
}

Benefits

This approach offers several advantages:

1. Simplified CI/CD pipeline: Only one image to build, test, and push
2. Efficient resource utilization: Each ECS task runs exactly one script with its own resource allocation
3. Independent scaling: You can scale each service independently by adjusting the desired count of each ECS task
4. Isolated execution: Each container instance runs independently, maintaining proper isolation
5. Consistent environment: All services use the same base image, ensuring dependency consistency
6. Simplified updates: When you need to update shared code, you only update one image

Local Development

For local development and testing, you can use Docker Compose:

---
services:
  location-history-refresher:
    build:
      context: ./build
      dockerfile: Dockerfile
    image: tracker-services:latest
    command: refresher
    restart: unless-stopped
    environment:
      - DB_HOST=${DB_HOST:-postgres}
      - DB_PORT=${DB_PORT:-5432}
      - DB_NAME=${DB_NAME:-postgres}
      - DB_USER=${DB_USER:-postgres}
      - DB_PASSWORD=${DB_PASSWORD:-postgres}
    deploy:
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3
        window: 120s

  tracker-report-fetcher:
    build:
      context: ./build
      dockerfile: Dockerfile
    image: tracker-services:latest
    command: fetcher --interval ${FETCH_INTERVAL:-300} --batch-size ${MAX_KEYS_PER_BATCH:-64} --max-age ${MAX_REPORT_AGE_DAYS:-7}
    restart: unless-stopped
    environment:
      - ANISETTE_SERVER=${ANISETTE_SERVER:-http://anisette:6969}
      - DB_HOST=${DB_HOST:-postgres}
      - DB_PORT=${DB_PORT:-5432}
      - DB_NAME=${DB_NAME:-postgres}
      - DB_USER=${DB_USER:-postgres}
      - DB_PASSWORD=${DB_PASSWORD:-postgres}
      - REDIS_HOST=${REDIS_HOST:-redis}
      - REDIS_PORT=${REDIS_PORT:-6379}
      - REDIS_PASSWORD=${REDIS_PASSWORD:-}
      - REDIS_DB=${REDIS_DB:-0}
      - FETCH_INTERVAL=${FETCH_INTERVAL:-300}
      - MAX_KEYS_PER_BATCH=${MAX_KEYS_PER_BATCH:-64}
      - MAX_REPORT_AGE_DAYS=${MAX_REPORT_AGE_DAYS:-7}
      - REQUEST_INTERVAL_HOURS=${REQUEST_INTERVAL_HOURS:-6}
      - ACCOUNT_STORE=/data/account.json
    volumes:
      - ./data:/data
    deploy:
      replicas: ${FETCHER_REPLICAS:-2}
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3
        window: 120s

Troubleshooting

If you encounter issues:

1. Check the container logs:

   docker logs <container-id>

2. Run the container in interactive mode:

   docker run -it --entrypoint /bin/bash tracker-services:latest

3. Verify the scripts are executable:

   ls -la /app/location_history_refresher.py
   ls -la /app/tracker_report_fetcher.py