Przejdź do treści

🏆 Best Practices: Glue Code

Establishing robust patterns and practices for integrating shell scripts with Infrastructure as Code tools to create maintainable, secure, and scalable automation workflows.


🎯 Core Principles

1. Separation of Concerns

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# ❌ Anti-pattern: Mixed responsibilities
#!/bin/bash
# This script does everything - provisioning, configuration, and application deployment
terraform apply
ansible-playbook configure.yml
kubectl apply -f app-manifests/
docker build -t myapp .
docker push myapp:latest

# ✅ Best practice: Clear separation
# terraform/main.tf        # Infrastructure provisioning
# ansible/configure.yml    # System configuration
# k8s/app-deploy.yml       # Application deployment
# scripts/build-image.sh   # Image building
# scripts/deploy-app.sh    # Application deployment orchestration

2. Idempotency Enforcement

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
# idempotent-operations.sh - Ensuring repeatable operations
#!/bin/bash
set -euo pipefail

# Idempotent directory creation
create_directory_idempotent() {
    local dir_path="$1"
    local permissions="${2:-755}"

    if [ ! -d "$dir_path" ]; then
        echo "Creating directory: $dir_path"
        mkdir -p "$dir_path"
        chmod "$permissions" "$dir_path"
    else
        echo "Directory already exists: $dir_path"
        # Ensure correct permissions
        current_perms=$(stat -c "%a" "$dir_path")
        if [ "$current_perms" != "$permissions" ]; then
            echo "Fixing permissions for $dir_path: $current_perms -> $permissions"
            chmod "$permissions" "$dir_path"
        fi
    fi
}

# Idempotent file creation
create_file_idempotent() {
    local file_path="$1"
    local content="$2"
    local permissions="${3:-644}"

    if [ ! -f "$file_path" ] || [ "$(cat "$file_path")" != "$content" ]; then
        echo "Creating/updating file: $file_path"
        echo "$content" > "$file_path"
        chmod "$permissions" "$file_path"
    else
        echo "File already exists with correct content: $file_path"
    fi
}

# Idempotent service management
ensure_service_state() {
    local service_name="$1"
    local desired_state="$2"  # running, stopped, enabled, disabled

    case "$desired_state" in
        running)
            if ! systemctl is-active --quiet "$service_name"; then
                echo "Starting service: $service_name"
                systemctl start "$service_name"
            else
                echo "Service already running: $service_name"
            fi
            ;;
        stopped)
            if systemctl is-active --quiet "$service_name"; then
                echo "Stopping service: $service_name"
                systemctl stop "$service_name"
            else
                echo "Service already stopped: $service_name"
            fi
            ;;
        enabled)
            if ! systemctl is-enabled --quiet "$service_name"; then
                echo "Enabling service: $service_name"
                systemctl enable "$service_name"
            else
                echo "Service already enabled: $service_name"
            fi
            ;;
        disabled)
            if systemctl is-enabled --quiet "$service_name"; then
                echo "Disabling service: $service_name"
                systemctl disable "$service_name"
            else
                echo "Service already disabled: $service_name"
            fi
            ;;
    esac
}

🛡️ Security Best Practices

Secure Credential Handling

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
# secure-credentials.sh - Safe credential management
#!/bin/bash
set -euo pipefail

# Use secure temporary files
create_secure_temp_file() {
    local suffix="${1:-}"
    local temp_file

    temp_file=$(mktemp "${TMPDIR:-/tmp}/secure_temp_XXXXXX${suffix}" 2>/dev/null) || {
        echo "Failed to create secure temporary file" >&2
        return 1
    }

    # Set restrictive permissions
    chmod 600 "$temp_file" || {
        rm -f "$temp_file"
        echo "Failed to set secure permissions" >&2
        return 1
    }

    echo "$temp_file"
    return 0
}

# Secure credential retrieval
get_secure_credential() {
    local credential_name="$1"
    local credential_value=""

    # Try multiple secure sources
    if [ -f "/run/secrets/$credential_name" ]; then
        # Docker secrets
        credential_value=$(cat "/run/secrets/$credential_name")
    elif [ -f "/var/run/secrets/kubernetes.io/serviceaccount/token" ] && [ "$credential_name" = "k8s-token" ]; then
        # Kubernetes service account token
        credential_value=$(cat "/var/run/secrets/kubernetes.io/serviceaccount/token")
    elif command -v aws >/dev/null 2>&1 && [ "$credential_name" = "aws-secret" ]; then
        # AWS Secrets Manager
        credential_value=$(aws secretsmanager get-secret-value --secret-id "$credential_name" --query SecretString --output text 2>/dev/null)
    elif [ -n "${!credential_name:-}" ]; then
        # Environment variable (least secure)
        credential_value="${!credential_name}"
    else
        echo "Credential not found: $credential_name" >&2
        return 1
    fi

    if [ -n "$credential_value" ]; then
        echo "$credential_value"
        return 0
    else
        echo "Failed to retrieve credential: $credential_name" >&2
        return 1
    fi
}

# Secure credential usage with automatic cleanup
use_secure_credential() {
    local credential_name="$1"
    local usage_function="$2"

    local credential_value
    credential_value=$(get_secure_credential "$credential_name") || return $?

    # Create secure temporary file for credential
    local temp_cred_file
    temp_cred_file=$(create_secure_temp_file ".cred") || return $?

    # Write credential to secure file
    echo "$credential_value" > "$temp_cred_file"

    # Execute function with credential file
    local result=0
    "$usage_function" "$temp_cred_file" || result=$?

    # Secure cleanup
    rm -f "$temp_cred_file"

    return $result
}

Input Validation and Sanitization

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
# input-validation.sh - Robust input handling
#!/bin/bash
set -euo pipefail

# Validate required parameters
validate_required_parameter() {
    local param_name="$1"
    local param_value="${!param_name:-}"

    if [ -z "$param_value" ]; then
        echo "Error: Required parameter $param_name is not set" >&2
        return 1
    fi

    # Validate parameter format
    case "$param_name" in
        EMAIL)
            if ! echo "$param_value" | grep -E '^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$' >/dev/null; then
                echo "Error: Invalid email format: $param_value" >&2
                return 1
            fi
            ;;
        URL)
            if ! echo "$param_value" | grep -E '^https?://[a-zA-Z0-9.-]+(/[a-zA-Z0-9._-]*)*$' >/dev/null; then
                echo "Error: Invalid URL format: $param_value" >&2
                return 1
            fi
            ;;
        PORT)
            if ! echo "$param_value" | grep -E '^[0-9]+$' >/dev/null || [ "$param_value" -lt 1 ] || [ "$param_value" -gt 65535 ]; then
                echo "Error: Invalid port number: $param_value" >&2
                return 1
            fi
            ;;
        FILE_PATH)
            # Prevent path traversal
            if echo "$param_value" | grep -E '(\.\./)|(\.\.\\)' >/dev/null; then
                echo "Error: Path traversal attempt detected: $param_value" >&2
                return 1
            fi

            # Validate allowed characters
            if echo "$param_value" | grep -E '[^a-zA-Z0-9._/-]' >/dev/null; then
                echo "Error: Invalid characters in file path: $param_value" >&2
                return 1
            fi
            ;;
    esac

    echo "Parameter $param_name validated successfully"
    return 0
}

# Safe command construction
construct_safe_command() {
    local base_command="$1"
    shift
    local safe_args=()

    # Validate and sanitize each argument
    for arg in "$@"; do
        # Prevent command injection
        if echo "$arg" | grep -E '[;&|`$()]' >/dev/null; then
            echo "Error: Dangerous characters detected in argument: $arg" >&2
            return 1
        fi

        # Escape special characters
        safe_arg=$(printf '%q' "$arg")
        safe_args+=("$safe_arg")
    done

    # Construct safe command
    echo "$base_command ${safe_args[*]}"
    return 0
}

📋 Error Handling and Logging

Comprehensive Error Handling

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
# error-handling.sh - Robust error management
#!/bin/bash
set -euo pipefail

# Global error tracking
ERROR_COUNT=0
MAX_ERRORS=10

# Error logging with context
log_error() {
    local error_message="$1"
    local error_code="${2:-1}"
    local timestamp
    timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ")

    echo "[$timestamp] ERROR: $error_message (Code: $error_code)" >&2

    # Increment error counter
    ERROR_COUNT=$((ERROR_COUNT + 1))

    # Check error threshold
    if [ "$ERROR_COUNT" -gt "$MAX_ERRORS" ]; then
        echo "[$timestamp] FATAL: Maximum error threshold exceeded ($MAX_ERRORS)" >&2
        exit 1
    fi

    return "$error_code"
}

# Warning logging
log_warning() {
    local warning_message="$1"
    local timestamp
    timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ")

    echo "[$timestamp] WARNING: $warning_message" >&2
}

# Info logging
log_info() {
    local info_message="$1"
    local timestamp
    timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ")

    echo "[$timestamp] INFO: $info_message"
}

# Debug logging (conditional)
log_debug() {
    if [ "${DEBUG:-false}" = "true" ]; then
        local debug_message="$1"
        local timestamp
        timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ")

        echo "[$timestamp] DEBUG: $debug_message"
    fi
}

# Safe command execution with error handling
execute_safe() {
    local command="$1"
    local description="${2:-$command}"

    log_info "Executing: $description"
    log_debug "Command: $command"

    if eval "$command"; then
        log_info "Successfully executed: $description"
        return 0
    else
        local exit_code=$?
        log_error "Failed to execute: $description" "$exit_code"
        return "$exit_code"
    fi
}

# Retry logic with exponential backoff
retry_with_backoff() {
    local command="$1"
    local max_attempts="${2:-3}"
    local base_delay="${3:-1}"

    local attempt=1
    local delay="$base_delay"

    while [ $attempt -le $max_attempts ]; do
        log_info "Attempt $attempt/$max_attempts: $command"

        if eval "$command"; then
            log_info "Command succeeded on attempt $attempt"
            return 0
        else
            local exit_code=$?

            if [ $attempt -eq $max_attempts ]; then
                log_error "Command failed after $max_attempts attempts" "$exit_code"
                return "$exit_code"
            fi

            log_warning "Command failed (exit code: $exit_code), retrying in $delay seconds..."
            sleep "$delay"

            # Exponential backoff
            delay=$((delay * 2))
            attempt=$((attempt + 1))
        fi
    done
}

Structured Logging

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
# structured-logging.sh - JSON and structured logging
#!/bin/bash
set -euo pipefail

# JSON logging function
log_json() {
    local level="$1"
    local message="$2"
    shift 2
    local timestamp
    timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ")

    # Build JSON object
    local json="{"
    json+="\"timestamp\":\"$timestamp\","
    json+="\"level\":\"$level\","
    json+="\"message\":\"$message\","
    json+="\"pid\":$$,"

    # Add additional fields
    while [ $# -gt 0 ]; do
        local key="$1"
        local value="$2"
        shift 2
        json+="\"$key\":\"$value\","
    done

    # Remove trailing comma and close object
    json="${json%,}}}"

    echo "$json"
}

# Structured error logging
log_error_json() {
    local message="$1"
    local error_code="${2:-1}"
    shift 2

    log_json "ERROR" "$message" \
        "error_code" "$error_code" \
        "$@"
}

# Structured info logging
log_info_json() {
    local message="$1"
    shift

    log_json "INFO" "$message" "$@"
}

# Log to multiple destinations
log_multi_dest() {
    local message="$1"

    # Log to stdout
    echo "$message"

    # Log to file if specified
    if [ -n "${LOG_FILE:-}" ] && [ -w "${LOG_FILE%/*}" ]; then
        echo "$message" >> "$LOG_FILE"
    fi

    # Log to syslog if available
    if command -v logger >/dev/null 2>&1; then
        echo "$message" | logger -t "glue-code" -p user.info
    fi
}

🔄 Integration Patterns

Standard Integration Framework

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
# integration-framework.sh - Reusable integration patterns
#!/bin/bash
set -euo pipefail

# IaC tool abstraction
class IaCTool {
    private tool_name: string;
    private tool_version: string;

    constructor(tool_name: string) {
        this.tool_name = tool_name;
        this.tool_version = this.get_tool_version();
    }

    private get_tool_version(): string {
        try {
            const result = child_process.execSync(`${this.tool_name} --version`, {
                encoding: 'utf8',
                timeout: 10000
            });
            return result.trim();
        } catch (error) {
            throw new Error(`Failed to get ${this.tool_name} version: ${error}`);
        }
    }

    async execute_command(subcommand: string, args: string[] = []): Promise<{ stdout: string; stderr: string }> {
        const full_command = `${this.tool_name} ${subcommand} ${args.join(' ')}`;

        return new Promise((resolve, reject) => {
            const child = child_process.exec(full_command, { timeout: 300000 }, (error, stdout, stderr) => {
                if (error) {
                    reject(new Error(`Command failed: ${stderr || error.message}`));
                } else {
                    resolve({ stdout: stdout.trim(), stderr: stderr.trim() });
                }
            });
        });
    }

    async validate_prerequisites(): Promise<boolean> {
        // Check if tool is installed
        try {
            await this.execute_command('--version');
            return true;
        } catch (error) {
            console.error(`Prerequisite check failed for ${this.tool_name}: ${error}`);
            return false;
        }
    }
}

// Terraform integration
class TerraformIntegration extends IaCTool {
    constructor() {
        super('terraform');
    }

    async initialize_backend(config_path: string): Promise<void> {
        await this.execute_command('init', ['-backend-config', config_path]);
    }

    async plan(variables: Record<string, string>): Promise<void> {
        const var_args = Object.entries(variables).map(([key, value]) => `-var=${key}=${value}`);
        await this.execute_command('plan', [...var_args, '-out=plan.out']);
    }

    async apply(auto_approve: boolean = false): Promise<void> {
        const args = auto_approve ? ['-auto-approve'] : [];
        await this.execute_command('apply', [...args, 'plan.out']);
    }
}

// Ansible integration
class AnsibleIntegration extends IaCTool {
    constructor() {
        super('ansible-playbook');
    }

    async run_playbook(playbook_path: string, inventory: string, extra_vars: Record<string, string> = {}): Promise<void> {
        const var_args = Object.entries(extra_vars).map(([key, value]) => `-e ${key}=${value}`);
        const args = [
            '-i', inventory,
            ...var_args,
            playbook_path
        ];

        await this.execute_command('', args);
    }
}

// Kubernetes integration
class KubernetesIntegration extends IaCTool {
    constructor() {
        super('kubectl');
    }

    async apply_manifest(manifest_path: string, namespace?: string): Promise<void> {
        const args = namespace ? ['-n', namespace, 'apply', '-f', manifest_path] : ['apply', '-f', manifest_path];
        await this.execute_command('', args);
    }

    async wait_for_ready(resource_type: string, resource_name: string, namespace?: string): Promise<void> {
        const args = namespace ?
            ['-n', namespace, 'wait', '--for=condition=ready', resource_type, resource_name, '--timeout=300s'] :
            ['wait', '--for=condition=ready', resource_type, resource_name, '--timeout=300s'];

        await this.execute_command('', args);
    }
}

Cross-Tool Orchestration

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
# cross-tool-orchestration.sh - Multi-tool workflow management
#!/bin/bash
set -euo pipefail

# Workflow manager
class WorkflowManager {
    private tools: Map<string, IaCTool> = new Map();
    private results: Map<string, any> = new Map();

    register_tool(name: string, tool: IaCTool): void {
        this.tools.set(name, tool);
    }

    async validate_all_prerequisites(): Promise<boolean> {
        console.log('Validating prerequisites for all registered tools...');

        for (const [name, tool] of this.tools.entries()) {
            try {
                const valid = await tool.validate_prerequisites();
                if (!valid) {
                    console.error(`Prerequisites validation failed for ${name}`);
                    return false;
                }
                console.log(` ${name} prerequisites validated`);
            } catch (error) {
                console.error(`Error validating ${name}: ${error}`);
                return false;
            }
        }

        return true;
    }

    async execute_step(step_name: string, step_function: () => Promise<any>): Promise<void> {
        console.log(`Executing step: ${step_name}`);

        try {
            const result = await step_function();
            this.results.set(step_name, result);
            console.log(` Step ${step_name} completed successfully`);
        } catch (error) {
            console.error(` Step ${step_name} failed: ${error}`);
            throw error;
        }
    }

    async run_deployment_workflow(config: DeploymentConfig): Promise<void> {
        // Validate prerequisites first
        if (!(await this.validate_all_prerequisites())) {
            throw new Error('Prerequisites validation failed');
        }

        try {
            // Step 1: Infrastructure provisioning
            await this.execute_step('infrastructure_provisioning', async () => {
                const terraform = this.tools.get('terraform') as TerraformIntegration;
                await terraform.initialize_backend(config.terraform.backend_config);
                await terraform.plan(config.terraform.variables);
                return await terraform.apply(true);
            });

            // Step 2: System configuration
            await this.execute_step('system_configuration', async () => {
                const ansible = this.tools.get('ansible') as AnsibleIntegration;
                return await ansible.run_playbook(
                    config.ansible.playbook,
                    config.ansible.inventory,
                    config.ansible.variables
                );
            });

            // Step 3: Application deployment
            await this.execute_step('application_deployment', async () => {
                const k8s = this.tools.get('kubernetes') as KubernetesIntegration;
                await k8s.apply_manifest(config.kubernetes.manifest);
                return await k8s.wait_for_ready('deployment', config.kubernetes.deployment);
            });

            // Step 4: Health verification
            await this.execute_step('health_verification', async () => {
                // Implement health checks
                const health_checks = [
                    this.verify_infrastructure_health(),
                    this.verify_application_health(),
                    this.verify_service_availability()
                ];

                return await Promise.all(health_checks);
            });

            console.log('🎉 Deployment workflow completed successfully');

        } catch (error) {
            console.error('Deployment workflow failed:', error);

            // Execute rollback if configured
            if (config.rollback_enabled) {
                await this.execute_rollback();
            }

            throw error;
        }
    }

    private async execute_rollback(): Promise<void> {
        console.log('Executing rollback procedure...');

        try {
            // Rollback steps in reverse order
            const terraform = this.tools.get('terraform') as TerraformIntegration;
            await terraform.execute_command('destroy', ['-auto-approve']);

            console.log('✅ Rollback completed successfully');
        } catch (error) {
            console.error('❌ Rollback failed:', error);
            throw error;
        }
    }

    get_results(): Map<string, any> {
        return new Map(this.results);
    }
}

// Configuration interface
interface DeploymentConfig {
    terraform: {
        backend_config: string;
        variables: Record<string, string>;
    };
    ansible: {
        playbook: string;
        inventory: string;
        variables: Record<string, string>;
    };
    kubernetes: {
        manifest: string;
        deployment: string;
    };
    rollback_enabled: boolean;
}

// Usage example
const workflow = new WorkflowManager();

// Register tools
workflow.register_tool('terraform', new TerraformIntegration());
workflow.register_tool('ansible', new AnsibleIntegration());
workflow.register_tool('kubernetes', new KubernetesIntegration());

// Configuration
const config: DeploymentConfig = {
    terraform: {
        backend_config: 'backend.hcl',
        variables: {
            environment: process.env.ENVIRONMENT || 'development',
            region: process.env.AWS_REGION || 'us-west-2'
        }
    },
    ansible: {
        playbook: 'site.yml',
        inventory: 'inventory.ini',
        variables: {
            app_version: process.env.APP_VERSION || 'latest'
        }
    },
    kubernetes: {
        manifest: 'k8s/app-deployment.yaml',
        deployment: 'myapp'
    },
    rollback_enabled: true
};

// Execute workflow
workflow.run_deployment_workflow(config)
    .then(() => {
        console.log('Deployment completed successfully');
        console.log('Results:', Object.fromEntries(workflow.get_results()));
    })
    .catch((error) => {
        console.error('Deployment failed:', error);
        process.exit(1);
    });

🧪 Testing and Validation

Comprehensive Test Framework

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
#!/bin/bash
# test-framework.sh - Testing framework for glue code

# Test environment setup
setup_test_environment() {
    TEST_TEMP_DIR=$(mktemp -d)
    cd "$TEST_TEMP_DIR"

    # Create mock directories
    mkdir -p {mock-bin,mock-config,mock-data}

    # Create mock executables
    cat > mock-bin/terraform << 'EOF'
#!/bin/bash
echo "Terraform mock version 1.0.0"
EOF

    cat > mock-bin/ansible-playbook << 'EOF'
#!/bin/bash
echo "Ansible mock version 2.10.0"
EOF

    chmod +x mock-bin/*

    # Add to PATH
    export PATH="$TEST_TEMP_DIR/mock-bin:$PATH"

    echo "Test environment setup completed in $TEST_TEMP_DIR"
}

# Unit test runner
run_unit_tests() {
    local test_name="$1"
    local test_function="$2"
    local expected_result="${3:-0}"

    echo "Running test: $test_name"

    local result=0
    "$test_function" || result=$?

    if [ "$result" -eq "$expected_result" ]; then
        echo "✅ Test $test_name passed"
        return 0
    else
        echo "❌ Test $test_name failed (expected $expected_result, got $result)" >&2
        return 1
    fi
}

# Integration test runner
run_integration_tests() {
    echo "Running integration tests..."

    # Test IaC tool integration
    test_terraform_integration() {
        terraform --version >/dev/null 2>&1
    }

    test_ansible_integration() {
        ansible-playbook --version >/dev/null 2>&1
    }

    # Run tests
    run_unit_tests "Terraform Integration" test_terraform_integration
    run_unit_tests "Ansible Integration" test_ansible_integration

    echo "Integration tests completed"
}

# Performance test runner
run_performance_tests() {
    echo "Running performance tests..."

    # Test execution time
    test_execution_time() {
        local start_time
        start_time=$(date +%s%N)

        # Execute some operation
        sleep 0.1

        local end_time
        end_time=$(date +%s%N)

        local duration=$(( (end_time - start_time) / 1000000 ))  # milliseconds

        if [ "$duration" -lt 200 ]; then  # Should complete in < 200ms
            return 0
        else
            return 1
        fi
    }

    run_unit_tests "Execution Time" test_execution_time
    echo "Performance tests completed"
}

# Security test runner
run_security_tests() {
    echo "Running security tests..."

    # Test for hardcoded credentials
    test_hardcoded_credentials() {
        # This would scan source files for patterns like:
        # password = "hardcoded"
        # api_key = "secret123"
        # AWS_ACCESS_KEY_ID = "AKIA..."

        # Mock implementation
        return 0  # Pass for demo
    }

    # Test for insecure file permissions
    test_file_permissions() {
        # Check that sensitive files have proper permissions
        local sensitive_files=("/etc/passwd" "/etc/shadow")

        for file in "${sensitive_files[@]}"; do
            if [ -f "$file" ]; then
                local perms
                perms=$(stat -c "%a" "$file")

                # Check permissions (this is a simplified check)
                case "$file" in
                    "/etc/passwd")
                        if [ "$perms" != "644" ]; then
                            return 1
                        fi
                        ;;
                    "/etc/shadow")
                        if [ "$perms" != "640" ]; then
                            return 1
                        fi
                        ;;
                esac
            fi
        done

        return 0
    }

    run_unit_tests "Hardcoded Credentials" test_hardcoded_credentials
    run_unit_tests "File Permissions" test_file_permissions

    echo "Security tests completed"
}

# Run all tests
run_all_tests() {
    setup_test_environment

    echo "=== Running All Tests ==="

    run_unit_tests "Basic Functionality" "echo 'test'" 0
    run_integration_tests
    run_performance_tests
    run_security_tests

    # Cleanup
    cd /
    rm -rf "$TEST_TEMP_DIR"

    echo "=== All Tests Completed ==="
}

# Execute tests if run directly
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
    run_all_tests
fi

🧾 Summary Best Practices

✅ Do's

  • Separate concerns: Keep IaC definitions separate from procedural scripts
  • Ensure idempotency: All operations should be safely repeatable
  • Handle errors gracefully: Implement comprehensive error handling and logging
  • Secure credentials: Never hardcode secrets, use secure credential management
  • Validate inputs: Sanitize and validate all external inputs
  • Test thoroughly: Implement unit, integration, and security tests
  • Document everything: Clear documentation for complex glue code
  • Monitor execution: Log execution details for audit and debugging

❌ Don'ts

  • Mix responsibilities: Don't combine provisioning, configuration, and deployment in one script
  • Ignore errors: Always check return codes and handle failures appropriately
  • Hardcode secrets: Never store credentials in plain text
  • Skip validation: Always validate inputs and outputs
  • Forget cleanup: Implement proper resource cleanup and temporary file management
  • Neglect testing: Untested glue code leads to production issues
  • Overcomplicate: Keep scripts simple and focused on specific tasks

🧾 Implementation Checklist

Pre-Implementation

  • [ ] Define clear separation of concerns
  • [ ] Identify integration points between tools
  • [ ] Design error handling strategy
  • [ ] Plan security requirements
  • [ ] Establish testing approach

Implementation

  • [ ] Create modular, reusable components
  • [ ] Implement comprehensive error handling
  • [ ] Add structured logging
  • [ ] Include input validation
  • [ ] Ensure idempotent operations
  • [ ] Secure credential handling

Testing

  • [ ] Unit tests for individual functions
  • [ ] Integration tests for tool interactions
  • [ ] Performance tests for execution speed
  • [ ] Security tests for vulnerabilities
  • [ ] End-to-end workflow tests

Deployment

  • [ ] Document usage and configuration
  • [ ] Implement monitoring and alerting
  • [ ] Plan rollback procedures
  • [ ] Establish maintenance procedures
  • [ ] Create operational runbooks

🧾 See Also