Przejdź do treści

📊 Echo for Structured Data Anti-Patterns

Using echo to generate structured data (JSON, XML, CSV, YAML) is a common but problematic practice that leads to fragile, insecure, and unmaintainable code. This anti-pattern demonstrates why proper data serialization is essential.

🎯 Core Problems

Manual Formatting Fragility

Hand-crafted structured data is prone to syntax errors and inconsistencies.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
# ❌ Anti-pattern: Manual JSON construction
generate_config() {
    local app_name="$1"
    local port="$2"

    echo "{"
    echo "  \"app\": \"$app_name\","
    echo "  \"port\": $port,"
    echo "  \"enabled\": true"
    echo "}"
}

# Problems:
# - Quotes must be escaped manually
# - Trailing commas cause parse errors
# - No validation of structure
# - Hard to maintain complex structures

# ❌ Fails with special characters
generate_config "My "App"" 8080  # Broken JSON

# ✅ Better approach: Use proper JSON generator
generate_config_proper() {
    local app_name="$1"
    local port="$2"

    if command -v jq >/dev/null 2>&1; then
        jq -n \
            --arg name "$app_name" \
            --argjson port "$port" \
            '{
                app: $name,
                port: $port,
                enabled: true
            }'
    else
        echo "Error: jq required for JSON generation" >&2
        return 1
    fi
}

Security Vulnerabilities

Manual data construction exposes injection vulnerabilities.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# ❌ Anti-pattern: Vulnerable to injection
create_user_json() {
    local username="$1"
    local role="$2"

    echo "{
        \"username\": \"$username\",
        \"role\": \"$role\"
    }"
}

# Security risk: Command injection
malicious_input='admin"; system("rm -rf /"); "'
create_user_json "$malicious_input" "user"
# Results in invalid/broken JSON and potential security issues

# ✅ Safer approach: Proper escaping
create_user_json_safe() {
    local username="$1"
    local role="$2"

    # Escape quotes and other special characters
    username_escaped=$(printf '%s' "$username" | sed 's/"/\\"/g')
    role_escaped=$(printf '%s' "$role" | sed 's/"/\\"/g')

    echo "{
        \"username\": \"$username_escaped\",
        \"role\": \"$role_escaped\"
    }"
}

🔧 Common Data Format Abuses

JSON Generation Issues

Improper JSON construction leads to parsing failures.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
# ❌ Anti-pattern: Multiple problems
generate_complex_json() {
    local name="$1"
    local items="$2"

    echo "{
        \"name\": \"$name\",
        \"items\": [
            $items
        ],
        \"metadata\": {
            \"created\": \"$(date)\",
            \"version\": \"1.0\"
        }
    }"
}

# Issues:
# 1. No array element separation
# 2. Date quoting issues
# 3. Special character handling
# 4. No validation

# ✅ Proper JSON generation
generate_complex_json_proper() {
    local name="$1"
    shift
    local items=("$@")

    if command -v jq >/dev/null 2>&1; then
        local jq_array="["
        local first=true

        for item in "${items[@]}"; do
            if [ "$first" = true ]; then
                jq_array="${jq_array}\"$item\""
                first=false
            else
                jq_array="${jq_array},\"$item\""
            fi
        done

        jq_array="${jq_array}]"

        jq -n \
            --arg name "$name" \
            --argjson items "$jq_array" \
            --arg created "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
            '{
                name: $name,
                items: $items,
                metadata: {
                    created: $created,
                    version: "1.0"
                }
            }'
    else
        echo "Error: jq required" >&2
        return 1
    fi
}

CSV Generation Problems

Manual CSV creation ignores escaping and formatting rules.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
# ❌ Anti-pattern: Broken CSV
generate_csv() {
    local name="$1"
    local email="$2"
    local city="$3"

    echo "$name,$email,$city"
}

# Fails with commas in data
generate_csv "John Doe" "john@example.com" "New York, NY"  # Broken CSV

# ✅ Proper CSV generation
generate_csv_proper() {
    local name="$1"
    local email="$2"
    local city="$3"

    # Escape quotes and wrap fields with commas/quotes
    escape_csv_field() {
        local field="$1"

        # If field contains comma, quote, or newline, wrap in quotes and escape quotes
        if echo "$field" | grep -E '[,",\n]' >/dev/null; then
            # Escape quotes by doubling them
            field_escaped=$(echo "$field" | sed 's/"/""/g')
            echo "\"$field_escaped\""
        else
            echo "$field"
        fi
    }

    local name_escaped=$(escape_csv_field "$name")
    local email_escaped=$(escape_csv_field "$email")
    local city_escaped=$(escape_csv_field "$city")

    echo "$name_escaped,$email_escaped,$city_escaped"
}

# Or use a proper CSV tool
generate_csv_tool() {
    if command -v csvformat >/dev/null 2>&1; then
        printf "%s\n%s\n%s\n" "$1" "$2" "$3" | csvformat
    else
        # Fallback with manual escaping
        generate_csv_proper "$1" "$2" "$3"
    fi
}

🎨 Advanced Anti-Patterns

Nested Structure Complexity

Manual nested data construction becomes unwieldy quickly.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
# ❌ Anti-pattern: Complex nested structure
generate_nested_config() {
    local service_name="$1"
    shift
    local endpoints=("$@")

    echo "{
        \"service\": {
            \"name\": \"$service_name\",
            \"endpoints\": ["

    for endpoint in "${endpoints[@]}"; do
        echo "                \"$endpoint\","
    done

    echo "            ],
            \"settings\": {
                \"timeout\": 30,
                \"retries\": 3
            }
        }
    }"
}

# Problems:
# - Trailing comma issues
# - Indentation inconsistency
# - Hard to modify
# - No validation

# ✅ Better approach: Template-based generation
generate_nested_config_proper() {
    local service_name="$1"
    shift
    local endpoints=("$@")

    # Create endpoints array in JSON format
    local endpoints_json=""
    local first=true

    for endpoint in "${endpoints[@]}"; do
        if [ "$first" = true ]; then
            endpoints_json="\"$endpoint\""
            first=false
        else
            endpoints_json="$endpoints_json,\"$endpoint\""
        fi
    done

    # Use jq for proper nested structure
    if command -v jq >/dev/null 2>&1; then
        jq -n \
            --arg name "$service_name" \
            --argjson endpoints "[$endpoints_json]" \
            '{
                service: {
                    name: $name,
                    endpoints: $endpoints,
                    settings: {
                        timeout: 30,
                        retries: 3
                    }
                }
            }'
    else
        echo "Error: jq required for complex JSON generation" >&2
        return 1
    fi
}

Dynamic Data Challenges

Handling dynamic content with manual formatting.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
# ❌ Anti-pattern: Dynamic content issues
generate_dynamic_report() {
    local title="$1"
    shift
    local metrics=("$@")

    echo "{
        \"title\": \"$title\",
        \"generated\": \"$(date)\",
        \"metrics\": {"

    for metric in "${metrics[@]}"; do
        key=$(echo "$metric" | cut -d: -f1)
        value=$(echo "$metric" | cut -d: -f2)
        echo "            \"$key\": $value,"
    done

    echo "        }
    }"
}

# Issues:
# - No type handling (strings vs numbers)
# - Date formatting inconsistencies
# - Parsing delimiter conflicts
# - No error handling

# ✅ Proper dynamic data handling
generate_dynamic_report_proper() {
    local title="$1"
    shift
    local metrics=("$@")

    if command -v jq >/dev/null 2>&1; then
        # Build metrics object
        local jq_metrics="{"
        local first=true

        for metric in "${metrics[@]}"; do
            key=$(echo "$metric" | cut -d: -f1)
            value=$(echo "$metric" | cut -d: -f2)

            # Determine if value is numeric
            if echo "$value" | grep -E '^[0-9]+(\.[0-9]+)?$' >/dev/null; then
                # Numeric value
                if [ "$first" = true ]; then
                    jq_metrics="${jq_metrics}\"$key\": $value"
                    first=false
                else
                    jq_metrics="${jq_metrics},\"$key\": $value"
                fi
            else
                # String value
                if [ "$first" = true ]; then
                    jq_metrics="${jq_metrics}\"$key\": \"$value\""
                    first=false
                else
                    jq_metrics="${jq_metrics},\"$key\": \"$value\""
                fi
            fi
        done

        jq_metrics="${jq_metrics}}"

        jq -n \
            --arg title "$title" \
            --argjson metrics "$jq_metrics" \
            --arg generated "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
            '{
                title: $title,
                generated: $generated,
                metrics: $metrics
            }'
    else
        echo "Error: jq required for report generation" >&2
        return 1
    fi
}

🧾 Summary of Issues

Common Problems with Manual Data Generation

Issue Impact Solution
Syntax errors Parsing failures Use proper generators
Injection vulnerabilities Security risks Escape special characters
Maintenance difficulty Hard to modify Use templates/tools
Type inconsistencies Data quality issues Explicit type handling
Validation absence Silent failures Validate output
Encoding problems Character issues Handle UTF-8 properly

Red Flags to Avoid

🚩 Manually concatenating quotes and braces 🚩 No escaping of special characters 🚩 Hard-coded delimiters in dynamic data 🚩 Mixed data types without explicit handling 🚩 No validation of generated output 🚩 Complex nested structures without tools

🧠 Prevention Strategies

Best Practices for Structured Data

  1. Use proper tools: jq for JSON, yq for YAML, xmlstarlet for XML
  2. Escape everything: Always escape user-provided data
  3. Validate output: Check that generated data parses correctly
  4. Handle types explicitly: Don't mix strings and numbers accidentally
  5. Template when possible: Use templating engines for complex structures
  6. Test edge cases: Special characters, empty values, nulls

Safe Data Generation Template

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
generate_structured_data_safely() {
    local data_type="$1"
    shift

    case "$data_type" in
        json)
            generate_json_safe "$@"
            ;;
        csv)
            generate_csv_safe "$@"
            ;;
        yaml)
            if command -v yq >/dev/null 2>&1; then
                generate_yaml_safe "$@"
            else
                echo "Error: yq required for YAML generation" >&2
                return 1
            fi
            ;;
        *)
            echo "Error: Unsupported data type: $data_type" >&2
            return 1
            ;;
    esac
}

# Always validate generated output
validate_generated_data() {
    local data="$1"
    local format="$2"

    case "$format" in
        json)
            if ! echo "$data" | jq . >/dev/null 2>&1; then
                echo "Error: Invalid JSON generated" >&2
                return 1
            fi
            ;;
        csv)
            # Basic CSV validation
            if ! echo "$data" | csvformat >/dev/null 2>&1; then
                echo "Warning: CSV validation failed" >&2
            fi
            ;;
    esac
}

🧾 See Also