Przejdź do treści

☀️ Solaris Debugging Tools

Solaris provides enterprise-grade debugging and diagnostic tools designed for large-scale systems. With features like DTrace, MDB, and extensive observability frameworks, Solaris offers unparalleled insight into system behavior and performance.

🔍 Core Debugging Utilities

truss – System Call Tracing

Solaris equivalent of strace/truss, traces system calls and signals.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Trace system calls of a command
truss ls

# Trace specific process
truss -p 1234

# Count syscalls by type
truss -c ./my_program

# Follow child processes
truss -f daemon_program

pstack – Process Stack Tracer

Shows stack traces of running processes without stopping them.

1
2
3
4
5
# Display stack trace of process
pstack 1234

# Continuous stack tracing
pstack -f 1234

mdb – Modular Debugger

Powerful debugger for live systems, crash dumps, and kernel modules.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Start MDB on live system
mdb -k

# Analyze crash dump
mdb /var/crash/vmdump.0

# Common MDB commands:
# ::status          - Show system status
# ::stack           - Display current stack
# ::ps              - List processes
# ::memstat         - Memory statistics

📊 Performance Analysis Tools

prstat – Process Statistics

Enhanced version of top with additional sorting and grouping options.

1
2
3
4
5
6
7
8
# Interactive process view
prstat

# Sort by CPU usage
prstat -s cpu

# Monitor specific user
prstat -u username

vmstat – Virtual Memory Statistics

Reports virtual memory, process scheduling, disk I/O, and interrupt statistics.

1
2
3
4
5
6
7
8
# Display every 2 seconds
vmstat 2

# Show disk-specific stats
vmstat -i

# Extended reporting
vmstat -s

sar – System Activity Reporter

Collects and reports historical system performance data.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# CPU utilization report
sar -u

# Memory usage trends
sar -r

# I/O statistics
sar -b

# View yesterday's data
sar -f /var/adm/sa/sa01

🗃️ Filesystem & Storage Diagnostics

pfiles – Process File Descriptors

Shows which files and sockets a process has open.

1
2
3
4
5
# Show open files for process
pfiles 1234

# Display network endpoints
pfiles 1234 | grep sock

iostat – I/O Statistics

Monitors disk I/O performance and device utilization.

1
2
3
4
5
6
7
8
# Extended device statistics
iostat -xn

# Update every 5 seconds
iostat 5

# Show NFS statistics
iostat -n

zpool / zfs – ZFS Management

Diagnostic tools for Oracle's ZFS filesystem.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Pool status and health
zpool status

# Dataset usage
zfs list

# I/O statistics
zpool iostat -v 5

# Scrub pool for errors
zpool scrub tank

🌐 Network Troubleshooting

netstat – Network Statistics

Displays network connections, routing tables, and interface statistics.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Active connections
netstat -an

# Listening ports
netstat -an | grep LISTEN

# Interface statistics
netstat -i

# Show routing table
netstat -rn

snoop – Packet Sniffer

Solaris-native packet capture tool.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Capture on default interface
snoop

# Filter HTTP traffic
snoop port 80

# Write capture to file
snoop -o capture.pcap

# Read from file
snoop -i capture.pcap

Manage and monitor network datalinks and physical interfaces.

1
2
3
4
5
6
7
8
# Show all links
dladm show-link

# Physical interface status
dladm show-phys

# VLAN information
dladm show-vlan

🧠 Memory Debugging

libumem – Userland Memory Allocator

Advanced memory debugging with leak detection and corruption tracking.

1
2
3
4
5
6
7
8
9
# Enable umem debugging
export UMEM_DEBUG=default
export LD_PRELOAD=/usr/lib/libumem.so.1

# Run application with umem
./my_program

# Analyze umem logs
umemstat

swap – Swap Space Management

Monitor and manage virtual memory swap areas.

1
2
3
4
5
6
7
8
# Show swap usage
swap -s

# List swap devices
swap -l

# Add temporary swap
swap -a /tmp/tempswap 1024m

🕵️‍♂️ Logs and System Messages

dmesg – Kernel Ring Buffer

Views boot-time messages and runtime kernel diagnostics.

1
2
3
4
5
# Recent kernel messages
dmesg | tail

# Highlight errors/warnings
dmesg | grep -E "(error|warning)"

svcs – Service Management

Query and diagnose SMF services.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# List all services
svcs

# Service status with dependencies
svcs -d svc:/network/http:apache24

# Failed services
svcs -x

# Service log files
svcs -L svc:/network/http:apache24

🧰 Advanced Diagnostic Tools

DTrace – Dynamic Tracing Framework

Revolutionary observability tool for live system analysis.

Example probe: 

1
2
3
4
syscall::open:entry
{
    printf("Opening file: %s\n", copyinstr(arg0));
}

Run with: 

1
2
3
4
5
# Execute D script
dtrace -s script.d

# One-liner probes
dtrace -n 'syscall::read:entry /pid == 1234/ { @["reads"] = count(); }'

ptree – Process Tree Viewer

Displays hierarchical process relationships.

1
2
3
4
5
# Show all processes in tree form
ptree

# Focus on specific process subtree
ptree 1234

plimit – Process Limits Display

Shows resource limits for processes.

1
2
3
4
5
# Current limits for process
plimit 1234

# Show all limits
plimit

🧾 Summary Table

Tool Purpose
truss System call tracing
pstack Stack trace analysis
mdb Modular kernel/application debugger
prstat Enhanced process monitoring
pfiles File descriptor inspection
snoop Network packet capture
DTrace Dynamic system-wide tracing
dmesg Kernel diagnostic messages
svcs Service management diagnostics

🧠 Best Practices

✅ Use truss for syscall-level debugging ✅ Leverage DTrace for production-safe observability ✅ Monitor ZFS health with zpool status ✅ Use libumem for advanced memory debugging ✅ Combine ptree and pstack for process analysis ✅ Store sar data for capacity planning

🧾 See Also