Cloud Performance (includes SmartOS, Linux, and LX zones)


Location: San Francisco

Cost: $4500

Instructor: Max Bruning


This course is about cloud performance from a systems perspective, focusing on the systems that make up the environment. This covers both the challenges of traditional systems performance and also the performance of virtualization and multi-tenant environments. It is intended for system support staff, system administrators, developers, and anyone who would like to learn how systems work and perform, and the additional characteristics of the cloud. This is for both users and operators of the cloud. Topics include operating system internals, performance analysis methodology, generic application performance, CPUs, memory, file systems, disks, networking, cloud computing, and benchmarking. The example operating system studied is SmartOS, however, much of the methodologies and techniques are appropriate for others as well. Numerous performance tools are covered, from top to DTrace. The basis for this course is the Prentice Hall book: Systems Performance: Enterprise and the Cloud. Additional labs are provided, which simulate real-world cases for the students to solve, with help from the instructor. For some students this is the highlight of the course, putting the methodologies learned into practice and seeing how they work first-hand.



  • systems and cloud performance
  • performance analysis objectives
  • application performance techniques, including caching, buffering, and concurrency
  • problems of load vs architecture
  • metrics: latency, IOPS, utilization, saturation
  • statistics: average types, multimodal distributions, outliers
  • essential operating system and kernel internals
  • identify performance features of modern kernels
  • how performance metrics are generated by the kernel
  • explain static and dynamic tracing
  • processor and CPU hardware architecture
  • kernel scheduler architecture and topics, including preemption and priority inversion
  • memory system architecture, including virtual memory and anonymous paging
  • physical memory architecture, including the MMU and MPSS
  • VFS and file system architecture and common performance characteristics
  • performance attributes of storage devices, including HDDs, SSDs, controllers, and transports
  • explain differences between logical and physical I/O
  • the role of the block device interface and the lower-level I/O stack
  • network latency sources
  • TCP performance characteristics, including buffering and listen queues
  • the lower level network stack
  • software and hardware virtualization technologies, and their architecture, overheads, and performance characteristics
  • resource controls used by cloud computing environments
  • multi-tenancy effects in cloud computing
  • benchmarking activities, types, and common mistakes
  • different performance analysis and tuning methodologies


  • resource and workload analysis
  • the problem statement method
  • the USE method
  • workload characterization method
  • CPU profiling, including generating flame graphs
  • syscall analysis using buffered tracing
  • I/O profiling
  • active benchmarking
  • solve performance issues using thread state analysis


  • DTrace one-liners and scripts for performance analysis
  • tools for CPU analysis, including vmstat, mpstat, prstat, cpustat, DTrace
  • tools for Memory analysis, including vmstat, prstat, pmap, DTrace
  • advanced memory analysis techniques, including allocation and fault tracing
  • tools for file system analysis, including vfsstat, truss, DTrace
  • advanced file system analysis techniques, including latency analysis from different stack layers
  • various tools for disk analysis, including iostat, iosnoop, DTrace
  • advanced disk analysis capabilities
  • tools for network analysis, including netstat, nicstat, snoop, DTrace
  • advanced network analysis techniques, including socket and TCP event tracing
  • performance analysis context for software and hardware virtualization as used in cloud computing
  • CPI


Familiarity with using a system via SSH, with root or equivalent privileges.