New Image Tools for SmartOS

Starting with our newer machine images we've introduced a few customer specific utilities to help you in the SmartOS world: Quickbackup, Smtools, and Dtracetools. These are packages updated frequently in which anyone can install and use immediately to reap the benefits and make your life easier.

  • Quickbackup - Provides a super fast and simple approach to creating Percona and MongoDB backups.
  • Smtools - A collection of utilities to handle various tasks around images such as changing the hostname, setting your timezone, monitoring resources, printing image information, listing creating removing databases and users, automatic tuning of MySQL, installing a blog, and more.
  • Dtracetools - Many DTrace scripts put in to a package for tracing and real time debugging of your image.

First we'll go over the basics of Quickbackup which is only available for Percona, MongoDB images and comes pre-installed.

Quickbackup (Percona)

Quickbackup is a command line tool wrapped around Percona Xtrabackup which provides a very simplistic, one command approach to backing up your data so you don't have to do any of the messy behind the scene configuration for Xtrabackup. When you provision a Percona image Quickbackup is already installed for you and you can begin backing up your MySQL data manually or on a schedule.

To take a backup manually:

# quickbackup backup* Backup for /backups/2012-10-12_19-57-41-backup.tar.gz* Backup completed successfully* Rotation completed successfully

Backups are stored at /backups with the proper time stamp. If you want to view the details of the backup you can view the Quickbackup log file at /var/log/mysql/quickbackup.log.

To restore a backup first shut down MySQL and then run the restore:

# svcadm disable mysql:percona

Verify it is down:

# svcs mysql:perconaSTATE          STIME    FMRIdisabled       14:19:21 svc:/network/mysql:percona

Restore by passing in the path to the back up file:

# quickbackup restore /backups/2011-04-16_20-20-00-backup.tar.gz* Restore from /backups/2011-04-16_20-20-00-backup.tar.gz* Restore completed successfully.

To take a backup on a schedule:

For backing up on a schedule Quickbackup is already installed as a SMF service which properties must be configured to specify when you want to backup. By default the service is set to backup every night at midnight but comes disabled by default. To enable this all you need to do is enable the service:

# svcadm enable mysql-quickbackup

Your backups will begin collecting at /backups every night at midnight. To set the service to perform a backup every 15 minutes of every hour Monday, Tuesday, Wednesday, Thursday, and Friday, you would use the following configuration commands to Quickbackup:

# svccfg -s mysql-quickbackup setprop quickbackup/minute = 0,15,30,45# svccfg -s mysql-quickbackup setprop quickbackup/hour = all# svccfg -s mysql-quickbackup setprop quickbackup/day = mon,tue,wed,thur,fri

After setting the configuration you must refresh and restart the service for changes to take:

# svcadm refresh mysql-quickbackup# svcadm restart mysql-quickbackup# svcs mysql-quickbackupSTATE          STIME    FMRIenabled        Apr_13   svc:/network/mysql-quickbackup

Quickbackup is highly configurable. To view the configuration settings:

# svcprop mysql-quickbackup | grep "quickbackup/"quickbackup/day astring allquickbackup/expiredays astring 3quickbackup/hour astring 0quickbackup/minute astring 0quickbackup/username astring qb-xxxxxxxxquickbackup/password astring xxxxxxxxquickbackup/backupdir astring /backupsquickbackup/stream astring xbstream

The only settings you should be concerned about are day, hour, minute, expiredays (age of backup files to be rotated), backupdir (specify where to store backups), and stream. The stream method is set to xbstream by default which allows super fast backups. In our testing we have done 10G backups in less than a minute!

Documentation for Quickbackup (Percona) can be found at docs.tritondatacenter.com/jpc/Joyent+Quickbackup+for+Percona+MySQL+Server

Quickbackup (MongoDB)

Quickbackup for MongoDB is very similar to the Percona version, only that it is powered on the back end by mongodump and mongorestore. The service comes installed by default on MongoDB images which has the SMF name svc:/pkgsrc/quickbackup-mongodb and provides backing up on a schedule. All of the commands and configuration are the same as above.

To take a backup manually:

# quickbackup-mongodb backup* Backup for /backups/2012-10-12_19-57-41-backup.tar.gz* Backup completed successfully* Rotation completed successfully

To restore a backup the MongoDB service must be running, then you can run the restore:

# svcs pkgsrc/quickbackup-mongodbSTATE          STIME    FMRIenabled        21:11:00 svc:/pkgsrc/quickbackup-mongodb:default

Then specify which backup to restore:

# quickbackup-mongodb restore /backups/2012-03-20_20-59-48-backup.tar.gz* Restore from /backups/2012-03-20_20-59-48-backup.tar.gz* Restore completed successfully

To view the details of the backups and restores see /var/log/mongodb/quickbackup-mongodb.log. For further configuration on Quickbackup for MongoDB, see docs.tritondatacenter.com/jpc/Joyent+Quickbackup+for+MongoDB

Smtools

Smtools is a collection of utilities (available in pkgsrc 2012Q1, 2012Q2, package 'smtools') that can help answer questions, and make tasks much more simple in your Image such as:

  • What image am I running?
  • Where can I find documentation for this Image?
  • What Pkgsrc quarter am I on?
  • How much memory am I using?
  • How much cpu am I using?
  • How much disk space am I using?
  • How do I upgrade my Image to the latest pkgsrc release?
  • What available timezones do I have on this Image?
  • I want to set my timezone to Greenwich.
  • I want to change my hostname to 'newhostname'.
  • I want to work with my databases but don't know any of the commands to do so.
  • I need to tune MySQL, but know nothing about MySQL tuning.
  • I need wordpress installed quickly.

On newer Images smtools comes installed by default, but can be installed through pkgsrc:

# pkgin install smtools

Tools are installed into /opt/local/bin and prefixed with sm-*. Let's see an example of some of them:

sm-summary Provides a summary of your Image

# sm-summary* Gathering SmartMachine summary..SM UUID             xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxSM ID               2Hostname            getest1.localSmartOS build       joyent_20120718T044524ZImage             percona 1.6.0Pkgsrc              http://pkgsrc.joyent.com/sdc6/2012Q1/i386/AllProcesses           26Memory (RSS) Cap    16384MMemory (RSS) Used   4137MMemory (RSS) Free   12247MSwap Cap            32768MSwap Used           9519M/tmp Used           12KDisk Quota          482G% Disk Used         2%

sm-list-timezones Provides a listing of available timezones for this Image

# sm-list-timezones* Listing timezones for xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx..Africa/AbidjanAfrica/AccraAfrica/Addis_AbabaAfrica/AlgiersAfrica/AsmaraAfrica/AsmeraAfrica/BamakoAfrica/BanguiAfrica/Banjul…

sm-set-timezone Sets the timezone in your Image

# sm-set-timezone Greenwich* Setting timezone for xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx..* Changed to timezone to Greenwich. You will need to reboot.

sm-meminfo Reports how much memory is being used for this Image

# sm-meminfo rss* Gathering rss memory infomation for xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx..Memory cap: 16.0 GiBMemory used: 4.0 GiBMemory free: 11.9 GiB

sm-set-hostname Sets the hostname of your Image

# sm-set-hostname testblah* Setting hostname for xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx..* Changed hostname to testblah. You will need to reboot.

sm-install-wordpress Installs the latest Wordpress to a specified path (You must still setup Apache with sm-create-vhost)

# sm-install-wordpress /home/admin* Downloading latest wordpress.* Configuring wordpress.* Created MySQL database wp_xxxxxxxx.* Created MySQL user wp_xxxxxxxxxx pass xxxxxxxxxxxxxx with access to wp_xxxxxxxx from anywhere.* Visit http://x.x.x.x/wp-admin/install.php to finish the install.

sm-list-dbs Lists MySQL or PostgreSQL databases

# sm-list-dbs mysql* Listing MySQL databases for xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx..information_schemamgtestmysqlperformance_schemawp_a1d5001f

sm-create-db Creates MySQL or PostgreSQL databases

# sm-create-db mysql testdb1* Created MySQL database testdb1.

sm-tune-mysql Automatic database tuning of MySQL or Percona

# sm-tune-mysql* Tuning MySQL, this can take up to 60 seconds..Your tuned configuration is at /mysql/my.cnf.tunedThere is more to just copying this file over your old one andrestarting. Review the configuration file first to make anynecessary changes, and because 'innodb_log_file_size' has likelybeen modified you need to remove the old log files first beforestarting. If you don't do this properly then MySQL won't start.

sm-rebuild-pkgsrc Upgrade pkgsrc to the latest release

# sm-rebuild-pkgsrcThis script upgrades userland software in /opt/local, using packagesfrom the following package set:http://pkgsrc.joyent.com/sdc6/2012Q2/x86_64In the process, your /opt/local path will be backed up, new bootstrap imagewill be downloaded and packages that you had installed, will be re-installedfrom the new package set.Consequently, if you had any software or files installed under /opt/localthat did not come from pkgsrc packages (when this machine was provisioned,or installed later on), it will be removed and only preserved in the backupcompressed tarball.? You do want to proceed with initial checks? y/N

See docs.tritondatacenter.com/jpc/SmartMachine+Tools+Package for more examples of Smtools.

Dtracetools

The Dtracetools is a pkgsrc package available since 2011Q4 which contains many (over 100!) DTrace scripts you can use to trace and debug issues in your Image. Dtracetools is available by default on all Images, but can be installed through pkgsrc with:

# pkgin install dtracetools

The tools will be installed at /opt/local/bin with the prefix dtrace-*. There are many useful Dscripts which provide tracing of PHP, Ruby, Perl, Python, Bash, Tcl, MongoDB, MySQL, and more. Below are a few examples:

Viewing MongoDB commands a second with dtrace-mongocmd.d

# dtrace-mongocmd.d -p `pgrep -x mongod`dtrace: script '/opt/local/bin/dtrace-mongocmd.d' matched 2 probesCPU     ID                    FUNCTION:NAME  6  71046                         :tick-1s  6  71046                         :tick-1s  6  71046                         :tick-1s mongo cmds/s: 18  6  71046                         :tick-1s mongo cmds/s: 0  6  71046                         :tick-1s mongo cmds/s: 0  6  71046                         :tick-1s mongo cmds/s: 0  6  71046                         :tick-1s mongo cmds/s: 0  6  71046                         :tick-1s mongo cmds/s: 0  6  71046                         :tick-1s mongo cmds/s: 18  6  71046                         :tick-1s mongo cmds/s: 0  6  71046                         :tick-1s mongo cmds/s: 0^C

Viewing MySQL queries longer than 1 millisecond with dtrace-mysql_pid_slow.d

# dtrace-mysql_pid_slow.d -p `pgrep -x mysqld` 1Tracing... Min query time: 1000000 ns. TIME(ms) CPU(ms)  QUERY 91       3        show databases\0 27       0        select * from mysql.user\0^C

Viewing MySQL query latency in a distribution with dtrace-mysql_pid_latency.d

# dtrace-mysql_pid_latency.d -p `pgrep -x mysqld`Tracing PID 10630... Hit Ctrl-C to end.MySQL queries/second: 1; query latency (ns):       value  ------------- Distribution ------------- count      131072 |                                         0      262144 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1      524288 |                                         0^CMySQL queries/second: 1; query latency (ns):       value  ------------- Distribution ------------- count       65536 |                                         0      131072 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1      262144 |                                         0

Viewing MySQL query average execution time with dtrace-mysql_pid_avg.d

# dtrace-mysql_pid_avg.d -p `pgrep -x mysqld`Tracing PID 10630...TIME                    QUERIES 1+sec_Qs  AVG(ms)  MAX(ms)2012 Oct 19 15:23:07          1        0        0        02012 Oct 19 15:23:08          1        0        0        02012 Oct 19 15:23:09          0        0        0        0^C

Viewing PHP function call counts with dtrace-php_funccalls.d

# dtrace-php_funccalls.dTracing... Hit Ctrl-C to end.^C FILE                             FUNC                                CALLS blah.php                         blastoff                                4 blah.php                         launch                                  4 blah.php                         one                                     4 blah.php                         sleep                                   4 blah.php                         three                                   4 blah.php                         two                                     4

Viewing PHP function flow with dtrace-php_flow.d

# dtrace-php_flow.dTIME(us)         -- FUNC20578568252193   <- sleep20578568252218   -> launch20578568252229     -> three20578568252269       -> two20578568252282         -> one20578568252296           -> blastoff20578568252314           <- blastoff20578568252320         <- one20578568252324       <- two20578568252329     <- three20578568252333   <- launch20578568252341   -> sleep

As you can see we can easily get information from our applications using the Dtracetools package. This package is still under extreme development where some scripts may need minor tweaking. We are constantly updating docs.tritondatacenter.com/jpc/Collecting+Data+with+DTraceTools with updates as it develops.



Post written by Derek Crudgington