Using DTrace-enabled Python in a Joyent Accelerator
Bryan wrote about a presentation he gave at Google (video) where he demo’ed DTrace in Python. While we did the dtrace probes for Ruby 1.8.5 and 1.8.6, I thought I’d show you how to do it on our Accelerators as well.
John Levon at Sun has written about the DTrace probes that he put into Python 2.4 before. So we have somewhere to start.
Let’s get Python 2.4.4 and the DTrace patch
mkdir /a/src/mkdir /a/1/languages/python244dcurl -O http://python.org/ftp/python/2.4.4/Python-2.4.4.tgztar zxvf Python-2.4.4.tgzcurl -O http://cvs.opensolaris.org/source/xref//jds/spec-files/trunk/patches/Python-07-dtrace.diff
The patch expects a directory call Python-new, so let’s create it
cp -R Python-2.4.4 Python-new
Let’s now apply the patch
gpatch -p0 < Python-07-dtrace.diff
The output of this working looks like
patching file Python-new/Include/frameobject.hpatching file Python-new/Makefile.pre.inpatching file Python-new/Objects/frameobject.cpatching file Python-new/Python/ceval.cpatching file Python-new/Python/phelper.dpatching file Python-new/Python/python.d
Move on in
cd Python-new/
Run configure
./configure --prefix=/a/1/languages/python244d --enable-shared
Now there’s going to be a couple of changes we’re going to make here so the thing builds, because if you type make && make install right now, it’ll fail. First in one way and then when you fix that, another.
Basically we have to bring the patch and Makefile up so that it’s a full 64-bit build (what our gcc compilers default to.
Open up the ustack helper file
nano Python/phelper.d
And take a look right at the top:
#define _SYS_STAT_H#include < stdio.h >#include "pyport.h"
And under the #include , add the two lines below
#define _SYS_STAT_H#include < stdio.h >#include < sys/socket.h >#include < sys/socketvar.h >#include "pyport.h"
Now the makefile
# Compiler optionsOPT= -DNDEBUG -g -O3 -Wall -Wstrict-prototypesBASECFLAGS= -fno-strict-aliasingCFLAGS= $(BASECFLAGS) $(OPT)CPPFLAGS= -I. -I$(srcdir)/IncludeLDFLAGS=LDLAST=SGI_ABI=CCSHARED= -fPICLINKFORSHARED=
Add the -m64 flag to LDFLAGS
# Compiler optionsOPT= -DNDEBUG -g -O3 -Wall -Wstrict-prototypesBASECFLAGS= -fno-strict-aliasingCFLAGS= $(BASECFLAGS) $(OPT)CPPFLAGS= -I. -I$(srcdir)/IncludeLDFLAGS= -m64LDLAST=SGI_ABI=CCSHARED= -fPICLINKFORSHARED=
And finally we’ll add a -64 flag to each of the dtrace compile lines, so that you’re doing a 64-bit build all the way around.
So lines like
dtrace -o $@ -DPHELPER $(DFLAGS) $(CPPFLAGS) -C -G -s $(srcdir)/Python/phelper.d
Become
# grep dtrace MakefileDTRACE_OBJS=Python/dtrace.o Python/phelper.odtrace -o $@ -DPHELPER $(DFLAGS) $(CPPFLAGS) -C -G -64 -s $(srcdir)/Python/phelper.ddtrace -o $@ $(DFLAGS) -C -64 -h -s $(srcdir)/Python/python.dPython/dtrace.o: $(srcdir)/Python/python.d Python/ceval.odtrace -o $@ $(DFLAGS) -C -G -64 -s $(srcdir)/Python/python.d Python/ceval.o
Let’s build this bad-boy
makemake installln -s /a/1/languages/python244d /a/1/languages/python
Then we can take a simple dtrace script, like John Levon’s pywrites.d and see some stuff:
libc.so.1`_write+0xa libc.so.1`_dowrite+0×54 libc.so.1`_ndoprnt+0×2cb6 libc.so.1`fprintf+0×132 libpython2.4.so.1.0`Py_Main+0×48c python`_start+0×6c 1 libc.so.1`_write+0xa libc.so.1`_ndoprnt+0×3535 libc.so.1`fprintf+0×132 libpython2.4.so.1.0`Py_Main+0×48c python`_start+0×6c 1
libc.so.1`_write+0xa libc.so.1`_fflush_u+0×34 libc.so.1`fclose+0×6b libpython2.4.so.1.0`file_close+0xbf libpython2.4.so.1.0`PyEval_EvalFrameReal+0×50c8 [ /a/1/languages/python244d/lib/python2.4/site-packages/django/core/management.py:803 (startproject) ] libpython2.4.so.1.0`PyEval_EvalFrame+0×1f libpython2.4.so.1.0`PyEval_EvalFrameReal+0×61dc [ /a/1/languages/python244d/lib/python2.4/site-packages/django/core/management.py:1599 (execute_from_command_line) ] libpython2.4.so.1.0`PyEval_EvalFrame+0×1f libpython2.4.so.1.0`PyEval_EvalCodeEx+0×727 libpython2.4.so.1.0`PyEval_EvalFrameReal+0×5195 [ /a/1/languages/python244d/bin/django-admin.py:5 (?) ] libpython2.4.so.1.0`PyEval_EvalFrame+0×1f libpython2.4.so.1.0`PyEval_EvalCodeEx+0×727 libpython2.4.so.1.0`PyEval_EvalCode+0×36 libpython2.4.so.1.0`PyRun_FileExFlags+0×97 libpython2.4.so.1.0`PyRun_SimpleFileExFlags+0×16f libpython2.4.so.1.0`Py_Main+0xac5 python`_start+0×6c 1 libc.so.1`_write+0xa libc.so.1`_fflush_u+0×34 libc.so.1`fclose+0×6b libpython2.4.so.1.0`file_close+0xbf libpython2.4.so.1.0`PyEval_EvalFrameReal+0×50c8 [ /a/1/languages/python244d/lib/python2.4/site-packages/django/core/management.py:783 (_start_helper) ] libpython2.4.so.1.0`PyEval_EvalFrame+0×1f libpython2.4.so.1.0`PyEval_EvalCodeEx+0×727 libpython2.4.so.1.0`PyEval_EvalFrameReal+0×5195 [ /a/1/languages/python244d/lib/python2.4/site-packages/django/core/management.py:795 (startproject) ] libpython2.4.so.1.0`PyEval_EvalFrame+0×1f libpython2.4.so.1.0`PyEval_EvalFrameReal+0×61dc [ /a/1/languages/python244d/lib/python2.4/site-packages/django/core/management.py:1599 (execute_from_command_line) ] libpython2.4.so.1.0`PyEval_EvalFrame+0×1f libpython2.4.so.1.0`PyEval_EvalCodeEx+0×727 libpython2.4.so.1.0`PyEval_EvalFrameReal+0×5195 [ /a/1/languages/python244d/bin/django-admin.py:5 (?) ] libpython2.4.so.1.0`PyEval_EvalFrame+0×1f libpython2.4.so.1.0`PyEval_EvalCodeEx+0×727 libpython2.4.so.1.0`PyEval_EvalCode+0×36 libpython2.4.so.1.0`PyRun_FileExFlags+0×97 libpython2.4.so.1.0`PyRun_SimpleFileExFlags+0×16f libpython2.4.so.1.0`Py_Main+0xac5 python`_start+0×6c 3
[jason:/tmp] root# ls jason/init.py manage.py settings.py urls.py
I think I’m going to bring the patch up to Python 2.5 as well.
Post written by jason