Jekyll2017-06-13T09:03:07-05:00//goodell.ioDave Goodell's notesDave Goodellcolors in tshark!2017-06-13T09:00:00-05:002017-06-13T09:00:00-05:00//2017/06/13/tshark-colors<p>[tl;dr: per-packet coloring is now supported in <code class="highlighter-rouge">tshark</code> (command-line
Wireshark) with <code class="highlighter-rouge">--color</code>]</p>
<p>I regularly use
<a href="https://www.wireshark.org/docs/wsug_html_chunked/AppToolstshark.html"><code class="highlighter-rouge">tshark</code></a>
when doing protocol work, often several times per week. It’s a great way to
take load off my brain instead of constantly parsing packets with my eyeballs
directly from a raw hex dump. <code class="highlighter-rouge">tshark</code> is like <code class="highlighter-rouge">tcpdump</code> on steroids,
providing me the majority of the benefits of Wireshark but without having to
leave the terminal, use the mouse, or slurp a <code class="highlighter-rouge">.pcap</code> file around between
machines. With <code class="highlighter-rouge">tshark</code> I can usually just view the <code class="highlighter-rouge">.pcap</code> file on the same
server I used to capture it.</p>
<!--fold-->
<p>One feature that is present in Wireshark but has traditionally been missing
from <code class="highlighter-rouge">tshark</code> is coloring of packets to help spot noteworthy packets in a
capture. This annoyed me, so <a href="https://code.wireshark.org/review/21325">I implemented the feature and contributed it to
upstream Wireshark</a>. Here’s what the
result looks like:</p>
<p><a href="/images/tshark-wireshark-color.png"><img src="/images/tshark-wireshark-color.png" alt="tshark/wireshark comparison screen shot" /></a></p>
<p>Under the hood it works by using <a href="https://gist.github.com/XVilka/8346728">a slightly exotic terminal escape sequence
to request 24-bit coloring</a>. Also, if
you’re a tmux user, you’ll need to upgrade to v2.4 or later if you want to
avoid a ragged right edge of background coloring, since tmux only implemented
better support for the <strong>CSI EL</strong> sequence fairly recently</p>
<p>If you want to take this functionality for a spin yourself, you’ll need a
build of <code class="highlighter-rouge">tshark</code> from Git that contains commit
<a href="https://code.wireshark.org/review/gitweb?p=wireshark.git;a=commit;h=2be411a215072052a3df879e707ace1948e3f3f6"><code class="highlighter-rouge">v2.5.0rc0-97-g2be411a215</code></a>.
Then just use tshark as you normally would, but add the <code class="highlighter-rouge">--color</code> flag.</p>
<p>Enjoy!</p>
<p><strong>P.S.</strong></p>
<p>Another tshark ProTip™: use <code class="highlighter-rouge">-V</code> or <code class="highlighter-rouge">-O ${protocol}</code> to get detailed
dissection info in your terminal</p>
<div class="highlighter-rouge"><pre class="highlight"><code>Frame 2: 66 bytes on wire (528 bits), 66 bytes captured (528 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Apr 25, 2017 01:45:23.435014000 CDT
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1493102723.435014000 seconds
[Time delta from previous captured frame: 0.029217000 seconds]
[Time delta from previous displayed frame: 0.029217000 seconds]
[Time since reference or first frame: 0.029217000 seconds]
Frame Number: 2
Frame Length: 66 bytes (528 bits)
Capture Length: 66 bytes (528 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:tcp]
Ethernet II, Src: 08:35:71:05:d5:30, Dst: f4:0f:24:3c:0f:f3
Destination: f4:0f:24:3c:0f:f3
Address: f4:0f:24:3c:0f:f3
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: 08:35:71:05:d5:30
Address: 08:35:71:05:d5:30
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 54.231.114.156, Dst: 192.168.6.253
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
0000 00.. = Differentiated Services Codepoint: Default (0)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 52
Identification: 0xb77a (46970)
Flags: 0x02 (Don't Fragment)
0... .... = Reserved bit: Not set
.1.. .... = Don't fragment: Set
..0. .... = More fragments: Not set
Fragment offset: 0
Time to live: 228
Protocol: TCP (6)
Header checksum: 0x6e20 [validation disabled]
[Header checksum status: Unverified]
Source: 54.231.114.156
Destination: 192.168.6.253
Transmission Control Protocol, Src Port: 443, Dst Port: 61617, Seq: 1, Ack: 1, Len: 0
Source Port: 443
Destination Port: 61617
[Stream index: 0]
[TCP Segment Len: 0]
Sequence number: 1 (relative sequence number)
Acknowledgment number: 1 (relative ack number)
Header Length: 32 bytes
Flags: 0x010 (ACK)
000. .... .... = Reserved: Not set
...0 .... .... = Nonce: Not set
.... 0... .... = Congestion Window Reduced (CWR): Not set
.... .0.. .... = ECN-Echo: Not set
.... ..0. .... = Urgent: Not set
.... ...1 .... = Acknowledgment: Set
.... .... 0... = Push: Not set
.... .... .0.. = Reset: Not set
.... .... ..0. = Syn: Not set
.... .... ...0 = Fin: Not set
[TCP Flags: ·······A····]
Window size value: 328
[Calculated window size: 328]
[Window size scaling factor: -1 (unknown)]
Checksum: 0xe37d [unverified]
[Checksum Status: Unverified]
Urgent pointer: 0
Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), SACK
TCP Option - No-Operation (NOP)
Kind: No-Operation (1)
TCP Option - No-Operation (NOP)
Kind: No-Operation (1)
TCP Option - SACK 513-1025
Kind: SACK (5)
Length: 10
left edge = 513 (relative)
right edge = 1025 (relative)
[TCP SACK Count: 1]
</code></pre>
</div>
<p>_</p>Dave Goodell[tl;dr: per-packet coloring is now supported in tshark (command-line Wireshark) with --color] I regularly use tshark when doing protocol work, often several times per week. It’s a great way to take load off my brain instead of constantly parsing packets with my eyeballs directly from a raw hex dump. tshark is like tcpdump on steroids, providing me the majority of the benefits of Wireshark but without having to leave the terminal, use the mouse, or slurp a .pcap file around between machines. With tshark I can usually just view the .pcap file on the same server I used to capture it.radix-calc: a programmer’s calculator for the CLI and Alfred2017-06-04T23:27:00-05:002017-06-04T23:27:00-05:00//2017/06/04/radix-calc-a-programmers-calculator-cli-alfred<p><em>tl;dr: a CLI-based and Alfred-based programmer’s calculator
<a href="https://github.com/goodell/radix-calc">is now available on GitHub</a> and via
<code class="highlighter-rouge">cargo install radix-calc</code></em></p>
<p><img src="/images/radix-calc-alfred1.png" alt="screenshot of the radix-calc Alfred workflow" /></p>
<p>At work I deal with a <em>lot</em> of hexadecimal and binary arithmetic. Some of it
I can do in my head with little effort – most addition is pretty easy, most
hex/decimal/binary conversions are easy for small values – but the rest of it
is tedious and distracting to convert by hand. There are lots of tools out
there to help with these calculations but none of them met my needs:</p>
<!--fold-->
<ol>
<li>It should always be near at hand, even when working on a remote
ssh/telnet/serial connection.</li>
<li>It understands decimal and hexadecimal (and ideally octal+binary as well)
input <strong>and</strong> output.</li>
<li>It interacts conveniently with my clipboard.</li>
<li>(bonus points) It performs speaks infix notation, not postfix/RPN.</li>
</ol>
<p>So to solve this problem I built a new CLI tool (<code class="highlighter-rouge">radix-calc</code>) that
understands typical infix expressions for integer programming arithmetic. On
top of that I layered an Alfred workflow that effectively replaces the builtin
calculator (e.g, <code class="highlighter-rouge">rc 5+5</code> instead of <code class="highlighter-rouge">=5+5</code>).</p>
<p>I wrote the expression parser using Rust and the
<a href="https://github.com/kevinmehall/rust-peg/">rust-peg crate</a>. Partly this was
for speed and safety, but primarily it was about learning a bit of Rust, which
has looked exciting for a while now. It would have been much faster to throw
this together in C (because I write it most days of the week) or with a
scripting language like Ruby or Python. Working with Rust was a lot of fun
and I’m looking forward to future projects using it.</p>
<p>You can get <a href="https://github.com/goodell/radix-calc">the source from GitHub</a>.
If you want to install the Alfred workflow, that can also be
<a href="https://github.com/goodell/radix-calc/raw/master/alfred/radix-calc.alfredworkflow">downloaded directly from GitHub</a>.
If you are only interested in the CLI utility and you already have <code class="highlighter-rouge">cargo</code>
installed, then <code class="highlighter-rouge">cargo install radix-calc</code> will do the trick.</p>
<p>Here’s a short demo of the CLI:
<script type="text/javascript" src="https://asciinema.org/a/54b0nfwpvc8lkjtend2fef574.js" id="asciicast-54b0nfwpvc8lkjtend2fef574" async=""></script></p>Dave Goodelltl;dr: a CLI-based and Alfred-based programmer’s calculator is now available on GitHub and via cargo install radix-calc At work I deal with a lot of hexadecimal and binary arithmetic. Some of it I can do in my head with little effort – most addition is pretty easy, most hex/decimal/binary conversions are easy for small values – but the rest of it is tedious and distracting to convert by hand. There are lots of tools out there to help with these calculations but none of them met my needs:vim-mscgen available on GitHub2016-08-27T15:27:00-05:002016-08-27T15:27:00-05:00//2016/08/27/vim-mscgen<p>[tl;dr: Vim syntax highlighting for the mscgen, msgenny, Xù languages is now
<a href="https://github.com/goodell/vim-mscgen">available on GitHub</a>.]</p>
<p>At work I’ve been working on a large design document that involves some wire
protocol work. It’s often helpful when explaining wire protocols to use
packet/protocol sequence diagrams (sometimes called ladder diagrams, but that
term is ambiguous if you’re a different type of electrical engineer).</p>
<!--fold-->
<p>Here’s an example:</p>
<p><img src="/images/mscgen-sip.svg" alt="" /></p>
<p>While drawing one of these is pretty simple on a whiteboard or even a piece of
paper, it’s more painful to do in PowerPoint, Inkscape, etc., especially if
you need to keep modifying the diagram as the protocol or example evolves.
So you can imagine how happy I was to discover the <a href="http://www.mcternan.me.uk/mscgen/">mscgen
language</a> and, better yet, the
<a href="https://github.com/sverweij/mscgen_js">mscgen.js suite of tools</a>.</p>
<p>You can think of the mscgen language as analogous to the
<a href="https://en.wikipedia.org/wiki/DOT_(graph_description_language)">DOT language</a>
and mscgen.js analogous to the
<a href="https://en.wikipedia.org/wiki/Graphviz">Graphviz package</a> for drawing
directed and undirected graphs.</p>
<p>Here’s an example of the mscgen code that generates the above image:</p>
<div class="highlighter-rouge"><pre class="highlight"><code># SIP protocol (source: wikipedia)
msc {
UAS, UAC;
UAS -> UAC [label="INVITE"];
UAS <- UAC [label="100 Trying"];
UAS <- UAC [label="180 Ringing"];
--- [label="the client plays ringing"];
UAS <- UAC [label="200 Ok"];
UAS -> UAC [label="ACK"];
...;
--- [label="the client hangs up"];
UAS <- UAC [label="BYE"];
UAS <- UAC [label="200 Ok"];
}
</code></pre>
</div>
<p>And here’s that same file in
<a href="https://github.com/sverweij/mscgen_js/blob/master/wikum/msgenny.md">the msgenny language</a>,
a relaxed and simplified version of mscgen (sort of like
<a href="https://hjson.org/">Hjson</a> is to JSON):</p>
<div class="highlighter-rouge"><pre class="highlight"><code># SIP protocol (source: wikipedia)
UAS, UAC;
UAS -> UAC : INVITE;
UAS <- UAC : 100 Trying;
UAS <- UAC : 180 Ringing;
--- : the client plays ringing;
UAS <- UAC : 200 Ok;
UAS -> UAC : ACK;
...;
--- : the client hangs up;
UAS <- UAC : BYE;
UAS <- UAC : 200 Ok;
</code></pre>
</div>
<p>You can even play around with these languages <a href="http://bit.ly/2a6JqYp">in your browser</a>!</p>
<p>Editing those files in Vim is not a big deal, but I figured it could be made a
little more pleasant with some syntax highlighting and other goodies. Hence,
<a href="https://github.com/goodell/vim-mscgen">vim-mscgen</a> was born in my spare time.</p>
<p>vim-mscgen includes:</p>
<ul>
<li>syntax highlighting</li>
<li>file type detection (based on file extension being
<code class="highlighter-rouge">.mscgen</code>/<code class="highlighter-rouge">.msgenny</code>/<code class="highlighter-rouge">.mscin</code>/<code class="highlighter-rouge">.xu</code>)</li>
<li>a rough <code class="highlighter-rouge">errorformat</code> setting for the
<a href="https://github.com/sverweij/mscgenjs-cli">mscgenjs-cli</a> utility</li>
</ul>
<p>Enjoy!</p>Dave Goodell[tl;dr: Vim syntax highlighting for the mscgen, msgenny, Xù languages is now available on GitHub.] At work I’ve been working on a large design document that involves some wire protocol work. It’s often helpful when explaining wire protocols to use packet/protocol sequence diagrams (sometimes called ladder diagrams, but that term is ambiguous if you’re a different type of electrical engineer).Blog Moved to GitHub Pages2016-08-27T14:27:00-05:002016-08-27T14:27:00-05:00//2016/08/27/github-pages<p>A few weeks ago I moved this blog from Blogger (davegoodell.blogspot.com) to
GitHub Pages (blog.goodell.io). Redirects should be in place from any old
links to their new home, but it’s possible that I missed something somehow.
I also updated the underlying feed backing
<a href="http://feeds.feedburner.com/davegoodellblog">the Feed Burner feed</a>, so any
subscribers using that might have seen the migration as a reposting of all the
old posts. Sorry for the spam in your feed reader.</p>
<p>If you see any problems, drop me a line
<a href="mailto:davidjgoodell@gmail.com">via email</a> or
<a href="https://twitter.com/davidjgoodell">on Twitter</a> and I’ll straighten things out.</p>Dave GoodellA few weeks ago I moved this blog from Blogger (davegoodell.blogspot.com) to GitHub Pages (blog.goodell.io). Redirects should be in place from any old links to their new home, but it’s possible that I missed something somehow. I also updated the underlying feed backing the Feed Burner feed, so any subscribers using that might have seen the migration as a reposting of all the old posts. Sorry for the spam in your feed reader. If you see any problems, drop me a line via email or on Twitter and I’ll straighten things out.dump packets in memory buffers with tshark without leaving gdb2014-10-01T22:50:00-05:002014-10-01T22:50:00-05:00//2014/10/01/dump-packets-in-memory-buffers-with<p>I got tired of running my error-prone eyeball packet decoder on GDB buffers, so I took a few minutes to figure out how to send packet data to <code class="highlighter-rouge">tshark</code> to let the computer do what it’s good at. Here’s the <code class="highlighter-rouge">~/.gdbinit</code> snippet if you want to use it:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>define pktdump
dump binary memory /tmp/dgoodell-pktdump.bin (char*)$arg0 (((char*)$arg0)+$arg1)
shell od -Ax -tx1 -v /tmp/dgoodell-pktdump.bin | text2pcap - - | tshark -V -x -i -
end
document pktdump
Dump a network packet in memory with tshark.
Example usage: pktdump ADDRESS LENGTH
end
</code></pre>
</div>
<p>(obviously change “<code class="highlighter-rouge">dgoodell</code>” to your user name or stick the file in your home directory, environment variables don’t work in GDB, unfortunately)
The results look like:</p>
<!--fold-->
<div class="highlighter-rouge"><pre class="highlight"><code>(gdb) x/16xb hdr_buf
0x60f5d0: 0xfc 0x99 0x47 0x25 0x2c 0x13 0xfc 0x99
0x60f5d8: 0x47 0x25 0x42 0xbd 0x08 0x00 0x45 0x88
(gdb) help pktdump
Dump a network packet in memory with tshark.
Example usage: pktdump ADDRESS LENGTH
(gdb) pktdump hdr_buf 64
Input from: Standard input
Output to: Standard output
Wrote packet of 64 bytes at 0
Read 1 potential packet, wrote 1 packet
Capturing on Standard input
Frame 1: 64 bytes on wire (512 bits), 64 bytes captured (512 bits) on interface 0
Interface id: 0
WTAP_ENCAP: 1
Arrival Time: Oct 1, 2014 14:46:42.000000000 PDT
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1412200002.000000000 seconds
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 0.000000000 seconds]
Frame Number: 1
Frame Length: 64 bytes (512 bits)
Capture Length: 64 bytes (512 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ip:udp:data]
Ethernet II, Src: Cisco_25:42:bd (fc:99:47:25:42:bd), Dst: Cisco_25:2c:13 (fc:99:47:25:2c:13)
Destination: Cisco_25:2c:13 (fc:99:47:25:2c:13)
Address: Cisco_25:2c:13 (fc:99:47:25:2c:13)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: Cisco_25:42:bd (fc:99:47:25:42:bd)
Address: Cisco_25:42:bd (fc:99:47:25:42:bd)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: IP (0x0800)
Padding: 0000
Frame check sequence: 0x00000000 [incorrect, should be 0x2cf4146a]
[FCS Good: False]
[FCS Bad: True]
[Expert Info (Error/Checksum): Bad checksum]
[Message: Bad checksum]
[Severity level: Error]
[Group: Checksum]
Internet Protocol Version 4, Src: 10.10.0.1 (10.10.0.1), Dst: 10.10.0.2 (10.10.0.2)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x88 (DSCP 0x22: Assured Forwarding 41; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
1000 10.. = Differentiated Services Codepoint: Assured Forwarding 41 (0x22)
.... ..00 = Explicit Congestion Notification: Not-ECT (Not ECN-Capable Transport) (0x00)
Total Length: 44
Identification: 0x0000 (0)
Flags: 0x00
0... .... = Reserved bit: Not set
.0.. .... = Don't fragment: Not set
..0. .... = More fragments: Not set
Fragment offset: 0
Time to live: 8
Protocol: UDP (17)
Header checksum: 0x9e23 [correct]
[Good: True]
[Bad: False]
Source: 10.10.0.1 (10.10.0.1)
Destination: 10.10.0.2 (10.10.0.2)
User Datagram Protocol, Src Port: 34066 (34066), Dst Port: 47559 (47559)
Source port: 34066 (34066)
Destination port: 47559 (47559)
Length: 24
Checksum: 0xf4fe [validation disabled]
[Good Checksum: False]
[Bad Checksum: False]
Data (16 bytes)
Data: 00000000000000000000000000000000
[Length: 16]
0000 fc 99 47 25 2c 13 fc 99 47 25 42 bd 08 00 45 88 ..G%,...G%B...E.
0010 00 2c 00 00 00 00 08 11 9e 23 0a 0a 00 01 0a 0a .,.......#......
0020 00 02 85 12 b9 c7 00 18 f4 fe 00 00 00 00 00 00 ................
0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
</code></pre>
</div>
<p>That’s also way easier to read on x86 than casting to a <code class="highlighter-rouge">struct ether_header *</code>, etc. and dereferencing, especially on x86 because all the fields end up displayed in the wrong endianness.</p>
<h3 id="addendum-2017-05-07">Addendum 2017-05-07</h3>
<p>If you’ve got GDB ≥v7.0 (and therefore python support), you can get around the lack of simple environment variable support with this version of the snippet:</p>
<div class="language-python highlighter-rouge"><pre class="highlight"><code><span class="n">python</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="k">class</span> <span class="nc">PktDump</span> <span class="p">(</span><span class="n">gdb</span><span class="o">.</span><span class="n">Command</span><span class="p">):</span>
<span class="s">"""Dump a network packet in memory with tshark.
Example usage: pktdump ADDRESS LENGTH"""</span>
<span class="k">def</span> <span class="nf">__init__</span> <span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="nb">super</span> <span class="p">(</span><span class="n">PktDump</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">__init__</span> <span class="p">(</span><span class="s">"pktdump"</span><span class="p">,</span> <span class="n">gdb</span><span class="o">.</span><span class="n">COMMAND_DATA</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">invoke</span> <span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">arg</span><span class="p">,</span> <span class="n">from_tty</span><span class="p">):</span>
<span class="n">argv</span> <span class="o">=</span> <span class="n">gdb</span><span class="o">.</span><span class="n">string_to_argv</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span>
<span class="n">tmpfile</span> <span class="o">=</span> <span class="s">'/tmp/'</span> <span class="o">+</span> <span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">[</span><span class="s">'USER'</span><span class="p">]</span> <span class="o">+</span> <span class="s">'-pktdump.bin'</span>
<span class="n">gdb</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s">'dump binary memory '</span> <span class="o">+</span> <span class="n">tmpfile</span> <span class="o">+</span> <span class="s">' (char*)'</span> <span class="o">+</span> <span class="n">argv</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">+</span> <span class="s">' (((char*)'</span> <span class="o">+</span> <span class="n">argv</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">+</span> <span class="s">')+'</span> <span class="o">+</span> <span class="n">argv</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">+</span> <span class="s">')'</span><span class="p">)</span>
<span class="n">gdb</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s">'shell od -Ax -tx1 -v '</span> <span class="o">+</span> <span class="n">tmpfile</span> <span class="o">+</span> <span class="s">' | text2pcap - - | tshark -V -x -i -'</span><span class="p">)</span>
<span class="n">PktDump</span><span class="p">()</span>
<span class="n">end</span>
</code></pre>
</div>Dave GoodellI got tired of running my error-prone eyeball packet decoder on GDB buffers, so I took a few minutes to figure out how to send packet data to tshark to let the computer do what it’s good at. Here’s the ~/.gdbinit snippet if you want to use it: define pktdump dump binary memory /tmp/dgoodell-pktdump.bin (char*)$arg0 (((char*)$arg0)+$arg1) shell od -Ax -tx1 -v /tmp/dgoodell-pktdump.bin | text2pcap - - | tshark -V -x -i - end document pktdump Dump a network packet in memory with tshark. Example usage: pktdump ADDRESS LENGTH end (obviously change “dgoodell” to your user name or stick the file in your home directory, environment variables don’t work in GDB, unfortunately) The results look like:Text::CharWidth::PurePerl available on GitHub2012-05-11T23:21:00-05:002012-05-11T23:21:00-05:00//2012/05/11/textcharwidthpureperl-available-on<p>I’ve put up a new perl module on github, <a href="https://github.com/goodell/text-charwidth-pureperl">Text::CharWidth::PurePerl</a>. As the name suggests, it’s a pure-perl implementation of <a href="http://search.cpan.org/%7Ekubota/Text-CharWidth-0.04/CharWidth.pm">Text::CharWidth</a>. I needed it to be able to ship <a href="https://github.com/goodell/nowrap">nowrap</a> as a script without any dependencies. Amusingly, it seems to be more correct than the “native” <code class="highlighter-rouge">wcwidth</code> routine.</p>
<p>It is packaged in the usual CPAN format, but I’m not currently planning on uploading it to CPAN (I don’t have a CPAN id right now). That seems like a real PITA right now, and I’m not sure if anyone else might even need it. If you do want me to go through the steps, feel free to add a comment here on the blog or over on the github issue tracker.</p>Dave GoodellI’ve put up a new perl module on github, Text::CharWidth::PurePerl. As the name suggests, it’s a pure-perl implementation of Text::CharWidth. I needed it to be able to ship nowrap as a script without any dependencies. Amusingly, it seems to be more correct than the “native” wcwidth routine. It is packaged in the usual CPAN format, but I’m not currently planning on uploading it to CPAN (I don’t have a CPAN id right now). That seems like a real PITA right now, and I’m not sure if anyone else might even need it. If you do want me to go through the steps, feel free to add a comment here on the blog or over on the github issue tracker.Debugging Wily Stack Overflows in MPI Programs2011-04-15T00:05:00-05:002011-04-15T00:05:00-05:00//2011/04/15/debugging-wily-stack-overflows-in-mpi<p>I recently was debugging a problem in MPICH2 with the new MPI-3 nonblocking collectives implementation that involved a stack overflow. The code in question was usually blowing up with a <code class="highlighter-rouge">SIGABRT</code> that came from a stack trace ending in <code class="highlighter-rouge">__stack_chk_fail</code>. This bug was particularly nasty because it was a <a href="http://en.wikipedia.org/wiki/Unusual_software_bug#Heisenbug">Heisenbug</a>: it kept changing the failure location and mode whenever I would insert debugging code in order to track it down (hence the aforementioned “usually”).</p>
<!--fold-->
<p>For reasons I will explain later, <a href="http://davegoodell.blogspot.com/2009/07/fstack-protector-valgrind-stack-array.html">my usual trick</a> for this sort of thing did not help. Adding in those macros would cause the bug to vanish, or to reappear in a different function that had not been instrumented with <code class="highlighter-rouge">MPIU_SG_</code> macros.</p>
<p>Luckily, I ran across <a href="http://www.outflux.net/blog/archives/2007/09/15/catching-stack-overflows-in-gdb-as-they-happen/">this very helpful blog post</a> that eventually helped me track down the real problem. My basic approach was the same as the one outlined there, but it needed modification due to the pain inherent in debugging a parallel MPI application. This particular bug required running at least 4 processes in order to manifest itself, and it would only appear after several iterations within the test program. So I used <code class="highlighter-rouge">mpiexec</code> + <code class="highlighter-rouge">xterm</code> + <code class="highlighter-rouge">gdb</code> + a gdb script file to handle the job.</p>
<p>First, here’s an example stack trace of the failure (using valgrind to easily obtain stack traces):</p>
<div class="highlighter-rouge"><pre class="highlight"><code>% mpiexec -l -n 4 valgrind -q ./icbarrier |& sort_labeled_output
=====================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= EXIT CODE: 6
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
=====================================================================================
APPLICATION TERMINATED WITH THE EXIT STRING: Abort trap (signal 6)
[1] ==47313==
[1] ==47313== Process terminating with default action of signal 6 (SIGABRT)
[1] ==47313== at 0x10027D616: __kill (in /usr/lib/libSystem.B.dylib)
[1] ==47313== by 0x1003022CB: __stack_chk_fail (in /usr/lib/libSystem.B.dylib)
[1] ==47313== by 0x10005D9D3: MPIDI_CH3I_Progress (ch3_progress.c:539)
[1] ==47313== by 0x100109B0C: MPIR_Wait_impl (wait.c:67)
[1] ==47313== by 0x100109FFA: PMPI_Wait (wait.c:171)
[1] ==47313== by 0x100004506: MPI_Barrier (nbc_pmpi_adapter.c:24)
[1] ==47313== by 0x10000141D: main (icbarrier.c:37)
[2] ==47314==
[2] ==47314== Process terminating with default action of signal 6 (SIGABRT)
[2] ==47314== at 0x10027D616: __kill (in /usr/lib/libSystem.B.dylib)
[2] ==47314== by 0x1003022CB: __stack_chk_fail (in /usr/lib/libSystem.B.dylib)
[2] ==47314== by 0x10005D9D3: MPIDI_CH3I_Progress (ch3_progress.c:539)
[2] ==47314== by 0x100109B0C: MPIR_Wait_impl (wait.c:67)
[2] ==47314== by 0x100109FFA: PMPI_Wait (wait.c:171)
[2] ==47314== by 0x100004506: MPI_Barrier (nbc_pmpi_adapter.c:24)
[2] ==47314== by 0x10000143B: main (icbarrier.c:45)
[3] ==47315==
[3] ==47315== Process terminating with default action of signal 6 (SIGABRT)
[3] ==47315== at 0x10027D616: __kill (in /usr/lib/libSystem.B.dylib)
[3] ==47315== by 0x1003022CB: __stack_chk_fail (in /usr/lib/libSystem.B.dylib)
[3] ==47315== by 0x10005D9D3: MPIDI_CH3I_Progress (ch3_progress.c:539)
[3] ==47315== by 0x100109B0C: MPIR_Wait_impl (wait.c:67)
[3] ==47315== by 0x100109FFA: PMPI_Wait (wait.c:171)
[3] ==47315== by 0x100004506: MPI_Barrier (nbc_pmpi_adapter.c:24)
[3] ==47315== by 0x10000143B: main (icbarrier.c:45)
</code></pre>
</div>
<p>So I took a quick gander at the assembly for <code class="highlighter-rouge">MPIDI_CH3I_Progress</code> to find the canary value initialization and comparison code:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>% objdump -d icbarrier > icbarrier.s
% sed -n -e '/MPIDI_CH3I_Progress>:/,/^$/ p' icbarrier.s | head -n 15
000000010005c7f0 <_MPIDI_CH3I_Progress>:
10005c7f0: 55 push %rbp
10005c7f1: 48 89 e5 mov %rsp,%rbp
10005c7f4: 41 57 push %r15
10005c7f6: 41 56 push %r14
10005c7f8: 41 55 push %r13
10005c7fa: 41 54 push %r12
10005c7fc: 53 push %rbx
10005c7fd: 48 81 ec 08 02 00 00 sub $0x208,%rsp
10005c804: 48 89 bd 10 fe ff ff mov %rdi,-0x1f0(%rbp)
10005c80b: 89 b5 0c fe ff ff mov %esi,-0x1f4(%rbp)
10005c811: 48 8b 05 60 c8 10 00 mov 0x10c860(%rip),%rax # 100169078 <_pvars+0x78>
10005c818: 48 8b 10 mov (%rax),%rdx
10005c81b: 48 89 55 c8 mov %rdx,-0x38(%rbp)
10005c81f: 31 d2 xor %edx,%edx
</code></pre>
</div>
<p>So in this code it looks like the offset is <code class="highlighter-rouge">-0x38</code> rather than <code class="highlighter-rouge">-0x14</code>. Let’s find the two addresses at which we will set our breakpoints:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>% sed -n -e '/MPIDI_CH3I_Progress>:/,/^$/ p' icbarrier.s | grep -- '-0x38(%rbp)'
10005c81b: 48 89 55 c8 mov %rdx,-0x38(%rbp)
10005ce61: 48 8b 55 c8 mov -0x38(%rbp),%rdx
</code></pre>
</div>
<p>And now for our gdb script:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>% cat gdb.script
break *0x10005c81b
break *0x10005ce61
commands 1
silent
set variable $cow = (unsigned long*)($rbp - 0x38)
watch *$cow
continue
end
set variable $count = 3
commands 2
silent
delete $count
set variable $count = $count + 1
continue
end
run
</code></pre>
</div>
<div class="highlighter-rouge"><pre class="highlight"><code>% mpiexec -n 4 xterm -e gdb -x gdb.script ./icbarrier
</code></pre>
</div>
<p>That command launches four xterm windows, each containing a single gdb instance that was running <code class="highlighter-rouge">icbarrier</code> as the inferior. Here’s the resulting xterm+gdb screenshot for the process that hit the watchpoint first:
<a href="http://3.bp.blogspot.com/-g7_aRxJ74GU/TafKKWBmiTI/AAAAAAAAACQ/R9T2XpDQgcA/s1600/gdb_screenshot.png"><img src="http://3.bp.blogspot.com/-g7_aRxJ74GU/TafKKWBmiTI/AAAAAAAAACQ/R9T2XpDQgcA/s1600/gdb_screenshot.png" alt="" /></a></p>
<p>Unlike in many situations, this backtrace alone was not sufficient for me to figure out what the real bug was. But, given that I changed some high level code (the nonblocking version of barrier, <code class="highlighter-rouge">MPIX_Ibarrier</code>), I knew it was extremely unlikely that I had tickled some long-latent bug in the frequently used routine <code class="highlighter-rouge">MPIDI_CH3U_Request_unpack_uebuf</code>. So after a few “WTFs” and some further head scratching, I took a hard look at the upper level code (error checking and boring code omitted for clarity):</p>
<div class="language-c highlighter-rouge"><pre class="highlight"><code><span class="kt">int</span> <span class="nf">MPIR_Ibarrier_inter</span><span class="p">(</span><span class="n">MPID_Comm</span> <span class="o">*</span><span class="n">comm_ptr</span><span class="p">,</span> <span class="n">MPID_Sched_t</span> <span class="n">s</span><span class="p">)</span>
<span class="p">{</span>
<span class="kt">int</span> <span class="n">mpi_errno</span> <span class="o">=</span> <span class="n">MPI_SUCCESS</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">size</span><span class="p">,</span> <span class="n">rank</span><span class="p">,</span> <span class="n">root</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">buf</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="cm">/* do a barrier on the local intracommunicator */</span>
<span class="n">mpi_errno</span> <span class="o">=</span> <span class="n">MPIR_Ibarrier_intra</span><span class="p">(</span><span class="n">comm_ptr</span><span class="o">-></span><span class="n">local_comm</span><span class="p">,</span> <span class="n">s</span><span class="p">);</span>
<span class="n">MPID_SCHED_BARRIER</span><span class="p">(</span><span class="n">s</span><span class="p">);</span>
<span class="cm">/* rank 0 on each group does an intercommunicator broadcast to the
remote group to indicate that all processes in the local group
have reached the barrier. We do a 1-byte bcast because a 0-byte
bcast will just return without doing anything. */</span>
<span class="cm">/* first broadcast from left to right group, then from right to
left group */</span>
<span class="k">if</span> <span class="p">(</span><span class="n">comm_ptr</span><span class="o">-></span><span class="n">is_low_group</span><span class="p">)</span> <span class="p">{</span>
<span class="n">root</span> <span class="o">=</span> <span class="p">(</span><span class="n">rank</span> <span class="o">==</span> <span class="mi">0</span><span class="p">)</span> <span class="o">?</span> <span class="n">MPI_ROOT</span> <span class="o">:</span> <span class="n">MPI_PROC_NULL</span><span class="p">;</span>
<span class="n">mpi_errno</span> <span class="o">=</span> <span class="n">MPIR_Ibcast_inter</span><span class="p">(</span><span class="o">&</span><span class="n">buf</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">MPI_BYTE</span><span class="p">,</span> <span class="n">root</span><span class="p">,</span> <span class="n">comm_ptr</span><span class="p">,</span> <span class="n">s</span><span class="p">);</span>
<span class="n">MPID_SCHED_BARRIER</span><span class="p">(</span><span class="n">s</span><span class="p">);</span>
<span class="cm">/* receive bcast from right */</span>
<span class="n">root</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="n">mpi_errno</span> <span class="o">=</span> <span class="n">MPIR_Ibcast_inter</span><span class="p">(</span><span class="o">&</span><span class="n">buf</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">MPI_BYTE</span><span class="p">,</span> <span class="n">root</span><span class="p">,</span> <span class="n">comm_ptr</span><span class="p">,</span> <span class="n">s</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">else</span> <span class="p">{</span>
<span class="cm">/* receive bcast from left */</span>
<span class="n">root</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="n">mpi_errno</span> <span class="o">=</span> <span class="n">MPIR_Ibcast_inter</span><span class="p">(</span><span class="o">&</span><span class="n">buf</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">MPI_BYTE</span><span class="p">,</span> <span class="n">root</span><span class="p">,</span> <span class="n">comm_ptr</span><span class="p">,</span> <span class="n">s</span><span class="p">);</span> <span class="n">MPID_SCHED_BARRIER</span><span class="p">(</span><span class="n">s</span><span class="p">);</span> <span class="cm">/* bcast to left */</span>
<span class="n">root</span> <span class="o">=</span> <span class="p">(</span><span class="n">rank</span> <span class="o">==</span> <span class="mi">0</span><span class="p">)</span> <span class="o">?</span> <span class="n">MPI_ROOT</span> <span class="o">:</span> <span class="n">MPI_PROC_NULL</span><span class="p">;</span>
<span class="n">mpi_errno</span> <span class="o">=</span> <span class="n">MPIR_Ibcast_inter</span><span class="p">(</span><span class="o">&</span><span class="n">buf</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">MPI_BYTE</span><span class="p">,</span> <span class="n">root</span><span class="p">,</span> <span class="n">comm_ptr</span><span class="p">,</span> <span class="n">s</span><span class="p">);</span>
<span class="p">}</span>
<span class="nl">fn_exit:</span>
<span class="k">return</span> <span class="n">mpi_errno</span><span class="p">;</span>
<span class="nl">fn_fail:</span>
<span class="k">goto</span> <span class="n">fn_exit</span><span class="p">;</span>
<span class="p">}</span>
</code></pre>
</div>
<p>This function is called, adding operations to a communication schedule, then an <code class="highlighter-rouge">MPI_Wait</code> call is made later to actually force the bulk of the schedule to complete.</p>
<p>Do you see the bug? I’d be surprised if you did, since there are few people familiar with the style of MPICH2 code, and even fewer people who know how the NBC implementation works. But if you do, give yourself a pat on the back, and then please send me bug reports and patches for my other bugs :)</p>
<p>Take a closer look at the buffer used for the intercomm broadcast. It’s a stack allocated variable, which worked great in the blocking version of this routine. Unfortunately, this is a nonblocking barrier, so this routine exits before the ibcast is actually ever performed. So now we’ve managed to schedule a broadcast to occur that will overwrite a location on the stack that no longer actually refers to <code class="highlighter-rouge">int buf;</code>. Instead, if we are unlucky it will refer to some random piece of a different function’s stack frame. In the backtraces above, this random piece was the <code class="highlighter-rouge">-fstack-protector</code>/SSP canary memory. The bug moved around when I would insert debugging variables because they would affect the specific location of the canary value in memory, sometimes yielding a corrupted parameter or local variable instead.</p>
<p><a href="http://git.mpich.org/mpich.git/commit/fc1d5cedd250d02b65bf226e56f0dbf200b12d29">Changing that buffer</a> to be <code class="highlighter-rouge">malloc</code>ed instead fixed the problem.</p>Dave GoodellI recently was debugging a problem in MPICH2 with the new MPI-3 nonblocking collectives implementation that involved a stack overflow. The code in question was usually blowing up with a SIGABRT that came from a stack trace ending in __stack_chk_fail. This bug was particularly nasty because it was a Heisenbug: it kept changing the failure location and mode whenever I would insert debugging code in order to track it down (hence the aforementioned “usually”).Roku stands behind their product2011-04-03T21:31:00-05:002011-04-03T21:31:00-05:00//2011/04/03/roku-stands-behind-their-product<p>My older Roku player (<a href="http://www.amazon.com/Roku-N1100-HD-Player/dp/B001PIBE8I">this model</a>, N1000-B) died a couple of months ago. The wireless NIC went out to lunch for some reason, showing 00:00:00:00:00:00 as its MAC address and totally unable to see any wireless networks, just <a href="http://forums.roku.com/viewtopic.php?f=28&t=29830&sid=7daae75e06560f6d33fc6c86ed639bc0">as described here</a>. I was busy and lazy at the time, so we just switched over to watching Netflix streaming content on our Wii instead.</p>
<!--fold-->
<p>Last Friday I finally got around to calling Roku about the issue and the technician that I got was very helpful. He didn’t give me the “reboot runaround”; he just asked me what troubleshooting steps I had taken, and ensured that my MAC address was showing all zeroes. After a bit more discussion, he offered to replace the unit for free, even though it was nearly a past its warranty expiration. They mailed the replacement unit (looks like it might be factory refurb?) via FedEx and 3 business days later the new unit was in my hands. The new unit setup was painless (as expected), and a return shipping label was included so even the round-trip shipping was free.</p>
<p>I was extremely pleased that Roku stood behind their product, even after it was out of warranty. They did exactly the right thing to change my mind from “I’ll never buy another one of these” to “I’d happily buy something from them again.”</p>
<p>(full disclosure: I have no affiliation with Roku, other than being a fairly satisfied customer)</p>Dave GoodellMy older Roku player (this model, N1000-B) died a couple of months ago. The wireless NIC went out to lunch for some reason, showing 00:00:00:00:00:00 as its MAC address and totally unable to see any wireless networks, just as described here. I was busy and lazy at the time, so we just switched over to watching Netflix streaming content on our Wii instead.m4_include and aclocal2011-03-05T10:50:00-06:002011-03-05T10:50:00-06:00//2011/03/05/m4include-and-aclocal<p>I ran into some surprising behavior a little while back while hacking on the MPICH2 build system, that I thought I should mention here. But first, some background info is in order. We have numerous subsystems that are currently configured (in the autoconf sense) by running an autoconf-generated configure script from various points in the top level configure. Think <code class="highlighter-rouge">AC_CONFIG_SUBDIRS</code>, but the subconfig happens at the place where the macro is used, rather than being deferred to <code class="highlighter-rouge">config.status</code>-time. There are some advantages to this scheme, but there are two fairly severe disadvantages:</p>
<!--fold-->
<ol>
<li>The subconfigure scripts tend to re-test a whole bunch of information about the build environment (e.g., six repeated checks for “how to run the C preprocessor”). Not only is this wasteful in terms of time, but there is always unlikely possibility that two configure scripts will come to subtly different conclusions about the build environment, which could cause wacky bugs later in the build.</li>
<li>There’s no easy mechanism to utilize the results of subconfigure tests that should be global. That is, if we add <code class="highlighter-rouge">-I/path/to/includes</code> to <code class="highlighter-rouge">CPPFLAGS</code> that should be used for all compilation, then this won’t be seen in the top-level configure without some additional work. We currently “pass” these values back via a “localdefs” file that is listed in the subconfigure’s <code class="highlighter-rouge">AC_CONFIG_FILES</code> and contains some <code class="highlighter-rouge">@FOO@</code> substitutions that are either <code class="highlighter-rouge">AC_SUBST</code>ed or “precious” (which implies <code class="highlighter-rouge">AC_SUBST</code> for that var). This stinks because it’s a fairly manual process, and it’s impenetrable to outsiders who are unfamiliar with our build system because the sourcing of “localdefs” is hidden inside a custom autoconf macro.</li>
</ol>
<p>So I’m taking a stab at converting these configure files to be configure.in fragments that are then <code class="highlighter-rouge">m4_include</code>d. This will cause those tests to be run in the same shell as the top-level configure and will only run the additional tests that are needed by the subsystem. In the process of that conversion, I ran into some odd error messages from the autotools that are represented by the following example program.</p>
<p><em>Quick Quiz</em>: The code in Listing A will successfully <code class="highlighter-rouge">autoreconf</code> and <code class="highlighter-rouge">configure</code>. The code in Listing B will fail during <code class="highlighter-rouge">autoreconf</code>’s invocation of <code class="highlighter-rouge">aclocal</code>. Why?</p>
<hr />
<p><strong>Common code</strong>:</p>
<div class="language-sh highlighter-rouge"><pre class="highlight"><code><span class="c">## contents of configure.in</span>
AC_INIT<span class="o">([</span>foo],[1.0]<span class="o">)</span>
AM_INIT_AUTOMAKE<span class="o">([</span>-Wall -Werror foreign 1.11 subdir-objects]<span class="o">)</span>
LT_INIT<span class="o">([])</span>
AC_PROG_CC
<span class="nv">var</span><span class="o">=</span><span class="s2">"yes"</span>
AM_CONDITIONAL<span class="o">([</span>COND],[test <span class="s2">"</span><span class="nv">$var</span><span class="s2">"</span> <span class="o">=</span> <span class="s2">"yes"</span><span class="o">])</span>
dnl this makes m4 bail, claiming that AM_COND_IF is undefined
m4_foreach<span class="o">([</span>subsys_i],[[subconfigure]],[m4_include<span class="o">(</span>subsys_i[.m4]<span class="o">)])</span>
AC_OUTPUT<span class="o">([</span>Makefile]<span class="o">)</span>
</code></pre>
</div>
<p><strong>Listing A</strong></p>
<div class="highlighter-rouge"><pre class="highlight"><code>## contents of subconfigure.m4
AC_PROG_GREP
</code></pre>
</div>
<p><strong>Listing B</strong></p>
<div class="highlighter-rouge"><pre class="highlight"><code>## contents of subconfigure.m4
AM_COND_IF([COND],[echo COND is true],[echo COND is false])
</code></pre>
</div>
<hr />
<p>Do you see the problem here? I sure didn’t, even after some moderate debugging. Go ahead and think about it for a minute, I’ll wait…</p>
<p>.</p>
<p>.</p>
<p>.</p>
<p>.</p>
<p>.</p>
<p>.</p>
<p>.</p>
<p>.</p>
<p>.</p>
<p>Luckily, the nice folks on the autoconf list <a href="http://thread.gmane.org/gmane.comp.sysutils.autoconf.general/13724/focus=13728">helped me understand what was going on here</a>. There are two things happening here:</p>
<ol>
<li>Not all macros are created equal. In my original world view, <code class="highlighter-rouge">AC_PROG_GREP</code> and <code class="highlighter-rouge">AM_COND_IF</code> are both just <code class="highlighter-rouge">AC_DEFUN</code>ed m4 macros and should behave more or less identically. However this assumption is totally false. <code class="highlighter-rouge">AC_PROG_GREP</code> is a macro that is built-in to autoconf, and therefore does not get distributed with your package via <code class="highlighter-rouge">aclocal.m4</code>, while <code class="highlighter-rouge">AM_COND_IF</code> comes with automake and is managed by aclocal, and must be placed into your package’s <code class="highlighter-rouge">aclocal.m4</code> in order to be properly expanded.</li>
<li>It turns out that aclocal isn’t as smart as I thought it was. When it traces the m4 files looking for macro definitions that it must add to aclocal.m4, it doesn’t perform real <code class="highlighter-rouge">m4_include</code>s at that time. Instead it uses some regex heuristics in order to determine what file is being included and then traces that file separately. Since the name of the file that is being included is being computed via an m4 macro in the example above, aclocal gets rather confused and just doesn’t trace the subconfigure.m4 file.</li>
</ol>
<p>The upshot of all of this is that I can’t use m4 code to programmatically include subsystem m4 fragments. They must either be hard-coded m4 file names or be autogenerated by some other non-autotools step, such as a quick shell script run before <code class="highlighter-rouge">autoreconf</code>.</p>Dave GoodellI ran into some surprising behavior a little while back while hacking on the MPICH2 build system, that I thought I should mention here. But first, some background info is in order. We have numerous subsystems that are currently configured (in the autoconf sense) by running an autoconf-generated configure script from various points in the top level configure. Think AC_CONFIG_SUBDIRS, but the subconfig happens at the place where the macro is used, rather than being deferred to config.status-time. There are some advantages to this scheme, but there are two fairly severe disadvantages:Midwest Ice Climbers Ice Festival, 20112011-01-21T22:46:00-06:002011-01-21T22:46:00-06:00//2011/01/21/midwest-ice-climbers-ice-festival-2011<p><a href="http://goodell.smugmug.com/Travel/Ice-Climbing-2011/15507532_ZrRFp#1161465500_T8Z6X"><img src="http://goodell.smugmug.com/Travel/Ice-Climbing-2011/DSC0261edited-1/1161465500_T8Z6X-L.jpg" alt="" /></a></p>
<p>Last weekend I got to try ice climbing out at the <a href="http://www.theicepit.org/">Ice Pit</a> near Green Bay, Wisconsin. It was a very cool setup, sort of an outdoor ice climbing gym in an active quarry. Normally you must be a member to climb there, but for the festival it was just $20 for the whole weekend. I did the trip with five other guys, all rock climbers, three of which also work at the Lab.</p>
<!--fold-->
<p>I found the climbing itself surprisingly difficult, despite my (at least modest) rock climbing abilities. According to others that I spoke with at the festival, most of the routes that we were climbing were in the <a href="http://en.wikipedia.org/wiki/Ice_climbing#Waterfall_ice_grading">WI3-WI4</a> range, all about 100 feet high. The hardest part was getting accustomed to the crampons and really trusting my feet. When I didn’t trust my feet enough, I would put too much weight on my hands which led to totally pumped out forearms after 100 feet of ice. We were each able to get three climbs in each of the two days, which was enough to tire me out pretty effectively.</p>
<p>There was a fair amount of demo gear to borrow from various vendors, including boots, crampons, and ice tools. However, we got in touch with <a href="http://downwindsports.com/mainSite/rentals/ice-climbing-gear/">Down Wind Sports</a> (in Michigan’s UP) via email before the festival and arranged to rent gear from them on Friday evening instead. That way we were able to save a lot of time each morning and were certain that we would have gear that fit. Bill @ Down Wind Sports was extremely nice and helpful, I’d do business with them again without hesitation. For $40/person we had boots and crampons for the whole weekend. Ice tools were easy to come by as demo gear, and someone in our party already had a set as well.</p>
<p>Overall it was a very enjoyable trip. I might do a little easy ice climbing in the future as the opportunity presents itself, but for now I’m not planning on picking it up as a regular hobby. I already have too many hobbies as it is, and this isn’t the cheapest or most accessible hobby around. A brand new set of top-rope ice equipment (boots, crampons, tools) adds up to $800-$1000, while a used set is probably half of that plus more time hunting for a decent deal. And northern Illinois isn’t exactly teeming with waterfall ice to climb.</p>
<p>You can find a few more photos of the event <a href="http://goodell.smugmug.com/Travel/Ice-Climbing-2011/15507532_ZrRFp#1161465500_T8Z6X">on my Smugmug page</a>.</p>Dave GoodellLast weekend I got to try ice climbing out at the Ice Pit near Green Bay, Wisconsin. It was a very cool setup, sort of an outdoor ice climbing gym in an active quarry. Normally you must be a member to climb there, but for the festival it was just $20 for the whole weekend. I did the trip with five other guys, all rock climbers, three of which also work at the Lab.