Advanced Windows Debugging by Mario Hewardt and Daniel Pravat
I came across this book by chance in Cambridge library. Unfortunately, I didn’t get time to read it too thoroughly in the time I could have it on load, but the parts I did get to read were exceptionally good.
Over the years I’ve used used several low level debuggers for debugging windows at the machine code level (and have also done lots of debugging on Unix systems to using adb and gdb). I used cdb in my early career working on Lisp systems, and have used, more recently, windbg and the excellent SOS extension for debugging .NET. This book covered everything I’ve ever learned in this area, organising the material logically and explaining all sorts of related concepts. It then moved on to cover common scenarios and how they could be debugged, offering background details on the implementation of parts of Windows as a means of mapping out a strategy for the debugging.
In particular, the book had chapters on
(1) Basic debugger commands and how to set up symbols files to make it easier to debug code.
(2) Debugger internals covering how the operating system sends events to the debugger, how the debugger interacts with Windows’ structured exceptions and how the debugger inserts break instructions into the instruction stream to regain control at certain points. It went on to talk about thread suspension and resumption, and the related notions of freezing and thawing.
(3) The Windows calling conventions together with a detailed look at the many forms of stack corruption.
(4) Windows heaps and the built in allocation mechanisms together with a detailed look at debugging heap corruption.
(5) Security and how to look at the security context of a process or a thread.
(6) Troubleshooting communications failures including LPC
(7) Debugging resource leaks such as the failure to free handles and memory.
(8) Discussion of synchronisation and debugging deadlocks.
(9) Writing debugger extension commands and how to take various kinds of memory dump.
(10) Great chapters on the changes to support 64 bit operating systems and the changes made with the introduction of Vista.
A fantastic book covering both the technical means and the strategy for debugging many kinds of system failure.