And a final batch of books

Working from home for six months has made it really easy to get a lot of reading done. This is the final list of books that I’ve finished during the period.

Good Strategy/Bad Strategy: The difference and why it matters by Richard Rumelt

We went through this book as part of the reading group for Tech Leads at work. The book looks at what a strategy is and contrasts it to the usual set of motivational ideas that we are often told is a strategy. The book emphasises a logical plan based on an analysis of the problem together with reasoning as to why a particular item was targeted. A mix of common sense and good tricks to get a good plan together.

An Elegant Puzzle: Systems of Engineering Management by Will Larson

This again was going to be the object of a reading group; however, the switch to working from home meant that this never happened, but I read my way through the book anyway. I must admit that I found the book hard going.

How Linux works by Brian Ward

I’m going to be moving back to using Linux day to day, and wanted a refresher on the lower level details of Linux. This book is brilliant for that purpose. As the blurb says, it covers all of the basics, though it does this in lots of interesting detail.

Webpack 5 Up and Running: A quick and practical introduction to the JavaScript application bundler by Tom Owens

I wanted to get up to speed by the new version of webpack. To be honest this book just feels like cut and pasted parts of the existing documentation, and it really feels like the book could do with some good proof reading to correct the typos and misspellings.

The Daemon, the Gnu, and the Penguin by Peter H Salus

A potted history of Unix and Linux. A quick read by interesting from a historical point of view.

Einstein’s Unfinished Revolution: The Search for What Lies Beyond the Quantum by Lee Smolin

Lee Smolin has written a loads of books over the years about the search for a unified theory of physics. This is another one that talks about his more recent ideas around quantum mechanics and how we can give it a realist interpretation. I must admit that I have enjoyed all of his books.

Programming Rust: Fast, Safe Systems Development by Jim Blandy

There are several people at work who are massive fans of Rust, and this book does a really good job both of explaining the language and discussing the benefits of using Rust to get runtime safety. The book is really good and I will certainly be using the language in the future.

Haskell’s Type System is really amazing

I guess it’s no surprise, but the Haskell Type system is really amazing. The support for GADTs and other interesting type level ideas like phantom types make the language a fascinating place to understand how much types can help with the construction of only valid programs.

I’ve recently been reading Sandy Maguire’s Thinking With Types which is, as the author claims, the comprehensive manual for type level programming.

The book starts from first principles, with Sum/Product and Exponential types and a discussion of the difference between terms, types and kinds. It then goes into detail about all of the type related extensions that GHC supports including GADTs and constraints, Rank-N and existential types. This is followed by a section on computing at the type level – understanding Haskell’s dependent type support was one of the main reasons that I bought this book. This also covers type families and defunctionalization.

The book is great but it is going to take a couple of reads to understand the material.

I also read Haskell Design Patterns: Take your Haskell and functional programming skills to the next level by exploring new idioms and design patterns by Ryan Lemmer

This book is also a quick read and about half of it is also connected to using the type system to make your programs more correct. However it also has some great sections on Haskell ideas like traversals, Monads and monad transformers, as well as a discussion of different styles of functional I/O.

At the recent Haskell Love online conference the talks on Haskell to Core helped me to understand some of these ideas as the presenter covers how some of the type equality constraints are presented in the de-sugered intermediate language that GHC users, and Richard Eisenberg’s Parameters of Many Flavours also discusses the different types of implicit parameters that are carried by a Haskell function definition (and also discusses what is erased and what is available at runtime).

OAuth 2 in Action

I’ve never really understood where authentication fits into the OAuth protocol – I kept finding articles that mentioned OpenID Connect was an extension of OAuth that did this, but was still confused as there is often an authentication step when you use OAuth to delegate you authorization decision. How is this not authentication?

I recently found a book that explained this really well – OAuth 2 In Action by Justin Richer and Antonio Sanso.

To me this was the perfect developer book. The authors explain what OAuth is, and go through each of the flows that are supported – OAuth works across many types of device and configuration, from simple delegation of authority between two web sites to handling a mobile device or desktop application talking to a REST backend. Most importantly they have examples in Javascript of each of these scenarios and take you through the Javascript code you need to implement to support the protocol. After covering OAuth the authors talk through the extensions which are built on top of OAuth. One of these extensions is OpenID Connect. Again, we get to implement this, which is definitely the way to get a good understanding of anything.

The book has an associated Github repository for the code examples here.

I did a lightning talk on this at work recently and there’s a good talk on YouTube that explains all of this.

System Design Interviews

I’ve become really interested in the design of big systems, in particular how you can scale using commodity machines to planet size applications. There are lots of sites around that offer material for this purpose (and many YouTube channels too). In particular in the past I’d done a fair amount of reading on the system design primer site. I recently noticed a book on Amazon that looked good for this subject.

System Design Interview: An Insider’s Guide by Alex Xu

This book offers some chapters that describe good ways of tackling system design interview questions, including an introductory chapter that takes a web application for a single machine with a small number of users and describes the techniques for scaling that up. After that the book contains 12 example system designs, ranging from consistent hashing and rate scaling, to designing Google Drive. The chapters are short and to the point, and there are numerous typos and sentences that don’t quite scan properly, but all in all the book is a really good read which covers the key points for such interviews. Also each of the chapters has a set of 10+ links to additional material such as papers and engineering blogs of the various companies that implemented the problem. This makes it a quick but informative read, and a good way to review the material around this area.

Some Big Data reading

So it’s off to join another company soon, and I’ve needed to get more familiar with Big Data ideas. What have I read as a means of doing that?

First a re-read of Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems

This is a great book that gives an overview of all kinds of distributed systems theory, including issues such as consensus, coverage of the techniques for storing and partitioning massive amounts of data, and an overview of the techniques of processing the data via map/reduce and streaming. Every time I read the book I find some idea that I hadn’t fully understood before.

Then, two free books that you can get from the Confluent web site.

Kafka: The Definitive Guide which is (unsurprisingly) a guide to Kafka. It covers how you install and use Kafka and has chapters on the internals of the system together with how you get the reliability guarantees that you need. The are also chapters on monitoring and administering a Kafka cluster, thought the chapter I found most interesting was the final chapter on streaming with Kafka, which gives a good overview on how you join streams and tables.

Designing Event-Driven Systems: Concepts and Patterns for Streaming Services with Apache Kafka. The book has sections on event driven systems, explaining what they are and why you should use them, particularly as a way of bringing together disparate data from across a company. It then talks about consistency of the data, concurrent access and how the systems can evolve over time. The book is quite short, but is full of interesting ideas.

The next good source of information was the Data Engineering Podcast. There are several hundred episodes available, including interesting episodes on Presto, Postgres, Kafka, time series databases, Zookeeper and many more. Every episode I have listened to so far has been informative and less than an hour so easy to listen to.

The are also a number of papers that have been interesting to read – Azure storage, Spanner, Chubby, Raft, LSM to name a few. Also, here are a couple of talks about adding transactions to Kafka and lessons learned in the production use of Kafka.

It’s been a while

After returning to my previous employer, there’s a been a lot to do. Though I’ve been reading books at the usual rate, I just haven’t had time to type up notes on them here. However, it’s time to catch up, so here’s a list of some good books and papers from recent times.

Linux Observability with BPF by David Calavera and Lorenzo Fontana

I did a lightning talk at work on eBPF. I’d been hearing about eBPF for a while in various blog posts, and the idea of being able to attach VM code at points in the kernel is fascinating, particularly if that code can change the behaviour. I read this book, which turned out to be a great introduction to the subject. It goes though the various parts of the eBPF story – the VM, the places you can attach the code, the BPF tools packages that sit on top of everything, and then covers the networking side of things where eBPF really started. It’s  a fairly thin book, but is dense – there’s a link to a GitHub repository full of lots of good examples, and also instructions on how to set up a VM to run them.

eBPF is an amazing technology and is well worth reading about. There’s a quick primer on writing a simple eBPF function here.

Type checking papers related to Haskell

We have been thinking about type checking in our weekly functional programming lunch at work. As part of that I’ve been reading a few papers related to type checking GADTs and type families in Haskell. This paper is a good introduction to type checking GADTS before you take on the full Outside In paper.

A means of implementing GADTs in OCaml is described here.

Microservices Patterns by Chris Richardson

There’s so much talk about Microservices, and we’ve been trying to get to grips with it at a weekly video lunch we have been having. This is a brilliant book, which pulled together all of the ideas that I’ve seen in a number of talks. It covers everything – the reasons that you’d take a monolinth and break it into Microservices (and when you’d keep a monolith), how you’d organise the services, how you get transactions between the independent services (sagas and compensation, with various mail box patterns to ensure delivery of messages), querying via an API gateway and read models, and testing strategies and contract tests.

The book is full of examples which makes the concepts a lot easier to follow.

Programming Typescript by Boris Cherny

This is a really good book on TypeScript (though it also taught me some JavaScript too). It’s very clear and well written and moves through the set of things that TypeScript offers, covering some of the advanced parts of the type system.

We’ve been using TypeScript at work and this book helped me understand some of the more advanced parts.

The type system of TypeScript can be used to do some complicated things at compile time. This repository shows some good examples.

More miscellaneous books

Another change of job, and hence another list of books I’ve read in the meantime.

Designing Distributed Systems by Brendan Burns

This book goes through a number of patterns that you might want to use when designing distributed systems using Kubernetes. It starts by looking at the single node patterns of Sidecars, Ambassadors and Adapters, and then moves on to serving patterns, where the author talks about replicated load-balancing, sharded services, scatter/gather, FAAS and ownership election. There’s then a third section on batch patterns, using generic work queues, event driven processing and coordinated batch processing.

The book is fairly thin, but each pattern is described with a motivating example and at least one example of an implementation using the pattern.

The Manager’s Path by Camille Fournier

The book looks at the various management roles in the high tech industry, and gives advice for getting the best out of each of the management roles. It starts with team level mentoring, looks at the Tech Lead role and then moves to more management-only focussed roles like engineering manager, CTO and VP of engineering. The book is filled with useful advice from the author who has been in each of the roles.

Talk to me by James Vlahos

This book talks about the history of voice controlled AI used for the current virtual assistants. To be honest, I didn’t find the book technical enough to be really interesting, though it was good to see where the various ideas had come from.

How Google works by Eric Schmidt and Jonathan Rosenberg

The book is now fairly old, but gives some interesting details on how Google is managed. In particular, for me, I found the definition of a “Smart Creative” interesting, and the book then discusses how traditional management structures do not really work for such people. The book has chapters on Culture, Strategy, Hiring, Decisions, Communication and Innovation, and all of the chapters discuss experiences at Google in the given area. The book is definitely worth a read.

Homo Deus – A brief history of tomorrow by Yuval Noah Harari

This is an interesting and thought provoking read. It looks at what may follow Humanism in the near-future, and where data driven decision making will take the human race.

 

 

 

More things for using K8s for real

In my current job, the team are working on deploying an application onto Azure using Kubernetes, so it was time to do some reading around the technologies that are being used.

Docker is used by the company as the universal packaging mechanism. The engine part of the application is written in C++, with a combination of Go and Python being used to wrap this to produce the final product. Multi-stage docker builds are used to build the application itself, as well as docker containers for running the various tests. The cloud version of the system is also deployed using Kubernetes  using more containers.

Docker Up & Running: Shipping Reliable Containers In Production by Sean Kane and Karl Matthias

This book was a great introduction to docker, with the first chapters providing a good introduction as to why containers are a good idea. It covers the common use cases, and goes through the many command line calls you will need to use to understand what is going on. The next chapter is also very good, with a good explanation of what is happening at the Linux level. There is also a chapter on docker-compose, which is interesting though not particularly relevant if you are deploying to Kubernetes.

Terraform Up & Running by  Yevgeniy Brikman

This book was also a good read. Terraform is used by the system at work to do the initial provisioning of the AKS cluster and associated networking and external resources. This short book gives as good overview of Terraform, and has a good running example that shows you how it is used for real.

The Master Algorithm: How the Quest for the ultimate learning machine will remake our world by Pedro Domingos

This was an easy read, which starts with a good overview to how machine learning has come of age. The book then describes the 5 tribes of machine learning – the symbolists, the connectionists, the evolutionaries, the Bayesians, and the analogisers, and gives a brief overview of how their techniques work. The rest of the book discusses how these techniques will need to be merged to get to a master algorithm, and the author covers some of the work he has done to do this on the Alchemy system.

And some more books

The Go Programming language by Alan Donovan and Brian Kernighan

I read this as preparation for a new job. It feels like the definitive guide to the Go language. The book is a great learning tool, highlighting practical uses of the language as it covers the syntax, though the language itself will definitely take some getting used to.

Clean Architecture by Robert C Martin

I’m not 100% sure of how I felt about this book. It mentioned lots of ideas that I had seen before – for example the SOLID principles, and linked them into ideas around components.  There is also a 45 year journey through the author’s career which was an interesting read, but I don’t think it helped much with the concepts.

I’ve also been watching a lot of videos on Linux and containers. This talk on the difference between containers and hypervisors is really good. The talk mentions technologies such as gvisor which is also covered in this blog post.

Some odds and ends

My time at Redgate is coming to an end, so I’ve been reading a fair few books on some of the new technologies that I’ll be using in my new job. More on that later.

First some interesting videos and posts from the last month or so.

I’ve been getting to grips with Comonads as part of my push to understand Haskell in more depth. This talk on implementing the Game of Life using Comonads was interesting,  and it also led me to watching this slightly more in depth talk. This talk on UIs being Comonads, part of a series on Purescript was how I started this path.

This article on KVM made sense, and gives enough of an overview to help you understand where KVM sits.

I’ve been using React a little, and have found it a great technology. Of course, I’d like to understand more about what is happening under the covers, and this series of blog posts  helped me understand the reconciliation process between the React tree and rendered browser DOM, by way of the new Fiber engine. This YouTube video is also a good overview.

MySQL High Availability

I found this book in the local library and thought I’d give it a read. I read the older version, which was written just after MySQL had extended the statement level replication to support row based replication. I enjoyed the level of detail, and the hands-on nature which shows you the commands you’ll need to run to start the replication process. The idea of using replicas so that writes go to the master and reads go to one or more slaves is an idea that seems to come up all the time in system design examples. This book gave me a good feel for how this all works. After I read this book, I found this blog post which gives a grief summary of the ideas.

Fluent Python

I have done bits and pieces in Python in the past, but wanted to get a better grip of the more advanced features of the language. I was lucky enough to find this book in a local bookshop.

The book has several sections:

• data structures, which talks about the basic sequence types, as well as dictionaries, sets and text
• functions as objects, which talks about functions as first class entities, and includes decorators and closures
• object oriented idioms which discusses object references, mutability and recycling, the meaning of a Pythonic object, interfaces and inheritance.
• control flow, which covers iterables, iterators and generators, contexts, coroutines and concurrency. This is a part of Python about which I knew nothing, and the book covered it really well.
• metaprogramming, which talks about adding dynamic attributes and properties to classes as well as class level metaprogramming

I really enjoyed the book. It discussed everything at the right level, with enough details to really understand what is happening under the covers. In fact, the author shows the byte code at times to help you understand what is going on.

Managing Kubernetes

This book talks about the operational side of managed a Kubernetes installation. As such you get to see a little more about what happens in the standard components, and it also covers monitoring, disaster recovery and authentication which are aspects you really need to understand if your business depends on it.

Pro .NET Memory Management

It’s impossible to describe how good this book is. It covers the implementation of the .NET garbage collector in great detail, often linking back to the sources. It works through various case studies to show you issues that you may need to work around, and it spends a lot of time showing you how to use ETW and perfview to diagnose issues in your application that are caused by garbage collection. It’s a massive book that takes quite a while to read, and I’m going to need to re-read it loads of times to really understand it all.

The level of detail in the book is truly awe-inspiring.

Quantum Space

I’ve always been interested int he unified theories of physics and wanted to understand the progress in Loop Quantum Gravity. This book covers some of the history of the theory and is a good read.