It’s been a while

After returning to my previous employer, there’s a been a lot to do. Though I’ve been reading books at the usual rate, I just haven’t had time to type up notes on them here. However, it’s time to catch up, so here’s a list of some good books and papers from recent times.

Linux Observability with BPF by David Calavera and Lorenzo Fontana

I did a lightning talk at work on eBPF. I’d been hearing about eBPF for a while in various blog posts, and the idea of being able to attach VM code at points in the kernel is fascinating, particularly if that code can change the behaviour. I read this book, which turned out to be a great introduction to the subject. It goes though the various parts of the eBPF story – the VM, the places you can attach the code, the BPF tools packages that sit on top of everything, and then covers the networking side of things where eBPF really started. It’s  a fairly thin book, but is dense – there’s a link to a GitHub repository full of lots of good examples, and also instructions on how to set up a VM to run them.

eBPF is an amazing technology and is well worth reading about. There’s a quick primer on writing a simple eBPF function here.

Type checking papers related to Haskell

We have been thinking about type checking in our weekly functional programming lunch at work. As part of that I’ve been reading a few papers related to type checking GADTs and type families in Haskell. This paper is a good introduction to type checking GADTS before you take on the full Outside In paper.

A means of implementing GADTs in OCaml is described here.

Microservices Patterns by Chris Richardson

There’s so much talk about Microservices, and we’ve been trying to get to grips with it at a weekly video lunch we have been having. This is a brilliant book, which pulled together all of the ideas that I’ve seen in a number of talks. It covers everything – the reasons that you’d take a monolinth and break it into Microservices (and when you’d keep a monolith), how you’d organise the services, how you get transactions between the independent services (sagas and compensation, with various mail box patterns to ensure delivery of messages), querying via an API gateway and read models, and testing strategies and contract tests.

The book is full of examples which makes the concepts a lot easier to follow.

Programming Typescript by Boris Cherny

This is a really good book on TypeScript (though it also taught me some JavaScript too). It’s very clear and well written and moves through the set of things that TypeScript offers, covering some of the advanced parts of the type system.

We’ve been using TypeScript at work and this book helped me understand some of the more advanced parts.

The type system of TypeScript can be used to do some complicated things at compile time. This repository shows some good examples.

More miscellaneous books

Another change of job, and hence another list of books I’ve read in the meantime.

Designing Distributed Systems by Brendan Burns

This book goes through a number of patterns that you might want to use when designing distributed systems using Kubernetes. It starts by looking at the single node patterns of Sidecars, Ambassadors and Adapters, and then moves on to serving patterns, where the author talks about replicated load-balancing, sharded services, scatter/gather, FAAS and ownership election. There’s then a third section on batch patterns, using generic work queues, event driven processing and coordinated batch processing.

The book is fairly thin, but each pattern is described with a motivating example and at least one example of an implementation using the pattern.

The Manager’s Path by Camille Fournier

The book looks at the various management roles in the high tech industry, and gives advice for getting the best out of each of the management roles. It starts with team level mentoring, looks at the Tech Lead role and then moves to more management-only focussed roles like engineering manager, CTO and VP of engineering. The book is filled with useful advice from the author who has been in each of the roles.

Talk to me by James Vlahos

This book talks about the history of voice controlled AI used for the current virtual assistants. To be honest, I didn’t find the book technical enough to be really interesting, though it was good to see where the various ideas had come from.

How Google works by Eric Schmidt and Jonathan Rosenberg

The book is now fairly old, but gives some interesting details on how Google is managed. In particular, for me, I found the definition of a “Smart Creative” interesting, and the book then discusses how traditional management structures do not really work for such people. The book has chapters on Culture, Strategy, Hiring, Decisions, Communication and Innovation, and all of the chapters discuss experiences at Google in the given area. The book is definitely worth a read.

Homo Deus – A brief history of tomorrow by Yuval Noah Harari

This is an interesting and thought provoking read. It looks at what may follow Humanism in the near-future, and where data driven decision making will take the human race.

 

 

 

More things for using K8s for real

In my current job, the team are working on deploying an application onto Azure using Kubernetes, so it was time to do some reading around the technologies that are being used.

Docker is used by the company as the universal packaging mechanism. The engine part of the application is written in C++, with a combination of Go and Python being used to wrap this to produce the final product. Multi-stage docker builds are used to build the application itself, as well as docker containers for running the various tests. The cloud version of the system is also deployed using Kubernetes  using more containers.

Docker Up & Running: Shipping Reliable Containers In Production by Sean Kane and Karl Matthias

This book was a great introduction to docker, with the first chapters providing a good introduction as to why containers are a good idea. It covers the common use cases, and goes through the many command line calls you will need to use to understand what is going on. The next chapter is also very good, with a good explanation of what is happening at the Linux level. There is also a chapter on docker-compose, which is interesting though not particularly relevant if you are deploying to Kubernetes.

Terraform Up & Running by  Yevgeniy Brikman

This book was also a good read. Terraform is used by the system at work to do the initial provisioning of the AKS cluster and associated networking and external resources. This short book gives as good overview of Terraform, and has a good running example that shows you how it is used for real.

The Master Algorithm: How the Quest for the ultimate learning machine will remake our world by Pedro Domingos

This was an easy read, which starts with a good overview to how machine learning has come of age. The book then describes the 5 tribes of machine learning – the symbolists, the connectionists, the evolutionaries, the Bayesians, and the analogisers, and gives a brief overview of how their techniques work. The rest of the book discusses how these techniques will need to be merged to get to a master algorithm, and the author covers some of the work he has done to do this on the Alchemy system.

And some more books

The Go Programming language by Alan Donovan and Brian Kernighan

I read this as preparation for a new job. It feels like the definitive guide to the Go language. The book is a great learning tool, highlighting practical uses of the language as it covers the syntax, though the language itself will definitely take some getting used to.

Clean Architecture by Robert C Martin

I’m not 100% sure of how I felt about this book. It mentioned lots of ideas that I had seen before – for example the SOLID principles, and linked them into ideas around components.  There is also a 45 year journey through the author’s career which was an interesting read, but I don’t think it helped much with the concepts.

I’ve also been watching a lot of videos on Linux and containers. This talk on the difference between containers and hypervisors is really good. The talk mentions technologies such as gvisor which is also covered in this blog post.

Some odds and ends

My time at Redgate is coming to an end, so I’ve been reading a fair few books on some of the new technologies that I’ll be using in my new job. More on that later.

First some interesting videos and posts from the last month or so.

I’ve been getting to grips with Comonads as part of my push to understand Haskell in more depth. This talk on implementing the Game of Life using Comonads was interesting,  and it also led me to watching this slightly more in depth talk. This talk on UIs being Comonads, part of a series on Purescript was how I started this path.

This article on KVM made sense, and gives enough of an overview to help you understand where KVM sits.

I’ve been using React a little, and have found it a great technology. Of course, I’d like to understand more about what is happening under the covers, and this series of blog posts  helped me understand the reconciliation process between the React tree and rendered browser DOM, by way of the new Fiber engine. This YouTube video is also a good overview.

MySQL High Availability

I found this book in the local library and thought I’d give it a read. I read the older version, which was written just after MySQL had extended the statement level replication to support row based replication. I enjoyed the level of detail, and the hands-on nature which shows you the commands you’ll need to run to start the replication process. The idea of using replicas so that writes go to the master and reads go to one or more slaves is an idea that seems to come up all the time in system design examples. This book gave me a good feel for how this all works. After I read this book, I found this blog post which gives a grief summary of the ideas.

Fluent Python

I have done bits and pieces in Python in the past, but wanted to get a better grip of the more advanced features of the language. I was lucky enough to find this book in a local bookshop.

The book has several sections:

• data structures, which talks about the basic sequence types, as well as dictionaries, sets and text
• functions as objects, which talks about functions as first class entities, and includes decorators and closures
• object oriented idioms which discusses object references, mutability and recycling, the meaning of a Pythonic object, interfaces and inheritance.
• control flow, which covers iterables, iterators and generators, contexts, coroutines and concurrency. This is a part of Python about which I knew nothing, and the book covered it really well.
• metaprogramming, which talks about adding dynamic attributes and properties to classes as well as class level metaprogramming

I really enjoyed the book. It discussed everything at the right level, with enough details to really understand what is happening under the covers. In fact, the author shows the byte code at times to help you understand what is going on.

Managing Kubernetes

This book talks about the operational side of managed a Kubernetes installation. As such you get to see a little more about what happens in the standard components, and it also covers monitoring, disaster recovery and authentication which are aspects you really need to understand if your business depends on it.

Pro .NET Memory Management

It’s impossible to describe how good this book is. It covers the implementation of the .NET garbage collector in great detail, often linking back to the sources. It works through various case studies to show you issues that you may need to work around, and it spends a lot of time showing you how to use ETW and perfview to diagnose issues in your application that are caused by garbage collection. It’s a massive book that takes quite a while to read, and I’m going to need to re-read it loads of times to really understand it all.

The level of detail in the book is truly awe-inspiring.

Quantum Space

I’ve always been interested int he unified theories of physics and wanted to understand the progress in Loop Quantum Gravity. This book covers some of the history of the theory and is a good read.

I’m hooked

It always amazes me how simple ideas can transfer from one domain to another.

A while ago I was looking at Azure Durable Functions. In this domain you want to write a function that can call out to another service, but for scalability reasons we’d like to avoid waiting for the result. Instead we’d like to stop the current execution and then reinstate the context when the value is available. Of course the difficulty is that this sounds like special compiler logic (like the type of transform used to implement generators in many languages). Instead, in the case of Azure Durable Functions, the designers demand that the functions are deterministic, and the trick is to store the previous results in an event log. We can then get to the same execution point by replaying the function – instead of making real outgoing calls, the system returns the previously logged return value at each point where the code tries to call out of the context, knowing that this will lead us down the same path that we executed the first time. This lets us wind forward to the same place and then execute the next step.

React has always had two ways of defining a component. You can use a class and override the various lifecycle methods (like componentDidMount) or you can write more functional (Pure) components that just return the mark up that they want to render. These functional components can be a lot easier to read, and it can be easier to share logic as it is easy for functions to call other functions. Adding state into such functional components is harder though, and the React team have introduced a concept called Hooks to do this. This again requires the idea of a function where outgoing calls are made in a  deterministic order, and this lets the system use this order to return different values.

There’s a good post that explains Hooks here, but the quickest introduction is a talk at React Conf with an informative excerpt here. In the example that they show in the talks, the functional approach makes the code much easier to understand than the class based approach – the latter can lead to related logic being spread across the class’s methods whereas the functional approach puts the code together.

While we are on the subject of React, there’s a post here talking about Flutter as an alternative to React Native.

Of CORSe, it’s easy to test

I read this informative post on CORS at the weekend, and realised that the best way to get to grips with it, is to try some experiments. I hadn’t realised before how easy this would be in C#. It’s easy to write a mini-web server that handles a single call using code like this.

            IPAddress ipAddress = IPAddress.Parse("127.0.0.1");
            TcpListener listener = new TcpListener(ipAddress, 8182);
            listener.Start();

            using (var clt = listener.AcceptTcpClient())
            using (NetworkStream ns = clt.GetStream())
            using (StreamReader sr = new StreamReader(ns))
            using (StreamWriter sw = new StreamWriter(ns))
            {
                var msg = sr.ReadLine();
                Console.WriteLine(msg);

                if (msg.StartsWith("GET"))
                {
                    string line;
                    while ((line = sr.ReadLine()) != "")
                        Console.WriteLine(line);

                    Console.WriteLine();
                    Console.WriteLine();

                    sw.WriteLine("HTTP/1.1 200 OK");
                    //sw.WriteLine("Access-Control-Allow-Origin: chrome://newtab");
                    sw.WriteLine();
                    sw.WriteLine("{a:2020}");
                }
            }

With that code running, you can start Chrome in Incognito mode, and then run the following code in the console.

var h = new XMLHttpRequest()
h.open("GET", "http://localhost:8182", true)
h.send()

which gives the error

(index):1 Access to XMLHttpRequest at 'http://localhost:8182/' from origin 'chrome://newtab' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

Uncommenting the Access Control line in the above code allows the request to succeed.