Blog Posts
Reasoning about design
Most engineering disciplines have some way to model the system that is being built and to reason about the design of the system before it is constructed. That does not mean that no testing is performed, for models are but approximations of the real thing and could be wrong! For example, see this video of a boeing wing test and as the test is performed and the wings fail, you can see the engineers smiling and cheering. Why? because it proved that their model. In fact having sufficient testing can have disastrous effects, as was shown in the construction and collapse of the Tacoma Narrows Bridge.
Software construction on the other hand, follows more closely this model from Calvin and Hobbes. The only validation most of us do is using testing. The problem with testing, is that in the presence of multi-threading and concurrency, it might not be possible to simulate all the possible executions paths, leading to Heisenbug’s that might be difficult to track down or even not be known till they occur in production.
As we all know, and as was reported in this paper , the relative cost of fixing a bug raises nearly exponentially the later it is found in the lifecycle, and being found in production is the worst.
So the question arises, is there a way to test the “design” of software systems, before we write code, so that we can be sure that the design itself does not have flaws? It’s important to note that, design bugs, like protocol flaws between systems that can lead to data corruption, would by definition, be close to impossible to fix using code, without changing the design itself.
When all you have is a hammer…
As Alan Kay says, “A change in perspective is worth 80 IQ points”. One interpretation of that is the way we look at the world colors the kinds of solutions and approaches we take to solve the perceived problems. Take the case of the three most popular frameworks for building rich client side applications (sometime also knows as Single Page Applications or SPA), namely, ReactJS, Angular and EmberJS.
All three are mature frameworks, being used with fairly complex websites running successfully in production. So when the question normally comes up, “which one should I use?”, it looks like any or all of them would work, right?
But then is there another way to think about this? Maybe the question is not which one to use, but rather, what is one trying to do…
Opinionated view of a 3rd generation serving framework
Or how to get 1 second render times on complex web pages …
In the previous post, we saw how we could classify internet serving architecture into different generations and what the implications of each generation are. We also observed there, how, as we enter 3rd generation and later, the need for concurrency becomes central to having a performant serving system.
In the world of complex web pages, like YouTube, etc., the question then arises on how to organize the code to balance ease of coding while still retaining the performance benefits. In this post we look at one such idea, which has proven to work well in practice.
Problem Definition
What do we mean by 1 second render time? It means that once the user types in the URL, the web page is visible for the user to interact with within or at 1 second. There is a lot of content in the web today, like this blog post or this book or this Google developer documentation that talk about browser page rendering and how to look at it from a front-end perspective.
But I’ve rarely seen folks talk about what it means from a backend web-server perspective on how to generate and organize the content in the first place.
Let’s start by quantifying the problem.
Evolution of Internet serving architectures
In the previous two blog posts, we looked at a simple example for asynchronously and concurrently fetching resources from the internet, first in Go , and then in Java. The solutions were useful as a means to to explore the different ways of approaching and reasoning about concurrency, based on each languages strengths. But if we were to take a step back and look at how web serving architectures and web page rendering having evolved over the past decades, it turns out that the problem itself is far more interesting…
This post explores why that might be so.
Architecture Generations
If we were to take a step back and look at how the web have evolved over the past decades, not from the perspective of the kinds of content, like the use of images, videos etc., but from the complexity and richness of the content mix, we can potentially classify web serving architectures into different generations.
Concurrency in practice - Java
Java
unlike Go, focuses on the values being produced as the main fulcrum for organizing and scheduling
concurrent and asynchronous tasks. Java 8 introduces the concept of a
CompletableFuture
as a way to compose these values together using a
ForkJoinPool
for running the actual computations. Further the introduction of
lambda expressions has made it
easy to represent the code for these transformations succinctly. This makes it easy to see the “flow” of
the computations / transformations independent of the actual worked performed at each step.
In this post we will look at a small I/O intesive problem and one possible way we could use these capabilities and to reason about and chain these asynchronous concurrent operations using Java 8.The complete code is available on GitHub at satyasm/JavaPageSize and can be built using maven as documented in the README.md on the project.
Problem Definition
Say, we have a list of webpage URLs, and we want to find out what the size of the webpage is.
We can define the size of the webpage as the size of the original HTML for the page, along with
the size of all the assets it references, namely, css, image and js files in link, img and
script tags respectively. So the idea is that given a webpage URL, fetch the page, parse the
HTML to find the unique references as above, download those resources and finally sum the total size of
the bytes received to get the total page weight.
Concurrency in practice - Go programming language
Continuing our exploration of the various concurrency models, the Go programming language is one of the more recent ones that explicitly supports concurrency as a first class concept. From the implementation of go routines as cheap ways to launch and run concurrent code, to channels to communicate between them, it makes reasoning about and writing concurrent code not only easy, but fun!
In this post, we will look at a small I/O intensive problem and explore different ways of doing this operation concurrently using the Go programming language, as a practical way of using concurrency in practice. The final code is hosted on Github at satyasm/gopagesize, and if you have a working Go environment, you can get it by running
go get github.com/satyasm/gopagesize
which should give you the gopagesize command to play with.
Problem Definition
Say, we have a list of webpage URLs, and we want to find out what the size of the webpage is.
We can define the size of the webpage as the size of the original HTML for the page, along with
the size of all the assets it references, namely, css, image and js files in link, img and
script tags respectively. So the idea is that given a webpage URL, fetch the page, parse the
HTML to find the references as above, download those resources and finally sum the total size of
the bytes received to get the total page weight.
Three concurrency models in mainstream languages
As Fred Brooks famously said, there are is no Silver Bullet to developing software. Most of the software development over the past decade, has focused on reducing the accidental complexity of software development, starting from Dijkstra’s Notes on Structured Programming to Object Oriented Programming, a term coined by Alan Kay, to the current resurgence of interest in Functional Programming.
In today’s world, Moore’s law has meant that computers have started going multicore. This has implied that in order to scale, and make better use of the hardware, programs have had to be beyond sequential models of execution. As we have learnt from practice, doing shared memory parallelism, with locks is very very hard.
So, the question arises, is there something we can do to improve the ability to compose these kinds of non-sequential programs?
A Monadic view of asynchronous programming using Guava
Let’s first start with a definition of what we mean by asynchronous programming.
In normal (sequential) programming, computation starts in one execution context and then continues by executing one instruction after another, within the same context. In this style, it is easy to follow the “flow” of the program as one can just step through the instructions, including function calls and returns to reason about how the program executes.
By asynchronous programming, we are referring to that style of programming, where the computation is spread across different execution contexts. This is useful if the cost of the computation is greater than the cost of switching across these different contexts.
This cost of switching is never free, as at the CPU level, it would involve suspending the current context, saving the program data, which includes things like register variables, program counters, stack references and then swapping in the new data for the next context. In case the data for the new context is not in the cache, there could be futher costs due to cache miss and data loads from memory.
In practice, this is most useful in the case of I/O related computation, since the cost of waiting for I/O is so large (compared to cache and memory access), that switching I/O operations to a different context and then resuming it when it completes, frees up the original context to perform other operations and thus lead to better utilization of the compute resources. For the end-user, it means that the same computer can now do “more work”.
While the benefit is clear, namely, more bang for the buck, a serious problem one quickly runs into with this style of programming, is that it becomes very hard to code and follow the “flow” of execution. Attempts to do this, typically lead to statements like “callback hell” etc., which reflects the frustration of trying to make this work.
So the question arises: Is there a way to represent this style of computation, such that we can reason about and see the “flow” of code across these execution contexts, independent of the actual steps being performed during each computation?