The blog entry is a very good read and every Java/OO programmer should be familiar with these concepts. However, I disagree with most of his statements about convention-over-configuration and functional programming in particular. Overall, the blog reads a bit like "Why OO design pattern are better than functional programming and why those hippie trends like convention-over-configuration are overrated". It's always hard to cover everything in detail in a blog entry and often you simply have to set a focus (that's why it's called a blog and not an academic paper, right?) and here the focus clearly was set on object-orientation and design pattern. Within the OSGi enterprise expert group we had a short off-topic discussion about this blog entry and someone pointed out that it would be nice to see some counter examples.
Challenge accepted! I will use Clojure to convert the examples to a more functional style.
Convention-over-configuration
In the first part of the blog entry Nick presented an API usage where he always parametrizes the use of a ThreadPoolExecutor instead of using a more simplistic approach driven by convention-over-configuration. The used configuration "depends entirely on the nature of the problem you're solving and how callers of this code behave". Nick states that this boilerplate code "is really important when you work at massive scale" and that it provides "a way to configure the behavior of the system". He further describes that this style of API usage leads to modular software.
I think that this has nothing to do with the language or even the paradigm you choose since it's simply a matter of good API design. Provide meaningful defaults but do not tie the user to unchangeable behavior. However, I think that we can already improve his code and that his requirements even show some weaknesses of object-oriented programming! First, lets define the requirements:
1. Create a parametrized ThreadPoolExecutor
2. Execute a future task
3. The configuration depends on the problem and the caller
To keep the code simple, I will limit the configuration to PoolSize (just to demonstrate it, I won't change the size) and ThreadFactory. I will use the ThreadFactory to assign different names to the created threads. This provides us with an easy way to validate our configuration during task execution.
First, we create variables to define the default values for the ThreadPoolExecutor and ThreadFactory. We will later override the values when we want to parametrize:
(def *pool-size* 10)
(def *thread-name* "default name")
We need a ThreadFactory that uses the *thread-name* variable every time it creates new threads:
(def thread-factory (proxy [ThreadFactory] []
(newThread [runnable] (Thread. runnable *thread-name*))))
For testing purposes we will need several ThreadPoolExecutors so lets create a factory function that creates and returns a new executor. The created executor will use the *pool-size* and the thread-factory which in turn uses the *thread-name*:
(defn create-executor
[]
(ThreadPoolExecutor. *pool-size*
10
(long 500)
(TimeUnit/SECONDS)
(LinkedBlockingQueue.)
thread-factory))
We also need a function that will get called by the executor. We create another factory function that returns FutureTasks. Since every Clojure function implements the java.lang.Runnable interface we can easily pass a Clojure function to the FutureTask constructor. The factory function takes a task-id as parameter. We will use this ID to identify the created task later. When the created tasks get called by the executor, they simply print their ID and the name of the current thread:
(defn create-demo-task
[task-id]
(FutureTask. (fn []
(println "Task ID:" task-id
"Thread Name:" (.getName (Thread/currentThread))))))
Lets try it (note: The print to stdout will happen in a different thread so you might not see the output in your normal console):
(.execute (create-executor) (create-demo-task 1))
=> Task ID: 1 Thread Name: default name
Nothing special so far. The task gets executed and prints its ID and the name of the thread. So far we followed the convention-over-configuration way since we simply used our default value of "default name" for the thread. Our executor usage is very simple and we only implemented requirement 1 and 2. Nick argued that this simplicity is only an illusion and that you therefore always have to write the boilerplate code (for requirement 3). However, Clojure has a simply yet powerful mechanism to get a simple API and a flexible configuration at the same time: Bindings! Lets assume our problem demands a different thread name and that therefore the caller wants to influence the configuration:
(binding [*thread-name* "my new name"]
(.execute (create-executor) (create-demo-task 2)))
=> Task ID: 2 Thread Name: my new name
Bindings can be nested and each binding has its own scope. In the next example, the inner binding overrides the name with "my second name":
(binding [*thread-name* "my first name"]
(.execute (create-executor) (create-demo-task 3))
(.execute (create-executor) (create-demo-task 4))
(binding [*thread-name* "my second name"]
(.execute (create-executor) (create-demo-task 5)))
(.execute (create-executor) (create-demo-task 6)))
=> Task ID: 3 Thread Name: my first name
=> Task ID: 4 Thread Name: my first name
=> Task ID: 5 Thread Name: my second name
=> Task ID: 6 Thread Name: my first name
As you can see, the use of binding is completely optional and we only need to use it when we want to provide a different configuration. Requirement 3 is a very important one: "The configuration depends on the problem and the caller" and this is where object-oriented programming shows some weaknesses:
- The caller (who knows what configuration makes sense) is often not the one who creates the object (where we need to apply the configuration)! Sometimes the objects get created later in the call chain, sometimes the objects were already created long before the caller knew something about it.
- Once our tangled web of objects is instantiated, it's quite hard to change it. We either have a global dependency injection configuration or we pass factories around.
- Whenever we create a new object (to apply a new configuration) we in fact create a new tree of objects since the object creation might imply the creation of child objects. This means that we often have a redundant set of trees with only slight differences. These trees often need to share some other objects so we again pass factories and references around etc. The beloved OO-design pattern are often just workarounds for problems introduced by object-oriented programming!
;; Create application executor
(def app-executor (create-executor))
;; Capture "random name" behavior
(def with-random-name #(binding [*thread-name* (str (rand))]
(%)))
The application code now only needs to use the with-random-name function and the *thread-name* variable is now an implementation detail of our application-wide configuration:
(with-random-name
#(.execute app-executor (create-demo-task 7)))
=> Task ID: 7 Thread Name: 0.5414362847600062
Easy, isn't it? Obviously, you can create as many functions, compose them, pass them to other functions etc. This is a very powerful programming concepts and I fail to see how "function composition is a degenerate case of the Decorator pattern", as Nick called it.
This style of programming is called context-oriented programming and I hope that we will see more use of it in the future.
The design patterns of modularity
The blog continues to describe the use of dependency injection, decorators and factories and shows why those pattern are important for modular application. Again, I argue that those pattern can be easily implemented in functional languages and that the result is often more flexible and probably easier to use. The examples heavily depend on OO constructs (e.g. class hierarchies) and therefore do not translate 1:1 to a functional style. However, this shouldn't be a big problem since the purpose is to demonstrate the design pattern, not the API of a query framework. We assume that we have a query function and simulate a long running operation with Thread/sleep:
(defn query [q]
(Thread/sleep (rand 1000))
(println "Run query:" q))
In practice, the function would contact the database, return the query data structure etc. The first task is to add a TimingOutQuery that cancels the query after a certain amount of time. To keep our Clojure code simple we instead create a timing decorator that prints how long the operation took (a timing function is already implemented in the Clojure core library):
(defn timing-query [q]
(time (query q)))
This function simply delegates to the original query function. With this design we would need to create a wrapper function for each database function (query/select/cancel) which is clearly not an option. Fortunately it's quite easy to generalize this abstraction:
(defn timing-fn [f & args]
(time (apply f args)))
The function timing-fn takes a function and an arbitrary number of arguments as parameters, calls the function with the arguments and wraps everything in Clojure's time function. We simply created a generic timing decorator that we can use to wrap any function:
(timing-fn query "query string")
Nick stated that "function composition is a degenerate case of the Decorator pattern" but it's the object-oriented implementation which is limited because e.g. only Query objects can be decorated. The TimingOutQuery and Query are even tied together via class inheritance, one of the most strict coupling you can use! Making the OO version generic would involve another abstraction, maybe new interfaces, etc.
One of the advantages of decorators is that you can test the code in isolation, no matter if you use the OO or the functional way. As described above, the timing-fn is ever easier to test since it does not depend on a specific type that it will wrap. For example, we could easily wrap the println function:
(timing-fn println "Hello World!")
After we decomposed everything into small functions, we need factories to put everything together again. The idea is that user code should never know what type of query (normal query, timing query, ...) it is using. We therefore pass factories to the user code. These factories return the actual query function that the user code should use. First we need to create 2 factory function, one for normal queries and one for timing queries:
(defn query-factory []
(fn [q] (query q)))
(defn timing-fn-factory [factory]
(fn [q] (timing-fn (factory) q)))
It is important to note that while the query-factory has no parameters, the timing-fn-factory takes another factory as parameter. Our timing factory knows nothing about the query function and is completely generic. Next we need a user code function that takes a factory as parameter and invokes it to get and use the actual query function:
(defn load-users [factory]
((factory) "load users"))
Lets use it with the normal query function:
(load-users query-factory)
Using the timing-fn-factory is a bit more tricky, since a) we want that the user code calls the factory and b) the user code does not know that the factory takes another factory as parameter. Hence we need something like "here is your factory and I already fill in the parameters for you" feature. This pattern is called partial application in functional languages and is probably one of the most basic operation:
(load-users (partial timing-fn-factory query-factory))
We simply call partial with the function and parameters.
As I wrote in the beginning, I think that every Java/OO programmer should be familiar with the concepts Nick presented and they indeed are very important. However, I hope that I was able to show that the functional programming world is powerful as well and that there are more reasons than just convention-over-configuration why "hipster programmers who loves Clojure, Ruby, Scala, Erlang, or whatever" are adopting functional programming languages.
