Round 1 - Clojure vs Blub Lang

Round 1 - Clojure vs Blub Lang - Parallelism

if you haven't already heard - I am very passionate about programming/development, and even moreso about all things Lisp, with my last-years obsession being Clojure.

Clojure is a wonderful lisp, combining elements of functional programming, immutability, and parallelism into the core of the language as "freebies".

This is a small thought experiment (looking for user feedback / suggestions / samples in your Blub Lang of choice in the comments or on the various social media posts).

It has to deal with parallelism and is a pretty simple/brief exercise. I have heard praise of certain languages providing wonderful parallelism abstractions (Go, Javascript, etc.) and would be interested in seeing if their solutions are more terse and robust than the one I'm going to start with (Clojure).

Your task, should you choose to accept it

In this hypothetical, you are tasked with the following feature for your system:

Business has a need to query 100 API endpoints and aggregate/store the resultant data in the same order it was fetched.
The requests must be robust (ie, each one must create a unique log entry in an in-memory array/vector for journaling).
The feature for this 'get-data' fetcher must be as performant as possible.

The technical requirements for this exercise are as follows:

The API request should be a mock/fake request, consisting of a function that receives a number (N), sleeps for 1 second, and returns the N after the sleep is done.
The journaling must occur within the fetch function to be closer to real logging with API requests in a system.
The dataset to map over / apply this 'fetch-data' function to should be the numbers 0 to 99 inclusive.

Round Results

Ding ding ding - the bell has rang, and the code has started.

Contender number one - my whipped-together Clojure implementation:

Clojure by Matthew Carter <m@ahungry.com>

This uses a few Clojure built-ins (atoms for atomicity, pmap for parallel map, that is using clojure futures - similar to threads, behind the scenes):

(def my-data-log (atom []))

(defn fetch-data! [n]
  (Thread/sleep 1e3)
  (swap! my-data-log conj (str "Fetched record: " n)) n)

(defn get-data []
  (pmap fetch-data! (range 100)))

(time (doall (get-data)))
;; "Elapsed time: 4001.424277 msecs"

Not bad - no third party libs needed to be included, no user specified thread / atomicity handling, and done quite fast with no tuning required.

You may be tempted to say, "well, that's stupid! it's just a toy problem!", to which I would retort - "that's exactly the point - if your blub language can't express a toy problem succintly, how well does it express a real problem?".

Jumping from here to actually doing API fetching would maybe increase the Clojure implementation by one or two lines of code.

Discuss

So - how did you do? Please ping me with your implementation and results.

Did you encounter any "gotchas" of parallelism? Perhaps your output array/vector was missing some entries and didn't have all 100 rows present (Clojure's did not thanks to atoms)?

Did you find that it was a lot of work and effort to manage chunking the requests if your language doesn't have builtins for it?

Were you able to accomplish similar using 3rd party packages in your blub lang?

Round 1 - Clojure vs Blub Lang - Parallelism

Table of Contents