So, maybe you have, or maybe you haven't heard of Clojure.
I had heard of it years ago, but never really gave it a lot of thought (or time) to it, despite being a huge fan of Common Lisp and a bit of a language polyglot. I think part of my initial reluctance may have been 'java' - as I would typically associate just the negatives to it (cumbersome language syntax, slow start up time, memory hog etc.).
About a month ago, I began listening to more of the Clojure talks by Rich Hickey (inventor of Clojure) as well as poking around the language a bit (checking out their new specification system, known as spec). It looked really neat! Despite being a bit of a 'static type' adherent, I could see a lot of merit in it. Anyways - this post is not about that - I had a need for some real work to be done, a customized browser for my own use, as recently GNU Icecat (Firefox 45 ESR), the browser I have used for many years, became mostly unusable. My lock-in to that older version was due to the Firefox 55+ quantum updates breaking my top level key binds, and introducing a massively gimped 'vim' like setup for my Firefox driving (I used Pentadactyl, and around the same time, a similar browser known as vimprobable2).
I needed a language I could easily get a GUI going on (preferably a light weight/simple one, such as webkit2). Doing a bit of shopping, I came across JavaFX (circa 2012) which integrates with webkit2 on all available OSes. Given that I love lisp and all, and Clojure was a lisp running on Java, it seemed like a perfect alignment of time and necessity that made me dive in and begin working on my own browser implementation, AHUBU (https://github.com/ahungry/ahubu).
So, enough with the backstory/ranting and on to why I believe it (Clojure) is cool.
This is the single biggest selling point to most lisps (and the overbearing factor in my love for them), and is completely unique, not just to development compared to a 'typical' language (compiled or interpreted), but unique/superior even to other languages that lean towards a REPL based approach (J / APL / Prolog / Elixir) of programming and trying out some code.
I like to term this as 'time between cause and effect' - or the time it takes between making a change, and seeing the result of that change.
In a compiled language, this is usually very large - a programmer will update the program, recompile it, execute it, and finally see the result of the change.
In an interpreted language, this is slightly less - the compilation step is removed.
In a test driven development (TDD) methodology, this is even less - the execute it scope is reduced from the entire world of the software, to the specific segment of code being developed (but, this requires a lot of boilerplate, and even the test runner is not always tightly coupled to an individual test at the editor level).
Some of the languages that offer a pleasant REPL experience (even PHP and Nodejs give a decent built in REPL) still run into the problem of having 2 separate environments - there is the actual world of the software (the execution runtime), and then an isolated 'interactive' environment - often a very manual and ephemeral experience - the user runs code - those commands run simply disappear when the session ends etc.
What if there was a coupling between the code itself, at an individual snippet level, and the REPL environment? And what if the REPL environment and the execution runtime/world were one in the same? With lisp(s), that's the case!
It was a bit mind blowing my first time trying it - "do I type code in the REPL and it saves it to a source file somewhere?" I asked myself (or perhaps some kind folks in IRC)? No! You actually modify your program in your source file, select an s-expression (a block of code surrounded by parenthesis) and evaluate just that block of code in your running REPL / runtime image. It was a bit mind blowing to realize, and a great revelation was revealed, as it became clear that all those parenthesis were for a very good reason (for this interactive functionality to work well - imagine trying to re-evaluate a single snippet of code in ANY other non-lisp language - good luck writing a parser that can handle redefining a single method without having to 'hot reload' an entire file (or, in many cases, an entire project!). Lets just say, it makes all those other times between cause and effect appear to be eons (even those of you who may have tried out front end with 'fast' hot module reloading like typescript fusebox, or create-react-app - try out something like Clojurescript and fighweel - you will not want to go back).
As I mentioned earlier - lately I've been a bit of a type adherent - things should all be strictly and statically typed! How can anyone read the code and know it should do what it's supposed to do without the types! Well - a lot of the time its just extra noise and hoops to jump through. Rich Hickey made a great reference about using 'tests' (unit tests, TDD) as a guard rail during driving, and has alluded to similar for heavily typed systems. When we drive, do we just let the car run against the guard rail to guide us to where we're going? or do we make an intelligent decision to follow the road on our own? Clojure (and lisps like Common Lisp / Emacs Lisp) are the latter, and the code to REPL to feedback cycle is the car on the road.
To illustrate the conciseness and the beauty of the code, lets compare two practical examples of achieving a task in Clojure vs Java - in this case, the task of extending JavaFX to use persistent cookies (cookies that are saved after closing and re-opening a browser).
I will go over it, broken down by class/need, and then by language.
Also - thanks to Manish for the github gist - it was a useful reference:
First up, we have our contender, Java:
/** * Taken from https://github.com/loopj/android-async-http */ package com.orb.net; import java.io.IOException; import java.io.ObjectInputStream; import java.io.ObjectOutputStream; import java.io.Serializable; import java.net.HttpCookie; /** * A simple wrapper for {@link HttpCookie} to work with {@link SiCookieStore2} * Gives power of serialization-deserialization to {@link HttpCookie} * @author Manish * */ public class SICookie2 implements Serializable { private HttpCookie mHttpCookie; /** * */ private static final long serialVersionUID = 2532101328282342578L; /** * */ public SICookie2(HttpCookie cookie) { this.mHttpCookie = cookie; } public HttpCookie getCookie() { return mHttpCookie; } private void writeObject(ObjectOutputStream out) throws IOException { out.writeObject(mHttpCookie.getName()); out.writeObject(mHttpCookie.getValue()); out.writeObject(mHttpCookie.getComment()); out.writeObject(mHttpCookie.getCommentURL()); out.writeBoolean(mHttpCookie.getDiscard()); out.writeObject(mHttpCookie.getDomain()); out.writeLong(mHttpCookie.getMaxAge()); out.writeObject(mHttpCookie.getPath()); out.writeObject(mHttpCookie.getPortlist()); out.writeBoolean(mHttpCookie.getSecure()); out.writeInt(mHttpCookie.getVersion()); } private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException { String name = (String) in.readObject(); String value = (String) in.readObject(); mHttpCookie = new HttpCookie(name, value); mHttpCookie.setComment((String) in.readObject()); mHttpCookie.setCommentURL((String) in.readObject()); mHttpCookie.setDiscard(in.readBoolean()); mHttpCookie.setDomain((String) in.readObject()); mHttpCookie.setMaxAge(in.readLong()); mHttpCookie.setPath((String) in.readObject()); mHttpCookie.setPortlist((String) in.readObject()); mHttpCookie.setSecure(in.readBoolean()); mHttpCookie.setVersion(in.readInt()); } }
Ok, it was a bit of a read, but I hope you, the reader, can see that it's providing two useful methods - one to write an object to disk, and one to instantiate an object from disk.
But, could we do better?
And next, we have Clojure stepping into the arena. Oh - did you know that in many (all?) lisps, code is data? What does that mean you ask? Do some searching and find out. The gist of it is that the code you see in the editor is the same code produced by the underlying language AST - you have no need for an 'abstract syntax tree' - the code is the derived output the computer uses, and as such, it is very easy to pass around and extend, serialize, unserialize, etc.
Please excuse my crude naming conventions for some of this - Clojure comes with a built in read-from-file function called 'slurp', and the Emacs equivalent to 'do the opposite of slurp' is usually termed 'barf'.
(def world (atom {:cookies {}})) (defn barf [file-name data] (with-open [wr (clojure.java.io/writer file-name)] (.write wr (pr-str data)))) (defn cookie-to-map [cookie] {:name (.getName cookie) :value (.getValue cookie) :domain (.getDomain cookie) :maxAge (.getMaxAge cookie) :secure (.getSecure cookie)}) (defn cookiemap-to-cookie [{name :name value :value domain :domain maxAge :maxAge secure :secure}] (let [cookie (java.net.HttpCookie. name value)] (doto cookie (.setVersion 0) (.setDomain domain) (.setSecure secure) (.setMaxAge maxAge)))) (defn save-cookies [] (barf "ahubu.cookies" (:cookies @world)))
Huh - weird, the Clojure one looks much smaller - that's 1/3rd the code - that's odd right (not to mention the Clojure includes the disk I/O, while the Java sample only illustrates how to map the objects to the disk equivalents)?
Well, each cookie we operate on in Clojure is a map (or map of maps of maps etc.) - similar to JSON / Javascript / Nodejs, the Clojure values are very easy to read the literal data of (and to write it).
Well, get ready for a big one…
/** * Inspired from https://github.com/loopj/android-async-http */ package com.orb.net; import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; import java.io.IOException; import java.io.ObjectInputStream; import java.io.ObjectOutputStream; import java.net.CookieStore; import java.net.HttpCookie; import java.net.URI; import java.net.URISyntaxException; import java.util.ArrayList; import java.util.Collection; import java.util.Collections; import java.util.HashMap; import java.util.HashSet; import java.util.Iterator; import java.util.List; import java.util.Locale; import java.util.Map; import java.util.Set; import android.annotation.SuppressLint; import android.content.Context; import android.content.SharedPreferences; import android.text.TextUtils; import com.orb.common.Log; /** * Implementation of {@link CookieStore} for persistence cookies, uses shared * preference for storing cookies. * * @author Manish * */ public class SiCookieStore2 implements CookieStore { private static final String LOG_TAG = "SICookieStore2"; private static final String COOKIE_PREFS = "com.orb.net.cookieprefs"; private static final String COOKIE_DOMAINS_STORE = "com.orb.net.CookieStore.domain"; private static final String COOKIE_DOMAIN_PREFIX = "com.orb.net.CookieStore.domain_"; private static final String COOKIE_NAME_PREFIX = "com.orb.net.CookieStore.cookie_"; /*This map here will store all domain to cookies bindings*/ private final CookieMap map; private final SharedPreferences cookiePrefs; /** * Construct a persistent cookie store. * * @param context * Context to attach cookie store to */ public SiCookieStore2(Context context) { cookiePrefs = context.getSharedPreferences(COOKIE_PREFS, 0); map = new CookieMap(); // Load any previously stored domains into the store String storedCookieDomains = cookiePrefs.getString(COOKIE_DOMAINS_STORE, null); if (storedCookieDomains != null) { String[] storedCookieDomainsArray = TextUtils.split(storedCookieDomains, ","); //split this domains and get cookie names stored for each domain for (String domain : storedCookieDomainsArray) { String storedCookiesNames = cookiePrefs.getString(COOKIE_DOMAIN_PREFIX + domain, null); //so now we have these cookie names if (storedCookiesNames != null) { //split these cookie names and get serialized cookie stored String[] storedCookieNamesArray = TextUtils.split(storedCookiesNames, ","); if (storedCookieNamesArray != null) { //in this list we store all cookies under one URI List<HttpCookie> cookies = new ArrayList<HttpCookie>(); for (String cookieName : storedCookieNamesArray) { String encCookie = cookiePrefs.getString(COOKIE_NAME_PREFIX + domain + cookieName, null); //now we deserialize or unserialize (whatever you call it) this cookie //and get HttpCookie out of it and pass it to List if (encCookie != null) cookies.add(decodeCookie(encCookie)); } map.put(URI.create(domain), cookies); } } } } } public synchronized void add(URI uri, HttpCookie cookie) { if (cookie == null) { throw new NullPointerException("cookie == null"); } uri = cookiesUri(uri); List<HttpCookie> cookies = map.get(uri); if (cookies == null) { cookies = new ArrayList<HttpCookie>(); map.put(uri, cookies); } else { cookies.remove(cookie); } cookies.add(cookie); // Save cookie into persistent store SharedPreferences.Editor prefsWriter = cookiePrefs.edit(); prefsWriter.putString(COOKIE_DOMAINS_STORE, TextUtils.join(",", map.keySet())); Set<String> names = new HashSet<String>(); for (HttpCookie cookie2 : cookies) { names.add(cookie2.getName()); prefsWriter.putString(COOKIE_NAME_PREFIX + uri + cookie2.getName(), encodeCookie(new SICookie2(cookie2))); } prefsWriter.putString(COOKIE_DOMAIN_PREFIX + uri, TextUtils.join(",", names)); prefsWriter.commit(); } public synchronized List<HttpCookie> get(URI uri) { if (uri == null) { throw new NullPointerException("uri == null"); } List<HttpCookie> result = new ArrayList<HttpCookie>(); // get cookies associated with given URI. If none, returns an empty list List<HttpCookie> cookiesForUri = map.get(uri); if (cookiesForUri != null) { for (Iterator<HttpCookie> i = cookiesForUri.iterator(); i.hasNext();) { HttpCookie cookie = i.next(); if (cookie.hasExpired()) { i.remove(); // remove expired cookies } else { result.add(cookie); } } } // get all cookies that domain matches the URI for (Map.Entry<URI, List<HttpCookie>> entry : map.entrySet()) { if (uri.equals(entry.getKey())) { continue; // skip the given URI; we've already handled it } List<HttpCookie> entryCookies = entry.getValue(); for (Iterator<HttpCookie> i = entryCookies.iterator(); i.hasNext();) { HttpCookie cookie = i.next(); if (!HttpCookie.domainMatches(cookie.getDomain(), uri.getHost())) { continue; } if (cookie.hasExpired()) { i.remove(); // remove expired cookies } else if (!result.contains(cookie)) { result.add(cookie); } } } return Collections.unmodifiableList(result); } public synchronized List<HttpCookie> getCookies() { List<HttpCookie> result = new ArrayList<HttpCookie>(); for (List<HttpCookie> list : map.values()) { for (Iterator<HttpCookie> i = list.iterator(); i.hasNext();) { HttpCookie cookie = i.next(); if (cookie.hasExpired()) { i.remove(); // remove expired cookies } else if (!result.contains(cookie)) { result.add(cookie); } } } return Collections.unmodifiableList(result); } public synchronized List<URI> getURIs() { List<URI> result = new ArrayList<URI>(map.getAllURIs()); result.remove(null); // sigh return Collections.unmodifiableList(result); } public synchronized boolean remove(URI uri, HttpCookie cookie) { if (cookie == null) { throw new NullPointerException("cookie == null"); } if (map.removeCookie(uri, cookie)) { SharedPreferences.Editor prefsWriter = cookiePrefs.edit(); prefsWriter.putString(COOKIE_DOMAIN_PREFIX + uri, TextUtils.join(",", map.getAllCookieNames(uri))); prefsWriter.remove(COOKIE_NAME_PREFIX + uri + cookie.getName()); prefsWriter.apply(); return true; } return false; } public synchronized boolean removeAll() { // Clear cookies from persistent store SharedPreferences.Editor prefsWriter = cookiePrefs.edit(); prefsWriter.clear(); prefsWriter.commit(); // Clear cookies from local store boolean result = !map.isEmpty(); map.clear(); return result; } /** * Serializes HttpCookie object into String * * @param cookie * cookie to be encoded, can be null * @return cookie encoded as String */ protected String encodeCookie(SICookie2 cookie) { if (cookie == null) return null; ByteArrayOutputStream os = new ByteArrayOutputStream(); try { ObjectOutputStream outputStream = new ObjectOutputStream(os); outputStream.writeObject(cookie); } catch (IOException e) { Log.e(LOG_TAG, "IOException in encodeCookie", e); return null; } return byteArrayToHexString(os.toByteArray()); } /** * Returns HttpCookie decoded from cookie string * * @param cookieString * string of cookie as returned from http request * @return decoded cookie or null if exception occured */ protected HttpCookie decodeCookie(String cookieString) { byte[] bytes = hexStringToByteArray(cookieString); ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes); HttpCookie cookie = null; try { ObjectInputStream objectInputStream = new ObjectInputStream(byteArrayInputStream); cookie = ((SICookie2) objectInputStream.readObject()).getCookie(); } catch (IOException e) { Log.e(LOG_TAG, "IOException in decodeCookie", e); } catch (ClassNotFoundException e) { Log.e(LOG_TAG, "ClassNotFoundException in decodeCookie", e); } return cookie; } /** * Using some super basic byte array <-> hex conversions so we don't * have to rely on any large Base64 libraries. Can be overridden if you * like! * * @param bytes * byte array to be converted * @return string containing hex values */ protected String byteArrayToHexString(byte[] bytes) { StringBuilder sb = new StringBuilder(bytes.length * 2); for (byte element : bytes) { int v = element & 0xff; if (v < 16) { sb.append('0'); } sb.append(Integer.toHexString(v)); } return sb.toString().toUpperCase(Locale.US); } /** * Converts hex values from strings to byte arra * * @param hexString * string of hex-encoded values * @return decoded byte array */ protected byte[] hexStringToByteArray(String hexString) { int len = hexString.length(); byte[] data = new byte[len / 2]; for (int i = 0; i < len; i += 2) { data[i / 2] = (byte) ((Character.digit(hexString.charAt(i), 16) << 4) + Character .digit(hexString.charAt(i + 1), 16)); } return data; } /** * Utility function to male sure that every time you get consistent URI * @param uri * @return */ private URI cookiesUri(URI uri) { if (uri == null) { return null; } try { return new URI(uri.getScheme(), uri.getHost(), null, null); } catch (URISyntaxException e) { return uri; } } /** * A implementation of {@link Map} for utility class for storing URL cookie map * @author Manish * */ private class CookieMap implements Map<URI, List<HttpCookie>> { private final Map<URI, List<HttpCookie>> map; /** * */ public CookieMap() { map = new HashMap<URI, List<HttpCookie>>(); } /* * (non-Javadoc) * * @see java.util.Map#clear() */ @Override public void clear() { map.clear(); } /* * (non-Javadoc) * * @see java.util.Map#containsKey(java.lang.Object) */ @Override public boolean containsKey(Object key) { return map.containsKey(key); } /* * (non-Javadoc) * * @see java.util.Map#containsValue(java.lang.Object) */ @Override public boolean containsValue(Object value) { return map.containsValue(value); } /* * (non-Javadoc) * * @see java.util.Map#entrySet() */ @Override public Set<java.util.Map.Entry<URI, List<HttpCookie>>> entrySet() { return map.entrySet(); } /* * (non-Javadoc) * * @see java.util.Map#get(java.lang.Object) */ @Override public List<HttpCookie> get(Object key) { return map.get(key); } /* * (non-Javadoc) * * @see java.util.Map#isEmpty() */ @Override public boolean isEmpty() { return map.isEmpty(); } /* * (non-Javadoc) * * @see java.util.Map#keySet() */ @Override public Set<URI> keySet() { return map.keySet(); } /* * (non-Javadoc) * * @see java.util.Map#put(java.lang.Object, java.lang.Object) */ @Override public List<HttpCookie> put(URI key, List<HttpCookie> value) { return map.put(key, value); } /* * (non-Javadoc) * * @see java.util.Map#putAll(java.util.Map) */ @Override public void putAll(Map<? extends URI, ? extends List<HttpCookie>> map) { this.map.putAll(map); } /* * (non-Javadoc) * * @see java.util.Map#remove(java.lang.Object) */ @Override public List<HttpCookie> remove(Object key) { return map.remove(key); } /* * (non-Javadoc) * * @see java.util.Map#size() */ @Override public int size() { return map.size(); } /* * (non-Javadoc) * * @see java.util.Map#values() */ @Override public Collection<List<HttpCookie>> values() { return map.values(); } /** * List all URIs for which cookies are stored in map * @return */ public Collection<URI> getAllURIs() { return map.keySet(); } /** * Get all cookies names stored for given URI * @param uri * @return */ public Collection<String> getAllCookieNames(URI uri) { List<HttpCookie> cookies = map.get(uri); Set<String> cookieNames = new HashSet<String>(); for (HttpCookie cookie : cookies) { cookieNames.add(cookie.getName()); } return cookieNames; } /** * Removes requested {@link HttpCookie} {@code httpCookie} from given {@code uri} value * @param uri * @param httpCookie * @return */ public boolean removeCookie(URI uri, HttpCookie httpCookie) { if (map.containsKey(uri)) { return map.get(uri).remove(httpCookie); } else { return false; } } } }
Ok - that was a pretty rough read through - I mean, it's good clean code, but its a LOT to keep in your head (close to 500 lines). It also includes an inline implementation of map type tracking, and a re-implementation of every store method.
Hold on to your seat-belt, it's going to get crazy:
(defn clean-uri [uri] (java.net.URI. (.getScheme uri) (.getHost uri) nil nil)) (defn add-cookie [store uri cookiemap] (let [cookie (cookiemap-to-cookie cookiemap) uri (clean-uri (java.net.URI. uri))] (-> store (.add uri cookie)))) (defn load-cookies [store] (when (.exists (clojure.java.io/file "ahubu.cookies")) (let [cookies (read-string (slurp "ahubu.cookies"))] (doseq [[uri uri-map] cookies] (doseq [[name cookie] uri-map] (add-cookie store uri cookie)))))) (defn push-cookie-to-uri-map [cookie mp] (let [name (:name cookie)] (assoc mp name cookie))) (defn push-cookie-to-cookie-map [cookie uri mp] (let [old (get mp uri)] (assoc mp uri (push-cookie-to-uri-map cookie old)))) (defn push-cookie-to-world [uri cookie] (swap! world (fn [old] (assoc old :cookies (push-cookie-to-cookie-map cookie uri (:cookies old)))))) (defn my-cookie-store [] (let [store (-> (java.net.CookieManager.) .getCookieStore) my-store (proxy [java.net.CookieStore] [] (add [uri cookie] (let [clean (clean-uri uri) u (.toString clean)] (.add store clean cookie) (push-cookie-to-world u (cookie-to-map cookie)))) (get [& [uri :as args]] (let [clean (clean-uri uri)] (.get store clean))) (getCookies [] (.getCookies store)) (getURIs [] (.getURIs store)) (remove [uri cookie] (.remove store uri cookie)) (removeAll [] (.removeAll store)))] (load-cookies my-store) my-store))
Oh, weird, 40 lines of code (1/10th the SLOC)? And it does much more? How is that even possible!
Well, go read up on using Clojure and find out! Lets just say, the functionality of the two is nearly identical. Now, ask yourself, which would you rather maintain if you inherited the codebase?
In conclusion, I hope you give Clojure (or most any lisp) a try - they are really great languages and worth using!
Let me know! m @ ahungry dot com (put it together :p )
Or hit me up on on Twitter @ahungry