For this we're going to tap into a few third party libraries (all available
via Quicklisp) to make our life easier.
This is done by either directly typing:
(ql:quickload :library-name-here)
or by adding to the language-popularity.asd file (ASDF - Another
System Definition Facility) and including in the :use portion of our
defpackage.
Now we should have a language-popularity.asd file that looks like:
:sxql
;; for other tasks
:drakma
:cl-json
:split-sequence
:vecto
:md5)
and a defpackage in src/model.lisp that looks like:
(defpackage language-popularity.model
(:use :cl
:cl-json
:drakma
:vecto
:md5
:split-sequence)
(:import-from :language-popularity.config
:config)
(:export :get-language-sub-stats
:pie-chart))
Ignore the #'pie-chart and :vecto portions will come later, so lets
just add a stub/placeholder function for that for now (in
src/model.lisp), as well as a very basic class structure for our
Language type class (classes in Common Lisp are done via a system
called CLOS, which is very awesome, but we're mainly using it as an
easy way to store and reference values):
(defclass Language ()
((Subscribers
:accessor subs
:initarg :subs
:initform 0)
(Last-Updated
:accessor last-updated
:initarg :last-updated
:initform 0)
(About
:accessor about
:initarg :about
:initform "Some details about the language."))
(:documentation "Language stats and details"))
(defun pie-chart (slices)
slices)
Next up, we need some functions to:
- Query a remote JSON endpoint (did you know you can add a .json extension to most any reddit page and get a JSON response? pretty sweet!)
- Decode the remote JSON into a string for cl-json to parse
- Parse the JSON string with cl-json to get an object we can use
Add this next (still in src/model.lisp unless I say otherwise!):
(defun char-vector-to-string (v)
(format nil "~{~a~}" (mapcar #'code-char (coerce v 'list))))
(defun remote-json-request (uri)
"Pull in remote JSON. Drakma returns it as a large vector of
character codes, so we have to parse it out to string form for
cl-json."
(let* ((json-response-raw (http-request uri))
(json-response-string (char-vector-to-string json-response-raw))
(json (decode-json-from-string json-response-string)))
json)
The #'http-request function is courtesy of drakma (similar to CURL in
other languages), while #'decode-json-from-string is from cl-json.
Also, for the Common Lisp novices, #' is a way to refer to a function,
so I'll typically add it in front to distinguish it as such.
The drakma #'http-request gives a vector set of ASCII character codes,
so our little #'char-vector-to-string changes it to a basic JSON
string.
Now that we have a way to remotely request some parsed out JSON, we
can tap into the reddit endpoints and store some instances of our new
Language objects with actual data.
For this, we'll just throw them in a hash table, as it'll give us an
easy way to query out details based on a language name.
Add this:
(defparameter *language-stats* (make-hash-table :test #'equal))
(defconstant +cache-time+ (* 60 60)) ;; 1 hour
(defmacro jkey (k &rest rest)
`(cdr (assoc ,k ,@rest)))
(defun set-language-stats (language)
"Build language stats into our lang class via external sources of
popularity."
(let ((lang-class (or (gethash language *language-stats*) (make-instance 'Language))))
(when (> (- (get-universal-time) (last-updated lang-class)) +cache-time+)
(let ((reddit-json
(remote-json-request
(format nil "http://reddit.com/r/~a/about.json"
language))))
(when (jkey :subscribers (jkey :data reddit-json))
(setf (subs lang-class) (jkey :subscribers (jkey :data reddit-json))))
(setf (last-updated lang-class) (get-universal-time))))
(setf (gethash language *language-stats*) lang-class)
(cons (intern (string-upcase language)) (subs lang-class))))
Holy freaking function Batman! May be your first reaction to seeing
this abomination of a function; I assure you it's not so bad, although
sometimes due to the interactive nature of the Common Lisp REPL and
adding little pieces at a time, it is easy to get carried away and not
properly modularize enough of the code.
So, lets take it line by line.
The language-stats and cache-time parameter and constants are there for
storing Language objects in the former, and keeping our API requests to
reddit down in the latter.
The macro #'jkey (ehh…not a function, but I'll steal the #' syntax) is
to reduce verbosity in the #'set-language-stats by a small amount.
It lets us quickly reference nested keys in a JSON alist created by
cl-json.
Now we enter the beast (I mean #'set-language-stats). It takes a
single parameter/argument (language) and runs a JSON request to
http://reddit.com/r/language/about.json, as long as it's been
at least an hour since our last request (restarting the instance of
your implementation is obviously going to reset this).
When the subscriber key exists (if you request an invalid endpoint, it
won't), it will update the Language object's Subscribers slot.
Following the subscriber update, it adds (or updates) the language's
value under it's name key in the language-stats parameter (yea, yea,
global state, mutability, not ideal, I get it).
Finally, after all of that, we have a little laziness tossed in. The
function ends up returning a cons of the language name interned (so
"haskell" becomes HASKELL) and the subscriber count.
« Back to article list