Simple is not always easy

There was a good question on one of the Clojure lists the other day. Someone asked the equivalent of why the following code returned the usual value of 3, and didn’t print the message.

user=> (binding [+ (fn [& args] (println "called"))] (+ 1 2))
3

The answer isn’t as easy as it might appear. There are several reasons why it is not possible to intercept a function using the binding mechanism.

The first is these is inlining. The compiler looks for metadata under the tags :inline and :inline-arities. If :inline-arities contains the integer corresponding to the number of arguments to which the Var is applied or it is not present or nil, then the function associated with :inline is called with those arguments.

user=> (defn myFunc
  { :inline (fn [x y & z] ‘(println "inlined")) :inline-arities #{2 3}}
  [& args]
  (println "Normal myFunc called"))
#’user/myFunc
user=> (myFunc)
Normal myFunc called
nil
user=> (myFunc 1)
Normal myFunc called
nil
user=> (myFunc 1 2)
inlined
nil
user=> (myFunc 1 2 3)
inlined
nil
user=> (myFunc 1 2 3 4)
Normal myFunc called
nil
user=> (defn boo [] (myFunc 1 2))
#’user/boo
user=> (boo)
inlined
nil

Notice how the inline function needs to return the code that the compiler goes on to process. Hence the body, in the example above, needs to be quoted so that we return the code and don’t evaluate it at compile time.
user=> (defn myFunc
  { :inline (fn [x y & z] `(println "inlined" ‘~x ‘~y ‘~z)) :inline-arities #{2 3}}
  [& args]
  (println "Normal myFunc called"))
#’user/myFunc
user=> (myFunc 1 #’boo 20)
inlined 1 (var boo) (20)
nil
user=> (myFunc 1 #’boo (* 20 50))
inlined 1 (var boo) ((* 20 50))
nil

We can write some functions to walk over all of the Vars in all of the namespaces and filter out those which aren’t tagged as inline.
user=> (defn is-inline [sym] (:inline (meta sym)))
#’user/is-inline
user=> (defn filter-ns [ns]
       (filter is-inline
        (map #(ns-resolve ns %)
          (map first (ns-map ns)))))
#’user/filter-ns
user=> (set (mapcat filter-ns (all-ns)))
#{#’clojure.core/= #’clojure.core/float #’clojure.core/neg? #’clojure.core/long #’clojure.core// #’clojure.core/== #’clojure.core/unchecked-multiply #’clojure.c
ore/unchecked-remainder #’clojure.core/+ #’clojure.core/* #’clojure.core/bit-xor #’clojure.core/floats #’clojure.core/>= #’clojure.core/bit-and #’clojure.core/u
nchecked-dec #’clojure.core/int-array #’clojure.core/unchecked-divide #’clojure.core/unchecked-inc #’clojure.core/longs #’clojure.core/bit-or #’clojure.core/acl
one #’clojure.core/ints #’clojure.core/unchecked-negate #’clojure.core/<= #’clojure.core/unchecked-subtract #’clojure.core/alength #’clojure.core/aset #’clojure
.core/byte #’clojure.core/aget #’clojure.core/pos? #’clojure.core/- #’clojure.core/long-array #’clojure.core/int #’clojure.core/< #’clojure.core/boolean #’cloju
re.core/zero? #’clojure.core/compare #’user/myFunc #’clojure.core/dec #’clojure.core/float-array #’clojure.core/double-array #’clojure.core/> #’clojure.core/cha
r #’clojure.core/unchecked-add #’clojure.core/inc #’clojure.core/double #’clojure.core/doubles #’clojure.core/bit-not #’clojure.core/short #’clojure.core/num}
user=> (doseq [var *1]
         (println var (:inline-arities (meta var))))
#’clojure.core/= #{2}
#’clojure.core/float nil
#’clojure.core/neg? nil
#’clojure.core/long nil
#’clojure.core// #{2}
#’clojure.core/== #{2}
#’clojure.core/unchecked-multiply nil
#’clojure.core/unchecked-remainder nil
#’clojure.core/+ #{2}
#’clojure.core/* #{2}
#’clojure.core/bit-xor nil
#’clojure.core/floats nil
#’clojure.core/>= #{2}
#’clojure.core/bit-and nil
#’clojure.core/unchecked-dec nil
#’clojure.core/int-array #{1 2}
#’clojure.core/unchecked-divide nil
#’clojure.core/unchecked-inc nil
#’clojure.core/longs nil
#’clojure.core/bit-or nil
#’clojure.core/aclone nil
#’clojure.core/ints nil
#’clojure.core/unchecked-negate nil
#’clojure.core/<= #{2}
#’clojure.core/unchecked-subtract nil
#’clojure.core/alength nil
#’clojure.core/aset #{3}
#’clojure.core/byte nil
#’clojure.core/aget #{2}
#’clojure.core/pos? nil
#’clojure.core/- #{1 2}
#’clojure.core/long-array #{1 2}
#’clojure.core/int nil
#’clojure.core/< #{2}
#’clojure.core/boolean nil
#’clojure.core/zero? nil
#’clojure.core/compare nil
#’user/myFunc #{2 3}
#’clojure.core/dec nil
#’clojure.core/float-array #{1 2}
#’clojure.core/double-array #{1 2}
#’clojure.core/> #{2}
#’clojure.core/char nil
#’clojure.core/unchecked-add nil
#’clojure.core/inc nil
#’clojure.core/double nil
#’clojure.core/doubles nil
#’clojure.core/bit-not nil
#’clojure.core/short nil
#’clojure.core/num nil
nil

We can see that the inlining arity for + is {2}, so a three argument call ought to allow intercepting by binding, and indeed it does.
user=> (binding [+ (fn [& args] (println "called"))] (+ 1 2 3))
called
nil

The compiler deals with inline functions in the analyzeSeq method, when it is processing a form
Object op = RT.first(form);
if(op == null)
    throw new IllegalArgumentException("Can’t call nil");
IFn inline = isInline(op, RT.count(RT.next(form)));
if(inline != null)
    return analyze(context, inline.applyTo(RT.next(form)));

The second way a Var can avoid interception is by being a macro definition. Vars are flagged as being macros by the presence of metadata. For example, let is a macro and hence contains a :macro marker in its metadata.
user=> (meta #’let)
{:macro true, :ns #<Namespace clojure.core>, :name let, :file "clojure/core.clj", :line 2592, :arglists ([bindings & body]), :doc "Evaluates the exprs in a lexi
cal context in which the symbols inn  the binding-forms are bound to their respective init-exprs or partsn  therein."}

The compiler checks for macros just before it processes inline functions.
Object me = macroexpand1(form);
if(me != form)
    return analyze(context, me, name);

Again, the macro (possibly) returns a modified form that the compiler will continue to process so the body will often need to be quoted.
user=> (defn myFunc
  { :macro true }
  [& args]
  (println "Macro function being called" args)
  ‘(println "Generated macro code called"))
#’user/myFunc
user=> (defn boo[] (myFunc (* 10 20)))
Macro function being called ((* 10 20))
#’user/boo
user=> (boo)
Generated macro code called
nil

Going back to our original question, the following macros are present in the base system, and so you can’t intercept these with binding.
user=> (defn macro-filter-ns [ns]
       (filter (comp :macro meta)
        (map #(ns-resolve ns %)
          (map first (ns-map ns)))))
#’user/macro-filter-ns
user=> (set (mapcat macro-filter-ns (all-ns)))
#{#’clojure.core/when-first #’clojure.core/.. #’clojure.core/areduce #’clojure.core/letfn #’clojure.core/doc #’clojure.core/amap #’clojure.core/with-out-str #’c
lojure.core/and #’clojure.core/-> #’clojure.main/with-bindings #’clojure.core/defmulti #’clojure.core/loop #’clojure.core/gen-interface #’clojure.core/defn #’cl
ojure.core/delay #’clojure.core/definline #’clojure.core/memfn #’clojure.core/condp #’clojure.core/io! #’clojure.core/doseq #’clojure.core/dosync #’clojure.core
/sync #’clojure.core/or #’clojure.core/with-in-str #’clojure.core/defonce #’clojure.core/let #’clojure.core/if-let #’clojure.core/pvalues #’clojure.core/ns #’cl
ojure.core/defmacro #’clojure.core/assert-args #’clojure.core/when #’clojure.core/if-not #’clojure.core/doto #’clojure.core/add-doc #’clojure.core/cond #’clojur
e.core/future #’clojure.core/with-open #’clojure.core/gen-class #’clojure.core/binding #’clojure.core/with-local-vars #’clojure.core/proxy-super #’clojure.core/
declare #’clojure.core/comment #’clojure.core/while #’clojure.core/proxy #’clojure.core/defn- #’clojure.core/refer-clojure #’clojure.core/def-aset #’clojure.cor
e/defmethod #’user/myFunc #’clojure.core/lazy-cat #’clojure.core/defstruct #’clojure.core/time #’clojure.core/assert #’clojure.core/when-let #’clojure.core/lazy
-seq #’clojure.core/for #’clojure.core/with-precision #’clojure.core/dotimes #’clojure.core/locking #’clojure.core/fn #’clojure.core/when-not}

Note that the defmacro macro is the best way to define macros rather than using the expanded syntax above – defmacro expands into code which calls the setMacro java method on the Var class to adjust the metadata for :macro to true.
user=> (defmacro myMacro [] (println "expand") ‘(print "expanded"))
#’user/myMacro
user=> (defn boo[] (myMacro))
expand
#’user/boo
user=> (boo)
expandednil

Of course, when I said we couldn’t use binding to catch uses, I meant in the sense of binding myMacro at runtime and hoping the rebound function will be used during the execution of boo. I can rebind the macro function and this will indeed by called when the compiler needs to use it to calculate the macro expansion.
user=> (binding [myMacro (fn [] ‘(print "expanded but different"))]
(eval ‘(defn boo[] (myMacro))))
#’user/boo
user=> (boo)
expanded but differentnil

There is another set of things that the compiler treats specially – the special forms. As an example of one of these, we can look at let. Let is a macro that expands into let*, but when we look up let* we find that it has no Var binding.
user=> (macroexpand ‘(let [x 3] 4))
(let* [x 3] 4)
user=> (resolve ‘let*)
nil
user=> (resolve ‘let)
#’clojure.core/let

The compiler has a set list of symbols that are treated specially.
static final public IPersistentMap specials = PersistentHashMap.create(
        DEF, new DefExpr.Parser(),
        LOOP, new LetExpr.Parser(),
        RECUR, new RecurExpr.Parser(),
        IF, new IfExpr.Parser(),
        LET, new LetExpr.Parser(),
        LETFN, new LetFnExpr.Parser(),
        DO, new BodyExpr.Parser(),
        QUOTE, new ConstantExpr.Parser(),
        THE_VAR, new TheVarExpr.Parser(),
        DOT, new HostExpr.Parser(),
        ASSIGN, new AssignExpr.Parser(),
        TRY, new TryExpr.Parser(),
        THROW, new ThrowExpr.Parser(),
        MONITOR_ENTER, new MonitorEnterExpr.Parser(),
        MONITOR_EXIT, new MonitorExitExpr.Parser(),
        NEW, new NewExpr.Parser(),
             ……..
);

If we attempt to define one of these special forms, our definition will be ignored, and the system definition will be used instead.
user=> (defn let* [] (print "boo"))
#’user/let*
user=> (let [x 3] 4)
4
user=> (resolve ‘let*)
#’user/let*
user=> (*1)
boonil
user=> (let* [x 3] 4)
4

I’d be interested to hear of any more!

Advertisements
This entry was posted in Computers and Internet. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s