1. 14
1.

2. 1

…any derivative or gradient, of any function you can program, or of any program that computes a function…

Well, technically automatic differentiation doesn’t applies to “any-any” function but to any function you specify as a series of compositions (and similar operations) of math functions with known derivative. sin(1+x^2) works. A recursively specified factorial wouldn’t work out of the box (but could work if you used gamma or whatever).

It is one of those “look how simple and beautiful this is - here’s a long page of caveats” things.

1. 2

A recursively specified factorial wouldnâ€™t work out of the box

Sure it would.

``````import Numeric.AD

fac :: (Num a, Eq a) => a -> a
fac = go 1 where
go acc 0 = acc
go acc n = go (n * acc) (n - 1)

dfac :: (Num a, Eq a) => a -> a
dfac = diff fac

fac' :: (Eq a, Num a) => a -> a
fac' 0 = 1
fac' n = n * fac' (n - 1)

dfac' :: (Eq a, Num a) => a -> a
dfac' = diff fac'

fac'' :: (Enum a, Num a) => a -> a
fac'' n = product [1..n]

dfac'' :: (Enum a, Num a) => a -> a
dfac'' = diff fac''
``````
1. 1

Just for kicks: this is the most abstract factorial definition I could come up with, and it’s still perfectly amenable to AD.

``````{-# LANGUAGE DeriveFunctor #-}
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE FunctionalDependencies #-}
{-# LANGUAGE MultiParamTypeClasses #-}

import Control.Arrow ((&&&))

newtype Fix f = Fix (f (Fix f))

class Functor f => Fixpoint f t | t -> f where
inF  :: f t -> t
outF :: t -> f t

para :: Fixpoint f t => (f (a, t) -> a) -> t -> a
para alg = alg . fmap (para alg &&& id) . outF

data NatF r =
ZeroF
| SuccF r
deriving Functor

instance (Ord a, Num a) => Fixpoint NatF a where
inF ZeroF = 0
inF (SuccF n) = n + 1
outF n
| n > 0     = SuccF (n - 1)
| otherwise = ZeroF

type Nat = Fix NatF

fac :: (Ord a, Num a) => a -> a
fac = para alg where
alg ZeroF = 1
alg (SuccF (f, n)) = f * (n + 1)

dfac :: (Ord a, Num a) => a -> a
dfac = diff fac
``````
1. 1

Can you explain how this computes the derivative?

What function does it result in? My naive interpretation of

``````fac' :: (Eq a, Num a) => a -> a
fac' 0 = 1
fac' n = n * fac' (n - 1)
``````

is that it gives the derivative of n! as n! and I don’t think that’s right. at least it wouldn’t conform to the gamma function [1], which I think is the only analytic function taking n! for it’s positive values.

Maybe I’m missing something.