I’m interested, although not quite able to grok the design from the documentation so far. The goal seems to be to build a simple but general framework that subsumes all (or at least most) kinds of machine learning methods into a model that’s made up of networks based on layers that are then trained using solvers. Although the terminology of nets and layers is a bit neural-net-ish, the docs say that it’s more general,
A layer can implement basically any behaviour: deep learning related like convolutions or LSTM, classical machine learning related like nearest neighbors or random forest, or utility related like logging or normalization.
It’s not immediately obvious to me how you’d implement a random forest (or CART-style decision tree, or boosting, or Bayesian inference) in this framework though. I mean if you can call out to arbitrary code in a layer then clearly you can implement anything at all, but I assume they mean something stronger than just calling out to a separate ML algorithm that doesn’t otherwise use the layer machinery. It’d really help to see an example of something not-at-all-NN-ish in this framework to get an understanding of how it’d work.
Browsing the source, it looks like neural nets are baked into the general “Solver” interface here. So it’s definitely geared toward neural nets, but that kinda makes sense because neural nets are super general and can fit almost any problem (even if a NN isn’t the most optimal solution).
Digging deeper, here is an example of a solver, Stochastic Gradient Descent. Click on [src] and you’ll see that the SDG uses the net field of the Solver struct to store it’s learned weights here. So maybe this net is just a matrix?
In any case it’s still very neural net-ish.
The backend to Leaf is called “collenchyma” which has this tensor abstraction which is just an array or a matrix as the docs state and could presumably handle any machine-learning algo.
I’m interested, although not quite able to grok the design from the documentation so far. The goal seems to be to build a simple but general framework that subsumes all (or at least most) kinds of machine learning methods into a model that’s made up of networks based on layers that are then trained using solvers. Although the terminology of nets and layers is a bit neural-net-ish, the docs say that it’s more general,
It’s not immediately obvious to me how you’d implement a random forest (or CART-style decision tree, or boosting, or Bayesian inference) in this framework though. I mean if you can call out to arbitrary code in a layer then clearly you can implement anything at all, but I assume they mean something stronger than just calling out to a separate ML algorithm that doesn’t otherwise use the layer machinery. It’d really help to see an example of something not-at-all-NN-ish in this framework to get an understanding of how it’d work.
Browsing the source, it looks like neural nets are baked into the general “Solver” interface here. So it’s definitely geared toward neural nets, but that kinda makes sense because neural nets are super general and can fit almost any problem (even if a NN isn’t the most optimal solution).
Digging deeper, here is an example of a solver, Stochastic Gradient Descent. Click on
[src]and you’ll see that the SDG uses thenetfield of theSolverstruct to store it’s learned weights here. So maybe thisnetis just a matrix?In any case it’s still very neural net-ish.
The backend to Leaf is called “collenchyma” which has this tensor abstraction which is just an array or a matrix as the docs state and could presumably handle any machine-learning algo.
Previously on lobste.rs without any comments.