:: Traditional ML in Mojo ::

Implementing Machine Learning algorithms from scratch in 🔥Mojo🔥

---

Inspired by this in which the author had implemented traditional Machine Learning algorithms in Python, I decided to try and port across the code for at least the few basic algorithms into Mojo. A systems programming language worked on by the team who developed Swift for iOS and the LLVM compiler. I am not the person to ask about why the LLVM compiler is awesome but all I know is it is heavily leant on. Maybe one day I’ll know enough about it to understand.

This is what Claude-3.5 had to say about it. I have no opinion on it but if it’s your jam here we go. Chris Lattner is a weapon and seriously impactful figure in the world of Apple (and Google’s) programming presence:

I will simply start with the basic but with an eye on standardising what I can for efficiency, performance and minimal code. Who knows what that looks like but it shall be attempted.
A. Github Repository for the Python Versions:
patrickloeber/MLfromscratch: Machine Learning algorithm implementations from scratch.
B. Mojo and Max:
MAX: Modular Accelerated Xecution Platform

LLVM 🤷‍♂️

# The LLVM (Low Level Virtual Machine) compiler is a modular and reusable collection of compiler and toolchain technologies. Its modern, SSA-based compilation strategy supports both static and dynamic compilation of multiple programming languages. LLVM’s architecture separates the front-end (language-specific parsing and semantic analysis) from the back-end (code generation and optimization), allowing for language-agnostic optimizations and code generation. This flexibility makes it easier to create new programming languages, optimize existing ones, and target multiple hardware architectures without rewriting the entire compiler. LLVM is widely used in industry and academia, powering compilers for languages like C, C++, Rust, Swift, and many others, as well as being used in various tools for program analysis, JIT compilation, and hardware design.

Algorithms:

1. Linear Regression: The determinant of what is often time series data moving toward the future. Either univariate or multivariate analysis of the form y = ax + b where a is the slope of the line. The differentiation process of the linear regression algorithm is a simple way of explaining the way In which OLS (Ordinary Least Squres) and Stochastic Gradient Descent work to decrease the error based on the data trained on:
2. Work out where the sum of errors at each point on the proposed function (a straight line) – the gap between the data points and the line are minimised.
3. Continue until this error is minimised.
4. ???
5. Profit! Ta da, learning!
6. Logistic Regression: Despite its misleading name it is a binary classifier, that is to say it outputs two options. It can be used for a rudimentary spam filter, hot dog, not hotdog, threat or not a threat. I’m not completely across the mathematics in which the decision boundary exists but I know that the Sigmoid function (it looks like an S and where x >=0 it quickly turns into a secondary function, and as with all machine learning the sum is taken across the datapoints and the max likelihood is determined. It is not too dissimilar to the softmax function which is used in multi-classification Machine Learning problems.
7. Decision Trees: Decision Trees and their composite cousins “Random Forests” are as described. Branches are taken at each step based on the mathematical operations applied to a “feature” (column) either numerical or categorical, though it is often best practice to encode the categorical variables. From a cursory look, the mathematical formulation evaluates each feature, at each branching to maximise the entropy value aka. Information gained at every step. For classification purposes the tree is traversed and the final node is the determinant of the class in which the dataset value belong.
8. A Random Forest as a not dissimilar and related algorithm is a multitude of decision trees in which the strongest candidate is identified as the most likely to represent the data without over fitting specifically to the training data.
9. Regularisation techniques will help along the way to minimise over fitting as well as hyperparmeter tuning based on the outputs of.

Mojo and Systems Programming:

Mojo is by a group called Modular who also happen to have as their CEO, the creator of the Swift programming language and the LLVM compiler aforementioned. This guy knows his stuff. Although I am attracted to continuing my Rust learning journey, his credibility and their efforts to integrate it tightly with the Python ecosystem makes it a logical choice given that the primitives and more complex memory management, and variable types are shared by the Rust Ecosystem. I don’t think I will get too far behind on my understanding by applying my nascent abilities to a language which is attempting to address the same issues but using a familiar syntax to many and a super set of the most popular language in the world.

What will I be using:

1. SIMD: Single instruction, multiple data: applying a data transformation, or calculation such as has a MatMul (Matrix Multiplication) in a vectorised highly efficient manner. Picture it as painting a glass fall of marbles. Instead of pulling one out, painting it, and putting it back. You can paint a whole lot of marbles (The SIMD width in float32 precision) in parallel.
2. This same vectorisation and functionality of the language also means that other operations can be performed in parallel and with memory safety.
3. Data structures such as Dynamic Vectors rather than your traditional Python Lists are high performance and memory safe.
4. Functions like parallel exist within the Mojo language for multi threading.
5. It is an important distinction but unlike Python which is single threaded (one lone worker bee) and is interpreted at runtime, Mojo is a fully compiled language in which you can use Python libraries as part of your workflow.

As mentioned, Chris Lattner of Swift and LLVM fame’s involvement in this young language ecosystem combined with its python like syntax makes it attractive. I don’t feel like I’m going backwards in my systems programming understanding but rather combining my understanding of Machine Learning to Systems Programming through the Mojo API.

Beyond Mojo there is the Directed Acyclic Graph pipelines that can be built using Modular Max. A world I haven’t even begun to conceptualise.

All the fun! Wish me lucky and have the fun!

— Snyata

• Mail