So you want to learn Machine Learning? It will be a long journey - one that requires a solid grasp of the fundamentals. Try and not skip any of the stages, and move on to the next once you have a full understanding of the current one. Good luck!
This is what essentially provides the mathematical framework for understanding and manipulating vectors and matrices, which are the building blocks of almost any ML algorithm. A full grasp of these concepts is essential. As always, Khan Academy is a great resource. Below are a list of the essentials, along with the appropriate Khan Academy course materials. You can always choose your own course if you wish to.
Calculus, and particularly derivatives and gradients, play a key role in optimization algorithms used in ML. You will rely on Calculus for optimization techniques such as gradient descent, and key components of DL such as Backpropagation - which heavily relies on the chain rule of calculus. Studying integrals and derivatives are fundamental for understanding the rate of change in functions and how they behave, which is crucial for interpreting and modeling real-world use-cases.
- Differential Equations
- Application of Integrals
- Parametric equations, polar coordinates, and vector-valued functions
Another essential building-block. Probability theory provides a math framework for quantifying uncertainty. In ML, models often need to make predictions or decisions based on incomplete or noisy data. With probability, we can easily express that uncertainty and reason about it. There's myriad other reasons for learning probability, of course; just keep in mind that Language Models generate text by taking your input and calculating the probability distribution of the next sequence of words that would follow it, and pick the most likely output to complete your input text.
- The Entire Khan Academy Statistics and Probability course
You can take only the lessons you think might be important and then take the Course Challenge.
- Discrete and continuous probability distributions: binomial and normal (Gaussian).
- Bayesian Statistics
That should probably be enough for Math. I might've missed a few essential things, but I will add more as I come across them.
The current programming language dominating the ML community is Python. Not surprising, since the ease of use allows you to focus on writing efficient code without needing to spend too much time learning the intricacies of the language's syntax. There's a good chance you already know Python, but we'll go over the basic steps anyway.
roadmap.sh Python Developer roadmap is an incredibly useful resource for this. What it essentially boils down to, however, is:
- Learn the Basic Syntax and Data Types
You'll need to familiarize yourself with Python's syntax, variables, data types (integers, floats, strings, lists, dicts), and basic operations (arithmetic, string manipulation, indexing, slicing).
- Control Flow
Understand conditional statements (
else), loops (
while), and logical operators (
not). Very important for implementing decision-making and repetition in your code.
- Functions and modules
Learn how to define and use functions to encapsulate reusable blocks of code. Also, you'll need to understand how to import and utilize modules (libs).
- Data Structures and Manipulation
Get yourself acquainted with fundamental data structures like lists, tuples, sets, and dictionaries. Learn how to manipulate and transform data.
A fundamental library for scientific computing in Python. You will need to gain proficiency in using NumPy arrays for efficient numerical computations.
You will often need Pandas DataFrames to clean, transform, filter, aggregate, and analyze your datasets.
- Plotting and Data Visualization
Become familiar with libraries such as Matplotlib and Seaborn for creating plots, charts, and visualization. Not strictly necessary, but recommended.
At this stage, you'll be sufficiently familiar with Python and ready to tackle the ML aspects of Python. Very exciting.
- Machine Learning Libraries
Explore the popular ML libraries, such as PyTorch, TensorFlow, or scikit-learn. It's recommended to focus on only one, and as of now, PyTorch is the most popular.
- Object-Oriented Programming (OOP)
Get yourself comfortable with the principles of OOP, including classes, objects, inheritance, and encapsulation. Allows for modular and organized code design.
At this point, you can follow whatever ML course you're comfortable with. A popular recommendation is fastai. It's a great resource for almost everything you'll need to learn about ML. Otherwise, pick any of the concepts below that interest you and get to learning.
- Supervised Learning
This Coursera curriculum on Supervised ML will be useful. As of writing this guide, the course is completely free. The main points to learn are:
- Classification: Predicting discrete class labels.
- Regression: Predicting continuous values.
- Unsupervised Learning
This course should provide the adequate amount of knowledge on Unsupervised Learning. The main points to learn are:
- Clustering: Grouping similar data points together.
- Dimensionality Reduction: Reducing the number of input features while preserving important information.
- Anomaly Detection: Identifying rare of abnormal instances in the data.
- Reinforcement Learning
Coursera provides this course on Reinforcement Learning, which should be a good starting point.
- Linear Regression
This resource should be a useful starting point. The main outtakes are:
- Understanding linear regression models and assumptions.
- Cost functions, including mean squared error.
- Gradient descent for parameter optimization.
- Evaluation metrics for regression models.
- Logistic Regression
Read through this resource as a starting point. Main outtakes will be:
- Modeling binary classification problems with logistic regression.
- Sigmoid function and interpretation of probabilities.
- Maximum likelihood estimation and logistic loss.
- Regularization techniques for logistic regression.
- Decision Trees and Random Forests
This incredible resource by Jake VanderPlas should be extremely useful. Main outtake are:
- Basics of decision tree learning.
- Splitting criteria and handling categorical variables.
- Ensemble learning with random forests.
- Feature importance and tree visualization.
- Support Vector Machines (SVM)
Read through this resource. The main outtakes are:
- Formulation of SVMs for binary classification.
- Kernel trick and non-linear decision boundaries.
- Soft margin and regularization in SVMs.
- SVMs for multi-class classification.
Read through this excellent Google for Developers course on Clustering. The main outtakes are:
- Overview of unsupervised learning and clustering.
- K-means clustering algorithm and initialization methods.
- Hierarchical clustering and density-based clustering.
- Evaluating clustering performance.
- Neural Networks and Deep Learning
The heart of the matter. Read through the papers for each:
- Evaluation and Validation
Read the following papers:
- Feature Engineering and Dimensionality Reduction
Take a look at this article for a general oveeview.
Also read these papers:
- Model Selection and Hyperparameter Tuning
This is where you're finally dabbling in model training. Good job! You will need to learn:
- Grid search, random search, and Bayesian optimization for hyperparameter tuning.
- Model selection techniques, including nested cross-validation.
- Overfitting, underfitting, and bias-variance tradeoff.
- Performance comparison of different models.
You're now ready to pre-train your own model, or fine-tune an existing one! For this, you should look into Transformers, a framework for developing state-of-the-art Machine Learning models using PyTorch.
The HuggingFace Transformers docs go into excruciating detail on how to use Transformers. Make sure to read them, as they might be all you'll need. Good luck on your journey!
At this point, you know all the essentials. ML is an ever-advancing field, with new innovations emerging everyday. You'll need to stay abreast of the latest developments, hound arXiv for the latest published ML papers, and attend conferences if you're able to.