Machine Learning From Scratch

(Few visualisations are limited to 2D data only, others can be used for any dimenntions)

View Github page

Table of contents

1. Logistic Regression

Code ans examples are here

from LogisticRegression import LR # given code
clf = LR(X,y,alpha=0.003,polyfit=True,degree=5,lambd=2)

for i in range(100):,y,itr=10)
W,b =clf.getWeight()

2. Deep Neural Network - Deeplearning

Code and examples are here

Full detail of implementation and use of code is describe here


### Download (right click and ‘save link as’):

3. Neural Network (simple structure-fully connected) with any number of layers (Matlab/Octave)

Code and examples here

Network can be created and trained as for example

W= NeuralNet(X,y,HL,Iterations,alpha,verbose);

% For 2 hidden layers with 5 and 3 neurons, 500 iteration and 0.1 alpha(learning rate)
% input and output layers are chosen according to data X,y provided

W= NeuralNet(X,y,[5,3],500,0.1,1); 

% for 8 hidden layers
W= NeuralNet(X,y,[15,10,10,10,5,5,5,3],100,0.1,1);

returns weights W of each layer

4. Kernel Learning & regularization


Kernal Learning

(Linear, Polynomial, Gaussian)

Code and examples with GUI are given here

5 Naive Bayes

Probabilistic model

Classifier based on Bayes rule:

Example with jupyter notebook here and Repository

Notebook include example of Iris data, Breast Cancer and Digit classification (MNIST)


                     Class file

here is code snippet

import numpy as np
import matplotlib.pyplot as plt

# For dataset
from sklearn import datasets
from sklearn.model_selection import train_test_split

# Library provided
from probabilistic import NaiveBayes

data = datasets.load_iris()
X =
y =

Xt,Xs,yt,ys = train_test_split(X,y,test_size=0.3)


# Fitting model (estimating the parameters)
clf = NaiveBayes(),yt)

# Prediction
ytp = clf.predict(Xt)
ysp = clf.predict(Xs)

print('Training Accuracy : ',np.mean(ytp==yt))
print('Testing  Accuracy : ',np.mean(ysp==ys))


# Visualization
fig = plt.figure(figsize=(12,10))

6 Decision Trees

Classification and Regression Tree

Requirement: All you need for this is Numpy and matplotlib** (Of course Python >=3.0)

See the examples in Jupyter-Notebook or Repository for more details




import numpy as np
import matplotlib.pyplot as plt

# Download and keep in current directory or give a path (if you know how to)
from trees import ClassificationTree, RegressionTree

# For examples
from sklearn import datasets
from sklearn.model_selection import train_test_split

Iris Data

data = datasets.load_iris()
X =
y =

feature_names = data.feature_names #Optional
Xt,Xs, yt, ys = train_test_split(X,y,test_size=0.3)

Initiate the classifier and train it

clf = ClassificationTree()

# verbose 0 for no progress, 1 for short and 2 for detailed.
# feature_names is you know, else leave it or set it to None,yt,verbose=2,feature_names=feature_names)  

Plot the decision tree

# Plot Tree that has been learned

Visualizing the tree building while training

Classification: Iris Data, Breast cancer Data Regression::Bostan House price Data

Visualization of decision tree after fitting a model

Option to show colored branch: Blue for True and Red for False Or just show all branches as blue with direction to indicate True and False branch

Iris data: Decesion Tree | Cancer data: Decesion Tree


Boston data: Decesion Tree

Visualizing the progress of tree building while training

Tree building for Cancer Data (Classification)

Detailed view

Short view

Follow @nikeshbajaj