Laura Diane Hamilton

Technical Product Manager at Groupon


Tutorial: Linear Algebra with Octave

Octave is a good language for rapid prototyping of machine learning algorithms. In particular, it has good support for the linear algebra operations that are so crucial in machine learning. Best of all, it's open source.

I've written a tutorial on how to get started doing basic linear algebra operations with Octave. This tutorial assumes a basic familiarity with linear algebra (matrix multiplication, matrix inversion, etc.)

First, download Octave. Then, run octave by typing this in a terminal window: octave Verify that you are using the latest version. (At the time of this blog post, the latest stable version is 3.6.4.)

First, the print command: printf "Hello World\n"

Next, variable assignment: x = 7
y = x
If you don't want it to print stuff in console, just put a semicolon at the end of the line. (You probably won't want to print a 500,000-row matrix to console.) x = 7; % This won't print anything

Checking if two values are equal: 1 == 1 % This is true
1 == 2 % And this is false
Note that Octave will return 1 for "true" and 0 for "false."

And here's how to write a for loop: for i = 1:10

Create a vector with three elements: my_vector = [1 2 3]

Or if we wanted a vector of all zeros: zeros_vector = zeros(1,7) Or a vector of all ones? ones_vector = ones(1,7)

Transpose a vector with an apostrophe: ones_vector'

You can create a matrix manually like this: A = [1 2 3; 4 5 6; 7 8 9] By convention, matrices get capital letters. This helps distinguish them from vectors and scalars in your code (which should be named with lowercase letters or words).

Transposing a matrix works the same way as transposing a vector; just add an apostrophe. A'

If you want to generate a matrix with less typing, you can create a magic matrix like so: B = magic(5) % Creates a magic 5x5 matrix The magic matrix is so named because all its sums are equal. The sum of the diagonals is equal to the sum of each of the columns, which is equal to the sum of all the rows. To wit: trace(B) % Calculate the sum of the diagonals, or trace
sum(B) % Add up all the columns
sum(B') % Add up all the rows
The magic matrix specifically isn't used for much, but it's handy to be able to create matrices quickly so that you can test out various matrix operations.

You can generate the identity matrix of arbitrary size with the "eye" command: I = eye(7)

Now, I will explain how to change individual elements of a matrix: C = ones(5,5) % Create a 5x5 matrix of all ones
C(1,1) = 7 % Change the element in the 1st row, 1st column to 7
C(1,2) = 5 % Change the element in the 1st row, 2nd column to 5
C(5,1) = 8 % Change the element in the 5th row, 1st column to 8
C(:,1) = 0 % Change the whole first column to zeros
C(2,:) = 3 % Change the whole second row to threes
C(:,3:end) = 17 % Change columns 3 and higher to 17
C(:,:) = 42 % Change all the elements to 42

Multiplying a matrix by a scalar: D = ones(4)
D * 2
2 * D % Doesn't matter if you put the scalar before or after the matrix

Multiplying a matrix by a vector: e = [2 2 2 2]
D * e % This won't work
% We can't multiply a 4x4 matrix with a 1x4 vector
e * D % Now we're multiplying a 1x4 with a 4x4, which is allowed
D * e' % Alternatively, we could multiply D by e transpose
Be really careful about the order in which you're multiplying matrices or vectors, because the order matters! Also, be careful to keep your dimensions straight. Otherwise, you may end up getting either the "noncomformant arguments" error or getting a result you do not want!

Often, in a machine learning algorithm you will want to use elementwise multiplication rather than matrix multiplication. Use a period before the asterisk to denote elementwise multiplication. See the difference: e * D % Gives us a 1x4 vector of eights
e .* D % Multiplies each element of D by 2, returning a 4x4 matrix

Exponentiation: F = ones(4,4) * 3 % Generating a 4x4 matrix of all threes
F ^2 % This gives us F * F (matrix multiplication)
F .^ 2 % This simply squares each element of F, returning a 4x4 matrix of all nines

And now for matrix inversion (only square matrices can be inverted): G = magic(5) % Create a magic 5x5 matrix
H = inv(G) % Set H as the inverse of G
G * H % When we multiply G and its inverse, we get a matrix very close to the identity matrix (with a bit of rounding error)
Not all matrices are invertible. (Non-invertible matrices are sometimes called "singular.") You can also use the pinv command, which tries to come up with a sensible "inverse" even if your matrix is technically singular.

Finally, to exit octave: quit

Those are the basic operations you need to know to begin writing machine learning algorithms in Octave. I hope this tutorial was helpful!

Next, you may be interested in learning how to do a linear regression in octave. is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to