Contents

Python NumPy For Your Grandma - 3.1 Broadcasting

Of all the challenging concepts in NumPy, the one that’s most crucial to wrap your head around is probably broadcasting. In this video, we’ll see how broadcasting works and why it’s useful.

So, what is broadcasting? Well, we’ve already seen it in action although you might not have picked up on it. When we add a scalar to a 1d array like this, the scalar gets added to each element of the array.

import numpy as np
np.array([1,2,3]) + 0.5
## array([1.5, 2.5, 3.5])

In essence, NumPy is expanding the scalar into 3-element array and then doing element-wise addition between the arrays. Of course under the hood, NumPy doesn’t actually do this because it’d be horribly inefficient, but in essence that’s what’s happening, and that’s an example of broadcasting.

In two dimensions, you could imagine we have a matrix of values, foo, like this

foo = np.array([
    [1,1,1],
    [2,2,2]
])

to which we’ll add a 1d array, boo, with the same number of elements as our matrix has columns.

boo = np.array([1,0,-1])
foo + boo
## array([[2, 1, 0],
##        [3, 2, 1]])

In this case, NumPy carries out the addition as if it first copies boo along a new vertical axis to match the shape of foo, and then does normal element-wise addition.

Now, what if boo had two elements.. perhaps NumPy would add boo to each column of foo

foo + boo[:2]  # error

Nope. In this case we get an error. So how does broadcasting work and when can we use it?

Formally speaking, suppose we want to add two arrays, A and B.

  1. Moving backwards from the last dimension of each array, we check if their dimensions are compatible. Dimensions are compatible if they are equal or either of them is 1.
  2. If all of A’s dimensions are compatible with B’s dimensions, or vice versa, they are compatible arrays.

Informally speaking, I think it really helps to visualize the arrays you’re working with and how they’d expand for element-wise operations. Let’s see some examples.

np.random.seed(1234)

A = np.random.randint(low = 1, high = 10, size = (3, 4))
print(A)
## [[4 7 6 5]
##  [9 2 8 7]
##  [9 1 6 1]]
B = np.random.randint(low = 1, high = 10, size = (3, 1))
print(B)
## [[7]
##  [3]
##  [1]]

What do you think A + B will return?

A + B
## array([[11, 14, 13, 12],
##        [12,  5, 11, 10],
##        [10,  2,  7,  2]])

Here, A is a 3x4 array and B is a 3x1 array. We start by comparing the last dimension of each array. Since the last dimension of A is 4 and the last dimension of B is 1, NumPy can expand B by making 4 copies of it along its second axis. So, these dimensions are compatible. Now we have to compare the 1st dimension of A and B. Since they’re both 3, they’re compatible. The only thing left for NumPy is to carry out whatever procedure we wanted on two equivalently sized 3x4 arrays. (Remember, NumPy doesn’t actually expand B like this because it’d be horribly inefficient.)

Let’s see another example.

np.random.seed(4321)

A = np.random.randint(low = 1, high = 10, size = (4, 4))
print(A)
## [[3 9 3 2]
##  [8 6 3 5]
##  [7 1 9 7]
##  [6 4 2 2]]
B = np.random.randint(low = 1, high = 10, size = (2, 1))
print(B)
## [[7]
##  [2]]

Again, what do you think A + B will return?

A + B  # ValueError: operands could not be broadcast together with shapes (4,4) (2,1) 

Here, A is a 4x4 array and B is a 2x1 array. The last dimension of A is 4 and the last dimension of B is 1, so these dimensions are compatible, and just like the last example we can temporarily transform B by making 4 copies of it along its 2nd axis. Now we compare the 1st dimension of each array. In this case, there isn’t an obvious way to expand B into a 4x4 array to match A or vice versa, so these arrays are not compatible.

Let’s see another example where A has 3 dimensions and B has 2 dimensions.

np.random.seed(1111)

A = np.random.randint(low = 1, high = 10, size = (3, 1, 4))
print(A)
## [[[8 6 2 3]]
## 
##  [[5 9 7 5]]
## 
##  [[9 7 3 7]]]
B = np.random.randint(low = 1, high = 10, size = (2, 1))
print(B)
## [[9]
##  [4]]

Once again, what do you think A + B will return?

A + B
## array([[[17, 15, 11, 12],
##         [12, 10,  6,  7]],
## 
##        [[14, 18, 16, 14],
##         [ 9, 13, 11,  9]],
## 
##        [[18, 16, 12, 16],
##         [13, 11,  7, 11]]])

As before, we start by comparing the last dimension of each array. In this case, A is 4 and B is 1, so we can expand B into a 2x4 array, making these dimensions compatible. Next, we compare the 2nd to last dimension of each array. In this case, A is 1 and B is 2. This time, we expand A, copying it twice along its second axis to match B. At this point, we’re out of B dimensions, so we know A and B are compatible. To complete our mental model of how math between these arrays would work, we can imagine copying B three times along a newly added 1st dimension. We’re left with two transformed arrays, each with shape 3x2x4, which we can easily add or subtract, or combine in some other way.

Before I close out this lecture, I just want to say that when I do this sort of broadcasting expansion process in my head, I don’t actually go through this formal process, thinking of the shape tuples. I kind of just visualize the shape of these arrays and I visualize how they’d expand into identical sizes.


Course Curriculum

  1. Introduction
    1.1 Introduction
  2. Basic Array Stuff
    2.1 NumPy Array Motivation
    2.2 NumPy Array Basics
    2.3 Creating NumPy Arrays
    2.4 Indexing 1-D Arrays
    2.5 Indexing Multidimensional Arrays
    2.6 Basic Math On Arrays
    2.7 Challenge: High School Reunion
    2.8 Challenge: Gold Miner
    2.9 Challenge: Chic-fil-A
  3. Intermediate Array Stuff
    3.1 Broadcasting
    3.2 newaxis
    3.3 reshape()
    3.4 Boolean Indexing
    3.5 nan
    3.6 infinity
    3.7 random
    3.8 Challenge: Love Distance
    3.9 Challenge: Professor Prick
    3.10 Challenge: Psycho Parent
  4. Common Operations
    4.1 where()
    4.2 Math Functions
    4.3 all() and any()
    4.4 concatenate()
    4.5 Stacking
    4.6 Sorting
    4.7 unique()
    4.8 Challenge: Movie Ratings
    4.9 Challenge: Big Fish
    4.10 Challenge: Taco Truck
  5. Advanced Array Stuff
    5.1 Advanced Array Indexing
    5.2 View vs Copy
    5.3 Challenge: Population Verification
    5.4 Challenge: Prime Locations
    5.5 Challenge: The Game of Doors
    5.6 Challenge: Peanut Butter
  6. Final Boss
    6.1 as_strided()
    6.2 einsum()
    6.3 Challenge: One-Hot-Encoding
    6.4 Challenge: Cumulative Rainfall
    6.5 Challenge: Table Tennis
    6.6 Challenge: Where’s Waldo
    6.7 Challenge: Outer Product

Additional Content

  1. Python Pandas For Your Grandpa
  2. Neural Networks For Your Dog
  3. Introduction To Google Colab