First 5 things to start learning PyTorch Tensors in Sagemaker Notebooks

Let’s begin our TorchAdventure in AWS! with this 11 basic functions distributed in the following sections:

Creating tensors: Empty() and Zeros()
Measuring tensors: Size() and Shape(), Sum() and Dimensions
Changing and copying tensors: Reshape(), view() and randn()
Modifying tensors: Unsqueeze()
Comparing tensors: Element-Wise Equality: Eq

First lets spin up a new sagemaker instance:

Now go to Jupyter Lab and import or create a new Notebook with this kernel for Pytorch:

Select the appropiate kernel, for this case it could be: conda_pytorch_p36

Here is where I will start from scratch.

Before we begin, let’s install and import PyTorch

*# Uncomment and run the appropriate command for your operating system, if required*

# Linux / Binder
# !pip install numpy torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

# Windows
# !pip install numpy torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

# MacOS
# !pip install numpy torch torchvision torchaudio

Import Pytorch from the Notebook instance

**import** **torch**

Function 1 — Empty and Zeros — how to initialize tensors

We need to start working with the basics pytorch functions, and the first thing is create our matrix

*# Creates a 3 x 2 matrix which is empty*
a = torch.empty(3, 2)
print(a)

tensor([[1.5842e-35, 0.0000e+00],
        [4.4842e-44, 0.0000e+00],
        [       nan, 0.0000e+00]])

Here is how PyTorch is allocating memory for this tensor. Whatever, it will not erase anything previous content in the memory.

by default, when you initializes a tensor is used the float32 dtype. you can review it here: https://pytorch.org/docs/stable/generated/torch.set_default_tensor_type.html#torch.set_default_tensor_type

But you can also start working with torch.zeros

*# Applying the zeros function and *
*# storing the resulting tensor*

a = torch.zeros([3, 5]) 
print("a = ", a) 

b = torch.zeros([2, 4]) 
print("b = ", b) 

c = torch.zeros([4, 1]) 
print("c = ", c) 

d = torch.zeros([4, 4, 2]) 
print("d = ", d)

the result will be:

a =  tensor([[0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0.]])
b =  tensor([[0., 0., 0., 0.],
        [0., 0., 0., 0.]])
c =  tensor([[0.],
        [0.],
        [0.],
        [0.]])
d =  tensor([[[0., 0.],
         [0., 0.],
         [0., 0.],
         [0., 0.]],

[[0., 0.],
         [0., 0.],
         [0., 0.],
         [0., 0.]],

[[0., 0.],
         [0., 0.],
         [0., 0.],
         [0., 0.]],

[[0., 0.],
         [0., 0.],
         [0., 0.],
         [0., 0.]]])

this tensor is filled with zeros, so PyTorch allocates memory and zero-initializes the tensor elements inside

You cannot change the way a tensor is created, if you create a zeros tensor, make sure is not referenced to any other.

*# correctly initialized*
a = torch.empty(3,3)
print(a)

#also correct
b = torch.empty(1,2,3)

print(b)

the results will be:

tensor([[2.4258e-35, 0.0000e+00, 1.5975e-43],
        [1.3873e-43, 1.4574e-43, 6.4460e-44],
        [1.4153e-43, 1.5274e-43, 1.5695e-43]])
tensor([[[2.3564e-35, 0.0000e+00, 1.4013e-45],
         [1.4574e-43, 6.4460e-44, 1.4153e-43]]])

Lets see now, how you cannot use the zeros function:

*# incorrect reference, you must create a new one*
c = torch.zeros(b,1) 
print("c = ", c)

output:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-35-c044e2995879> in <module>()
**      1** # incorrect reference, you must create a new one
----> 2 c = torch.zeros(b,1)
**      3** print("c = ", c)

TypeError: zeros() received an invalid combination of arguments - got (Tensor, int), but expected one of:
 * (tuple of ints size, *, tuple of names names, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
 * (tuple of ints size, *, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)

lets review the zero method closely:

Our documentation says:

Syntax: torch.zeros(size, out=None)

Parameters:
size: a sequence of integers defining the shape of the output tensor
out (Tensor, optional): the output tensor

Return type: A tensor filled with scalar value 0, of same shape as size.

torch.zeros and torch.empty are the first functions to start working with pytorch tensors and learning a little bit of matrix and vectors

Function 2 — Tensor Size, Shape and Dimension Operations

Lets understand dimensions in Pytorch.

Now lets create some tensors and determine the size of every one

**import** **torch** 
*# Create a tensor from data*
c = torch.tensor([[3.2 , 1.6, 2], [1.3, 2.5 , 6.9]])
print(c)

print(c.size())

output

tensor([[3.2000, 1.6000, 2.0000],
        [1.3000, 2.5000, 6.9000]])
torch.Size([2, 3])

Lets see torch.shape and take a closer look at how size is given here

Now in the next example lets use shape functions to get the size of a tensor

In [ ]:

x = torch.tensor([
     [1, 2, 3],
     [4, 5, 6]
   ])
x.shape
torch.Size([2, 3])

Out[ ]:

torch.Size([2, 3])

Shape and Size give us the same correct dimensions of the tensor.

in this case we have a 3D-tensor (with 3 dimensions)

Dimension 0 Dimension 1 and Dimension 2

lets create a new tensor:

In [ ]:

y = torch.tensor([
     [
       [1, 2, 3],
       [4, 5, 6]
     ],
     [
       [1, 2, 3],
       [4, 5, 6]
     ],
     [
       [1, 2, 3],
       [4, 5, 6]
     ]
   ])

y.shape

Out[ ]:

torch.Size([3, 2, 3])

Lets se how we can make operations using a 3d tensor now for every dimension layer and see how it behaves

In [ ]:

sum1 = torch.sum(y, dim=0)
print(sum1)

sum2 = torch.sum(y, dim=1)
print(sum2)

sum3 = torch.sum(y, dim=2)
print(sum3)

tensor([[ 3,  6,  9],
        [12, 15, 18]])
tensor([[5, 7, 9],
        [5, 7, 9],
        [5, 7, 9]])
tensor([[ 6, 15],
        [ 6, 15],
        [ 6, 15]])

we can see now a 3d tensor is more complicated as we advance, and we can perform custom operations within every dimension

its limited right now to 3 dims so we cannot perform this:

In []

sum2 = torch.sum(y, dim=3)
print(sum1)

Out [ ]:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-56-7caf723c6102> in <module>()
----> 1 sum2 = torch.sum(y, dim=3)
**      2** print(sum1)

IndexError: Dimension out of range (expected to be in range of [-3, 2], but got 3)

to accomplish adding more layers of dimensions we can review the unsqueeze functions → .unsqueeze() but for now let’s go to the next basic function

Function 3 — Reshape, View and Random

There is a function in NunPy called ndarray.reshape() for reshaping an array.

Now in pytorch, there is torch.view(tensor) for the same purpose, but at the same time, there is also a torch.reshape(tensor).

let’s figure out the differences between them and when you should use either of them.

First of all we are going to use a new functions for randomize our tensor.

In [ ]:

**import** **torch** 
x = torch.randn(5, 3)

print(x)

Out []

tensor([[-0.5793,  0.6999,  1.7417],
        [-0.9810,  0.0626,  0.4100],
        [-0.6519, -0.0595, -1.2156],
        [-0.3973, -0.3103,  1.6253],
        [ 0.2775, -0.0045, -0.2985]])

this is basic usage of torch.randm, so now lets use View from another variable “y” and describe all elements

In [ ]:

*# Return a view of the x, but only having *
*# one dimension and max number of elements*
y = x.view(5 * 3)

*#lets see the size of every tensor*
print("lets see the size of every tensor")
print('Size of x:', x.size())
print('Size of y:', y.size())

*#and the elements of very tensor to compare*
print("and the elements of very tensor to compare")
print("X:", x)
print("Y:", y)

Out []

lets see the size of every tensor
Size of x: torch.Size([5, 3])
Size of y: torch.Size([15])
and the elements of very tensor to compare
X: tensor([[-0.5793,  0.6999,  1.7417],
        [-0.9810,  0.0626,  0.4100],
        [-0.6519, -0.0595, -1.2156],
        [-0.3973, -0.3103,  1.6253],
        [ 0.2775, -0.0045, -0.2985]])
Y: tensor([-0.5793,  0.6999,  1.7417, -0.9810,  0.0626,  0.4100, -0.6519, -0.0595,
        -1.2156, -0.3973, -0.3103,  1.6253,  0.2775, -0.0045, -0.2985])

take a look at Y tensor, it only has 1 dimension. so Viewing another tensor may be difficult for some operations

Now lets use Reshape to replicate the exact dimensions of X

In [ ]:

*# Get back the original tensor with reshape()*
z = y.reshape(5, 3)
print(z)

tensor([[-0.2927,  0.0329,  0.8485],
        [ 1.9581,  0.8313, -0.1529],
        [-0.2330, -0.1887,  1.8206],
        [ 1.5252,  1.0909,  0.0547],
        [-0.1231, -0.4238, -0.6724]])

we cannot only reshape the original, we can also change the dimensions with some limited actions related to the maximum elements:

first lets reshape to 1 more dimension

In [ ]:

*# Get back the original tensor with reshape()*
z = y.reshape(15)
print(z)

#reshaping 15 elements, 1 dim
z = y.reshape(3*5)
print(z)

#reshaping in different order, 3 dimensions
z = y.reshape(3,5)
print(z)

#Reshaping with more dimensions but its 15 elements always
z = y.reshape(3,5,1)
print(z)

tensor([-0.5793,  0.6999,  1.7417, -0.9810,  0.0626,  0.4100, -0.6519, -0.0595,
        -1.2156, -0.3973, -0.3103,  1.6253,  0.2775, -0.0045, -0.2985])
tensor([-0.5793,  0.6999,  1.7417, -0.9810,  0.0626,  0.4100, -0.6519, -0.0595,
        -1.2156, -0.3973, -0.3103,  1.6253,  0.2775, -0.0045, -0.2985])
tensor([[-0.5793,  0.6999,  1.7417, -0.9810,  0.0626],
        [ 0.4100, -0.6519, -0.0595, -1.2156, -0.3973],
        [-0.3103,  1.6253,  0.2775, -0.0045, -0.2985]])
tensor([[[-0.5793],
         [ 0.6999],
         [ 1.7417],
         [-0.9810],
         [ 0.0626]],

        [[ 0.4100],
         [-0.6519],
         [-0.0595],
         [-1.2156],
         [-0.3973]],

        [[-0.3103],
         [ 1.6253],
         [ 0.2775],
         [-0.0045],
         [-0.2985]]])

Now lets reshape exceeding the number of elements in the tensor:

In [ ]:

z = y.reshape(16)
print(z)

Out []

---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-78-c7ae174fce73> in <module>()
----> 1 z = y.reshape(16)
**      2** print(z)

RuntimeError: shape '[16]' is invalid for input of size 15

it will fail also for z = y.reshape(3*6) or putting more elements that does not exist in the tensor.

Now let’s keep going to the next section.

Function 4 — Unsqueeze()

Mainly, it allows us to add more dimensions at specific index you define.

lets take a look:

In [ ]:

**import** **torch**

#dim=1, that is (3)
x = torch.tensor([1, 2, 3])
print('x: ', x)
print('x.size: ', x.size())

#x1 becomes a matrix of (3,1)
x1 = torch.unsqueeze(x, 1)
print('x1: ', x1)
print('x1.size: ', x1.size())

Out []

x:  tensor([1, 2, 3])
x.size:  torch.Size([3])
x1:  tensor([[1],
        [2],
        [3]])
x1.size:  torch.Size([3, 1])
x2:  tensor([[1, 2, 3]])
x2.size:  torch.Size([1, 3])

Our initial tensor is Tensor([1,2,3]), and the output size is [3].

And then we proceed with adding 1 dimenson with unsqueeze operation, namely torch.unsqueeze(x, 1), the size of x1 is [3,1].

In [ ]:

*#x2 becomes a matrix of (1,3)*
x2 = torch.unsqueeze(x, 0)
print('x2: ', x2)
print('x2.size: ', x2.size())

Out []

x2:  tensor([[1, 2, 3]])
x2.size:  torch.Size([1, 3])

When we perform torch.unsqueeze(x, 0), the size of x2 is [1,3].

In [ ]:

*# Example 3 - breaking (to illustrate when it breaks)*


print(x.unsqueeze())

Out []

---------------------------------------------------------------------------
TypeError  Traceback (most recent call last)
<ipython-input-89-5a320a828907> in <module>()
**      2** 
**      3** 
----> 4 print(x.unsqueeze())

TypeError: unsqueeze() missing 1 required positional arguments: "dim"

We must specified the dimension correctly, although we are just adding 1 dim, it is necessary to put like this: x.unqueeze(0)

Function 5 — Torch Eq (Element Wise equality)

This function is under comparison category and it computes equality in element-wise and returns a boolean tensor. True if equal, False otherwise.

Lets review how we can operate with different sizes of tensors:

In [ ]:

*# Example 1 - working *
x1 = torch.tensor([[1, 2], [3, 4.]])
x2 = torch.tensor([[2, 2], [2, 5]])
x3 = torch.randn(3,5)

#size x1 and z2
print(x1.size())
print(x2.size())

# tensors 1 and 2
print(x1)
print(x2)


#size x3
print(x3.size())

#tensors 3
print(x3)

torch.eq(x1,x2)

torch.Size([2, 2])
torch.Size([2, 2])
tensor([[1., 2.],
        [3., 4.]])
tensor([[2, 2],
        [2, 5]])
torch.Size([3, 5])
tensor([[-1.3040, -0.4658, -0.5269,  0.7409,  0.9135],
        [ 1.0780,  2.0584, -0.9629, -1.1412, -0.3105],
        [ 0.3613, -1.4196,  2.1145,  0.3649,  0.2037]])

Out[ ]:

tensor([[False,  True],
        [False, False]])

x1 and x3 have the same size, but x3 is [3,5], has bigger size.

comparing x1 and x2 is Ok.

In [ ]:

*# Example 2 - working (with broadcasting)*
x4 = torch.tensor([[1, 2], [3, 4]])
print(x4.size())
x5 = torch.tensor([2, 5])
print(x5.size())
torch.eq(x4, x5)

torch.Size([2, 2])
torch.Size([2])

Out[ ]:

tensor([[False, False],
        [False, False]])

we can also compare, different sizes only if the seconf value, in this case x5, that has size of [2] is broadcastable with the frist one thst is [2,2]

In [ ]:

*# Example 3 - breaking (to illustrate when it breaks)*
x6 = torch.tensor([[0, 2, 4], [3, 4, 5]])
print(x6.size())
x7 = torch.tensor([[2, 3], [2, 4]])
print(x7.size())

torch.eq(x6, x3)

torch.Size([2, 3])
torch.Size([2, 2])

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-98-ac2ad1ecd5b0> in <module>()
      5 print(x7.size())
      6 
----> 7 torch.eq(x6, x3)

RuntimeError: The size of tensor a (3) must match the size of tensor b (5) at non-singleton dimension 1

finally, we cant compare different sizes if the second arguments shape is not broadcastable with the first argument.

Conclusion

we review 5 basic topics covering more than 10 PyTorch functions.

in the next post i’ll talk about Linear Regression.

Reference Links

Provide links to your references and other interesting articles about tensors

Official documentation for tensor operations: https://pytorch.org/docs/stable/torch.html
Squeeze and unsqueeze functions here: https://www.programmersought.com/article/65705814717/
Understanding dimensions in Pytorch: https://towardsdatascience.com/understanding-dimensions-in-pytorch-6edf9972d3be

#aws #reinvent2020 #awsperu #awsugperu #awscloud

Carlos Cortez — AWS UG Perú Leader / AWS ML Community Builder ccortez@aws.pe @ccortezb Podcast: imperiocloud.com @imperiocloud twitch.tv/awsugperu cennticloud.thinkific.com Carlos Cortez ML Engineer — Senior Cloud Architect — Pro Services Cluster — BGH Tech Partner |… Hello Everyone, I’m the Founder of AWS User Group Perú 🇵🇪 (2014) and also an AWS Solutions Architect Certified…www.linkedin.com

First 5 things to start learning PyTorch Tensors in Sagemaker Notebooks

Comments

More from this blog

Starting with NLP on AWS from scratch — NLP Series part 1 | by Carlos Cortez | Breaking the Cloud

Crear un API rest de Análisis de sentimiento con SAM y Amazon Comprehend | NLP Series parte 3

Ingestando noticias de mi propia web (Text Analytics basics ) — NLP Series parte 2 | por Carlos…

Hi my dear mortals

DeepFridays, Tu camino hacia el aprendizaje de AWS DeepRacer

Function 1 — Empty and Zeros — how to initialize tensors

Function 2 — Tensor Size, Shape and Dimension Operations

Function 3 — Reshape, View and Random

Function 4 — Unsqueeze()

Function 5 — Torch Eq (Element Wise equality)

Conclusion

Reference Links

Command Palette

Comments

More from this blog

Function 1 — Empty and Zeros — how to initialize tensors

Function 2 — Tensor Size, Shape and Dimension Operations

Function 3 — Reshape, View and Random

Function 4 — Unsqueeze()

Function 5 — Torch Eq (Element Wise equality)

Conclusion

Reference Links