# First 5 things to start learning PyTorch Tensors in Sagemaker Notebooks

Let’s begin our TorchAdventure in AWS! with this 11 basic functions distributed in the following sections:

**Creating tensors:**Empty() and Zeros()**Measuring tensors:**Size() and Shape(), Sum() and Dimensions**Changing and copying tensors:**Reshape(), view() and randn()**Modifying tensor**s: Unsqueeze()**Comparing tensors**: Element-Wise Equality: Eq

First lets spin up a new sagemaker instance:

Now go to Jupyter Lab and import or create a new Notebook with this kernel for Pytorch:

Select the appropiate kernel, for this case it could be: conda_pytorch_p36

Here is where I will start from scratch.

Before we begin, let’s install and import PyTorch

```
*# Uncomment and run the appropriate command for your operating system, if required*
# Linux / Binder
# !pip install numpy torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
# Windows
# !pip install numpy torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
# MacOS
# !pip install numpy torch torchvision torchaudio
```

Import Pytorch from the Notebook instance

```
**import** **torch**
```

## Function 1 — Empty and Zeros — how to initialize tensors

We need to start working with the basics pytorch functions, and the first thing is create our matrix

```
*# Creates a 3 x 2 matrix which is empty*
a = torch.empty(3, 2)
print(a)
tensor([[1.5842e-35, 0.0000e+00],
[4.4842e-44, 0.0000e+00],
[ nan, 0.0000e+00]])
```

Here is how PyTorch is allocating memory for this tensor. Whatever, it will not erase anything previous content in the memory.

by default, when you initializes a tensor is used the float32 dtype. you can review it here: https://pytorch.org/docs/stable/generated/torch.set_default_tensor_type.html#torch.set_default_tensor_type

But you can also start working with torch.zeros

```
*# Applying the zeros function and *
*# storing the resulting tensor*
a = torch.zeros([3, 5])
print("a = ", a)
b = torch.zeros([2, 4])
print("b = ", b)
c = torch.zeros([4, 1])
print("c = ", c)
d = torch.zeros([4, 4, 2])
print("d = ", d)
```

the result will be:

```
a = tensor([[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.]])
b = tensor([[0., 0., 0., 0.],
[0., 0., 0., 0.]])
c = tensor([[0.],
[0.],
[0.],
[0.]])
d = tensor([[[0., 0.],
[0., 0.],
[0., 0.],
[0., 0.]],
[[0., 0.],
[0., 0.],
[0., 0.],
[0., 0.]],
[[0., 0.],
[0., 0.],
[0., 0.],
[0., 0.]],
[[0., 0.],
[0., 0.],
[0., 0.],
[0., 0.]]])
```

this tensor is filled with zeros, so PyTorch allocates memory and zero-initializes the tensor elements inside

You cannot change the way a tensor is created, if you create a zeros tensor, make sure is not referenced to any other.

```
*# correctly initialized*
a = torch.empty(3,3)
print(a)
#also correct
b = torch.empty(1,2,3)
print(b)
```

the results will be:

```
tensor([[2.4258e-35, 0.0000e+00, 1.5975e-43],
[1.3873e-43, 1.4574e-43, 6.4460e-44],
[1.4153e-43, 1.5274e-43, 1.5695e-43]])
tensor([[[2.3564e-35, 0.0000e+00, 1.4013e-45],
[1.4574e-43, 6.4460e-44, 1.4153e-43]]])
```

Lets see now, how you cannot use the zeros function:

```
*# incorrect reference, you must create a new one*
c = torch.zeros(b,1)
print("c = ", c)
```

output:

```
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-35-c044e2995879> in <module>()
** 1** # incorrect reference, you must create a new one
----> 2 c = torch.zeros(b,1)
** 3** print("c = ", c)
TypeError: zeros() received an invalid combination of arguments - got (Tensor, int), but expected one of:
* (tuple of ints size, *, tuple of names names, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
* (tuple of ints size, *, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
```

lets review the zero method closely:

Our documentation says:

```
Syntax: torch.zeros(size, out=None)
Parameters:
size: a sequence of integers defining the shape of the output tensor
out (Tensor, optional): the output tensor
Return type: A tensor filled with scalar value 0, of same shape as size.
```

torch.zeros and torch.empty are the first functions to start working with pytorch tensors and learning a little bit of matrix and vectors

## Function 2 — Tensor Size, Shape and Dimension Operations

Lets understand dimensions in Pytorch.

Now lets create some tensors and determine the size of every one

```
**import** **torch**
*# Create a tensor from data*
c = torch.tensor([[3.2 , 1.6, 2], [1.3, 2.5 , 6.9]])
print(c)
print(c.size())
```

output

```
tensor([[3.2000, 1.6000, 2.0000],
[1.3000, 2.5000, 6.9000]])
torch.Size([2, 3])
```

Lets see torch.shape and take a closer look at how size is given here

Now in the next example lets use shape functions to get the size of a tensor

In [ ]:

```
x = torch.tensor([
[1, 2, 3],
[4, 5, 6]
])
x.shape
torch.Size([2, 3])
```

Out[ ]:

```
torch.Size([2, 3])
```

Shape and Size give us the same correct dimensions of the tensor.

in this case we have a 3D-tensor (with 3 dimensions)

Dimension 0 Dimension 1 and Dimension 2

lets create a new tensor:

In [ ]:

```
y = torch.tensor([
[
[1, 2, 3],
[4, 5, 6]
],
[
[1, 2, 3],
[4, 5, 6]
],
[
[1, 2, 3],
[4, 5, 6]
]
])
y.shape
```

Out[ ]:

```
torch.Size([3, 2, 3])
```

Lets se how we can make operations using a 3d tensor now for every dimension layer and see how it behaves

In [ ]:

```
sum1 = torch.sum(y, dim=0)
print(sum1)
sum2 = torch.sum(y, dim=1)
print(sum2)
sum3 = torch.sum(y, dim=2)
print(sum3)
tensor([[ 3, 6, 9],
[12, 15, 18]])
tensor([[5, 7, 9],
[5, 7, 9],
[5, 7, 9]])
tensor([[ 6, 15],
[ 6, 15],
[ 6, 15]])
```

we can see now a 3d tensor is more complicated as we advance, and we can perform custom operations within every dimension

its limited right now to 3 dims so we cannot perform this:

In []

```
sum2 = torch.sum(y, dim=3)
print(sum1)
```

Out [ ]:

```
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-56-7caf723c6102> in <module>()
----> 1 sum2 = torch.sum(y, dim=3)
** 2** print(sum1)
IndexError: Dimension out of range (expected to be in range of [-3, 2], but got 3)
```

to accomplish adding more layers of dimensions we can review the unsqueeze functions → .unsqueeze() but for now let’s go to the next basic function

## Function 3 — Reshape, View and Random

There is a function in NunPy called ndarray.reshape() for reshaping an array.

Now in pytorch, there is torch.view(tensor) for the same purpose, but at the same time, there is also a torch.reshape(tensor).

let’s figure out the differences between them and when you should use either of them.

First of all we are going to use a new functions for randomize our tensor.

In [ ]:

```
**import** **torch**
x = torch.randn(5, 3)
print(x)
```

Out []

```
tensor([[-0.5793, 0.6999, 1.7417],
[-0.9810, 0.0626, 0.4100],
[-0.6519, -0.0595, -1.2156],
[-0.3973, -0.3103, 1.6253],
[ 0.2775, -0.0045, -0.2985]])
```

this is basic usage of torch.randm, so now lets use View from another variable “y” and describe all elements

In [ ]:

```
*# Return a view of the x, but only having *
*# one dimension and max number of elements*
y = x.view(5 * 3)
*#lets see the size of every tensor*
print("lets see the size of every tensor")
print('Size of x:', x.size())
print('Size of y:', y.size())
*#and the elements of very tensor to compare*
print("and the elements of very tensor to compare")
print("X:", x)
print("Y:", y)
```

Out []

```
lets see the size of every tensor
Size of x: torch.Size([5, 3])
Size of y: torch.Size([15])
and the elements of very tensor to compare
X: tensor([[-0.5793, 0.6999, 1.7417],
[-0.9810, 0.0626, 0.4100],
[-0.6519, -0.0595, -1.2156],
[-0.3973, -0.3103, 1.6253],
[ 0.2775, -0.0045, -0.2985]])
Y: tensor([-0.5793, 0.6999, 1.7417, -0.9810, 0.0626, 0.4100, -0.6519, -0.0595,
-1.2156, -0.3973, -0.3103, 1.6253, 0.2775, -0.0045, -0.2985])
```

take a look at Y tensor, it only has 1 dimension. so Viewing another tensor may be difficult for some operations

Now lets use Reshape to replicate the exact dimensions of X

In [ ]:

```
*# Get back the original tensor with reshape()*
z = y.reshape(5, 3)
print(z)
tensor([[-0.2927, 0.0329, 0.8485],
[ 1.9581, 0.8313, -0.1529],
[-0.2330, -0.1887, 1.8206],
[ 1.5252, 1.0909, 0.0547],
[-0.1231, -0.4238, -0.6724]])
```

we cannot only reshape the original, we can also change the dimensions with some limited actions related to the maximum elements:

first lets reshape to 1 more dimension

In [ ]:

```
*# Get back the original tensor with reshape()*
z = y.reshape(15)
print(z)
#reshaping 15 elements, 1 dim
z = y.reshape(3*5)
print(z)
#reshaping in different order, 3 dimensions
z = y.reshape(3,5)
print(z)
#Reshaping with more dimensions but its 15 elements always
z = y.reshape(3,5,1)
print(z)
tensor([-0.5793, 0.6999, 1.7417, -0.9810, 0.0626, 0.4100, -0.6519, -0.0595,
-1.2156, -0.3973, -0.3103, 1.6253, 0.2775, -0.0045, -0.2985])
tensor([-0.5793, 0.6999, 1.7417, -0.9810, 0.0626, 0.4100, -0.6519, -0.0595,
-1.2156, -0.3973, -0.3103, 1.6253, 0.2775, -0.0045, -0.2985])
tensor([[-0.5793, 0.6999, 1.7417, -0.9810, 0.0626],
[ 0.4100, -0.6519, -0.0595, -1.2156, -0.3973],
[-0.3103, 1.6253, 0.2775, -0.0045, -0.2985]])
tensor([[[-0.5793],
[ 0.6999],
[ 1.7417],
[-0.9810],
[ 0.0626]],
[[ 0.4100],
[-0.6519],
[-0.0595],
[-1.2156],
[-0.3973]],
[[-0.3103],
[ 1.6253],
[ 0.2775],
[-0.0045],
[-0.2985]]])
```

Now lets reshape exceeding the number of elements in the tensor:

In [ ]:

```
z = y.reshape(16)
print(z)
```

Out []

```
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-78-c7ae174fce73> in <module>()
----> 1 z = y.reshape(16)
** 2** print(z)
RuntimeError: shape '[16]' is invalid for input of size 15
```

it will fail also for z = y.reshape(3*6) or putting more elements that does not exist in the tensor.

Now let’s keep going to the next section.

## Function 4 — Unsqueeze()

Mainly, it allows us to add more dimensions at specific index you define.

lets take a look:

In [ ]:

```
**import** **torch**
#dim=1, that is (3)
x = torch.tensor([1, 2, 3])
print('x: ', x)
print('x.size: ', x.size())
#x1 becomes a matrix of (3,1)
x1 = torch.unsqueeze(x, 1)
print('x1: ', x1)
print('x1.size: ', x1.size())
```

Out []

```
x: tensor([1, 2, 3])
x.size: torch.Size([3])
x1: tensor([[1],
[2],
[3]])
x1.size: torch.Size([3, 1])
x2: tensor([[1, 2, 3]])
x2.size: torch.Size([1, 3])
```

Our initial tensor is Tensor([1,2,3]), and the output size is [3].

And then we proceed with adding 1 dimenson with unsqueeze operation, namely torch.unsqueeze(x, 1), the size of x1 is [3,1].

In [ ]:

```
*#x2 becomes a matrix of (1,3)*
x2 = torch.unsqueeze(x, 0)
print('x2: ', x2)
print('x2.size: ', x2.size())
```

Out []

```
x2: tensor([[1, 2, 3]])
x2.size: torch.Size([1, 3])
```

When we perform torch.unsqueeze(x, 0), the size of x2 is [1,3].

In [ ]:

```
*# Example 3 - breaking (to illustrate when it breaks)*
print(x.unsqueeze())
```

Out []

```
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-89-5a320a828907> in <module>()
** 2**
** 3**
----> 4 print(x.unsqueeze())
TypeError: unsqueeze() missing 1 required positional arguments: "dim"
```

We must specified the dimension correctly, although we are just adding 1 dim, it is necessary to put like this: x.unqueeze(0)

## Function 5 — Torch Eq (Element Wise equality)

This function is under comparison category and it computes equality in element-wise and returns a boolean tensor. True if equal, False otherwise.

Lets review how we can operate with different sizes of tensors:

In [ ]:

```
*# Example 1 - working *
x1 = torch.tensor([[1, 2], [3, 4.]])
x2 = torch.tensor([[2, 2], [2, 5]])
x3 = torch.randn(3,5)
#size x1 and z2
print(x1.size())
print(x2.size())
# tensors 1 and 2
print(x1)
print(x2)
#size x3
print(x3.size())
#tensors 3
print(x3)
torch.eq(x1,x2)
torch.Size([2, 2])
torch.Size([2, 2])
tensor([[1., 2.],
[3., 4.]])
tensor([[2, 2],
[2, 5]])
torch.Size([3, 5])
tensor([[-1.3040, -0.4658, -0.5269, 0.7409, 0.9135],
[ 1.0780, 2.0584, -0.9629, -1.1412, -0.3105],
[ 0.3613, -1.4196, 2.1145, 0.3649, 0.2037]])
```

Out[ ]:

```
tensor([[False, True],
[False, False]])
```

x1 and x3 have the same size, but x3 is [3,5], has bigger size.

comparing x1 and x2 is Ok.

In [ ]:

```
*# Example 2 - working (with broadcasting)*
x4 = torch.tensor([[1, 2], [3, 4]])
print(x4.size())
x5 = torch.tensor([2, 5])
print(x5.size())
torch.eq(x4, x5)
torch.Size([2, 2])
torch.Size([2])
```

Out[ ]:

```
tensor([[False, False],
[False, False]])
```

we can also compare, different sizes only if the seconf value, in this case x5, that has size of [2] is broadcastable with the frist one thst is [2,2]

In [ ]:

```
*# Example 3 - breaking (to illustrate when it breaks)*
x6 = torch.tensor([[0, 2, 4], [3, 4, 5]])
print(x6.size())
x7 = torch.tensor([[2, 3], [2, 4]])
print(x7.size())
torch.eq(x6, x3)
torch.Size([2, 3])
torch.Size([2, 2])
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-98-ac2ad1ecd5b0> in <module>()
5 print(x7.size())
6
----> 7 torch.eq(x6, x3)
RuntimeError: The size of tensor a (3) must match the size of tensor b (5) at non-singleton dimension 1
```

finally, we cant compare different sizes if the second arguments shape is not broadcastable with the first argument.

## Conclusion

we review 5 basic topics covering more than 10 PyTorch functions.

in the next post i’ll talk about Linear Regression.

## Reference Links

Provide links to your references and other interesting articles about tensors

Official documentation for tensor operations: https://pytorch.org/docs/stable/torch.html

Squeeze and unsqueeze functions here: https://www.programmersought.com/article/65705814717/

Understanding dimensions in Pytorch: https://towardsdatascience.com/understanding-dimensions-in-pytorch-6edf9972d3be

#aws #reinvent2020 #awsperu #awsugperu #awscloud

Carlos Cortez — AWS UG Perú Leader / AWS ML Community Builder
ccortez@aws.pe
@ccortezb
Podcast: imperiocloud.com @imperiocloud
twitch.tv/awsugperu
cennticloud.thinkific.com
**Carlos Cortez ML Engineer — Senior Cloud Architect — Pro Services Cluster — BGH Tech Partner |…**
*Hello Everyone, I’m the Founder of AWS User Group Perú 🇵🇪 (2014) and also an AWS Solutions Architect Certified…*linkedin.com