How do I get the value of a tensor in PyTorch?

You can use x.item() to get a Python number from a tensor that has one element.

Convert tensor to numpy:

x.numpy()[0]

To get a value from single element tensor x.item() works always:

Example : Single element tensor on CPU

x = torch.tensor([3])x.item()

Output:

3

Example : Single element tensor on CPU with AD

x = torch.tensor([3.], requires_grad=True)x.item()

Output:

3.0

NOTE: We needed to use floating point arithmetic for AD

Example : Single element tensor on CUDA

x = torch.tensor([3], device='cuda')x.item()

Output:

3

Example : Single element tensor on CUDA with AD

x = torch.tensor([3.], device='cuda', requires_grad=True)x.item()

Output:

3.0

Example : Single element tensor on CUDA with AD again

x = torch.ones((1,1), device='cuda', requires_grad=True)x.item()

Output:

1.0

To get a value from non single element tensor we have to be careful:

The next example will show that PyTorch tensor residing on CPU shares the same storage as numpy array na

Example: Shared storage

import torcha = torch.ones((1,2))print(a)na = a.numpy()na[0][0]=10print(na)print(a)

Output:

tensor([[1., 1.]])[[10.  1.]]tensor([[10.,  1.]])

Example: Eliminate effect of shared storage, copy numpy array first

To avoid the effect of shared storage we need to copy() the numpy array na to a new numpy array nac. Numpy copy() method creates the new separate storage.

import torcha = torch.ones((1,2))print(a)na = a.numpy()nac = na.copy()nac[0][0]=10​print(nac)print(na)print(a)

Output:

tensor([[1., 1.]])[[10.  1.]][[1. 1.]]tensor([[1., 1.]])

Now, just the nac numpy array will be altered with the line nac[0][0]=10, na and a will remain as is.

Example: CPU tensor requires_grad=True

import torcha = torch.ones((1,2), requires_grad=True)print(a)na = a.detach().numpy()na[0][0]=10print(na)print(a)

Output:

tensor([[1., 1.]], requires_grad=True)[[10.  1.]]tensor([[10.,  1.]], requires_grad=True)

In here we call:

na = a.numpy()

This would cause: RuntimeError: Can't call numpy() on Tensor that requires grad. Use tensor.detach().numpy() instead., because tensors that require_grad=True are recorded by PyTorch AD.

This is why we need to detach() them first before converting using numpy().

Example: CUDA tensor requires_grad=False

a = torch.ones((1,2), device='cuda')print(a)na = a.to('cpu').numpy()na[0][0]=10print(na)print(a)

Output:

tensor([[1., 1.]], device='cuda:0')[[10.  1.]]tensor([[1., 1.]], device='cuda:0')

​In here we just don't convert the CUDA tensor to CPU. There is no effect of shared storage here.

Example: CUDA tensor requires_grad=True

a = torch.ones((1,2), device='cuda', requires_grad=True)print(a)na = a.detach().to('cpu').numpy()na[0][0]=10​print(na)print(a)

Output:

tensor([[1., 1.]], device='cuda:0', requires_grad=True)[[10.  1.]]tensor([[1., 1.]], device='cuda:0', requires_grad=True)

Without detach() method the error RuntimeError: Can't call numpy() on Tensor that requires grad. Use tensor.detach().numpy() instead. will be set.

Without .to('cpu') method TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first. will be set.