PyTorch memory model: "torch.from_numpy()" vs "torch.Tensor()"
from_numpy()
automatically inherits input array dtype
. On the other hand, torch.Tensor
is an alias for torch.FloatTensor
.
Therefore, if you pass int64
array to torch.Tensor
, output tensor is float tensor and they wouldn't share the storage. torch.from_numpy
gives you torch.LongTensor
as expected.
a = np.arange(10)ft = torch.Tensor(a) # same as torch.FloatTensorit = torch.from_numpy(a)a.dtype # == dtype('int64')ft.dtype # == torch.float32it.dtype # == torch.int64
The recommended way to build tensors in Pytorch is to use the following two factory functions: torch.tensor
and torch.as_tensor
.
torch.tensor
always copies the data. For example, torch.tensor(x)
is equivalent to x.clone().detach()
.
torch.as_tensor
always tries to avoid copies of the data. One of the cases where as_tensor
avoids copying the data is if the original data is a numpy array.
This comes from _torch_docs.py
; there is also a possible discussion on the "why" here.
def from_numpy(ndarray): # real signature unknown; restored from __doc__ """ from_numpy(ndarray) -> Tensor Creates a :class:`Tensor` from a :class:`numpy.ndarray`. The returned tensor and `ndarray` share the same memory. Modifications to the tensor will be reflected in the `ndarray` and vice versa. The returned tensor is not resizable. Example:: >>> a = numpy.array([1, 2, 3]) >>> t = torch.from_numpy(a) >>> t torch.LongTensor([1, 2, 3]) >>> t[0] = -1 >>> a array([-1, 2, 3]) """ pass
Taken from the numpy
docs:
Different
ndarrays
can share the same data, so that changes made in one ndarray may be visible in another. That is, anndarray
can be a “view” to anotherndarray
, and the data it is referring to is taken care of by the “base”ndarray
.
Pytorch docs
:
If a
numpy.ndarray
,torch.Tensor
, ortorch.Storage
is given, a new tensor that shares the same data is returned. If a Python sequence is given, a new tensor is created from a copy of the sequence.