RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same

You get this error because your model is on the GPU, but your data is on the CPU. So, you need to send your input tensors to the GPU.

inputs, labels = data                         # this is what you hadinputs, labels = inputs.cuda(), labels.cuda() # add this line

Or like this, to stay consistent with the rest of your code:

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")inputs, labels = inputs.to(device), labels.to(device)

The same error will be raised if your input tensors are on the GPU but your model weights aren't. In this case, you need to send your model weights to the GPU.

model = MyModel()if torch.cuda.is_available():    model.cuda()

Here is the documentation for cuda() and cpu(), its opposite.

The new API is to use .to() method.

The advantage is obvious and important.Your device may tomorrow be something other than "cuda":

• cpu
• cuda
• mkldnn
• opengl
• opencl
• ideep
• hip
• msnpu
• xla

So try to avoid model.cuda()It is not wrong to check for the device

dev = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")

or to hardcode it:

dev=torch.device("cuda") 

same as:

dev="cuda"

In general you can use this code:

model.to(dev)data = data.to(dev)

As already mentioned in the previous answers, the issue can be that your model is trained on the GPU, but it's tested on the CPU. If that's the case then you need to port your model's weights and the data from the GPU to the CPU like this:

device = args.device # "cuda" / "cpu"if "cuda" in device and not torch.cuda.is_available():    device = "cpu"data = data.to(device)model.to(device)

NOTE: Here we still check if the configuration arguments are set to GPU or CPU, so that this piece of code can be used for both training (on the GPU) and testing (on the CPU).