Press "Enter" to skip to content

PyTorch框架学习(二) — 一文精通张量操作

PyTorch框架学习(二) — 张量操作与线性回归

 

1 张量的操作

 

1.1 拼接

 

torch.cat()

 

torch.cat(tensors, dim=0, *, out=None) → Tensor

 

【功能】:将张量按照 dim 维度进行拼接。

 

∙ \bullet ∙ tensors: 任何相同类型的python张量序列,其中的非空张量除了拼接维度上的形状可以不同,其他维度上的形状必须相同。注:python的序列数据只有 listtuple

∙ \bullet ∙
dim:

要拼接的维度,其中,dim

 

∈ \in ∈

 

[0, len(tensor[0])) 。

 

【代码】:

 

import torch
t = torch.randn(2, 3)
t_0 = torch.cat([t, t, t], dim=0)
t_1 = torch.cat([t, t, t], dim=1)
print("t = {} shape = {}
t_0 = {} shape = {}
t_1 = {} shape = {}"
      .format(t, t.shape, t_0, t_0.shape, t_1, t_1.shape))

 

【结果】:

 

t = tensor([[-0.6014, -1.0122, -0.3023],
        [-1.2277,  0.9198, -0.3485]]) shape = torch.Size([2, 3])
t_0 = tensor([[-0.6014, -1.0122, -0.3023],
       [-1.2277,  0.9198, -0.3485],
       [-0.6014, -1.0122, -0.3023],
       [-1.2277,  0.9198, -0.3485],
       [-0.6014, -1.0122, -0.3023],
       [-1.2277,  0.9198, -0.3485]]) shape = torch.Size([6, 3])
t_1 = tensor([[-0.6014, -1.0122, -0.3023, -0.6014, -1.0122, -0.3023, -0.6014, -1.0122,
        -0.3023],
       [-1.2277,  0.9198, -0.3485, -1.2277,  0.9198, -0.3485, -1.2277,  0.9198,
        -0.3485]]) shape = torch.Size([2, 9])

 

torch.stack()

 

torch.stack(tensors, dim=0, *, out=None) → Tensor

 

【功能】:沿着一个新维度对输入张量序列进行连接。序列中所有的张量都应该为相同形状。可以理解为:把多个2维的张量凑成一个3维的张量;多个3维的凑成一个4维的张量…以此类推,也就是在增加新的维度进行堆叠。

 

∙ \bullet ∙ tensors: 待连接的张量序列。注:python的序列数据只有 listtuple

 

∙ \bullet ∙ dim: 新的维度, 其中 dim ∈ \in ∈ [0, len(out)) 。注: len(out) 是生成数据的维度大小,也就是 out 的维度值。

 

【代码】:

 

import torch
t1 = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
t2 = torch.tensor([[10, 20, 30], [40, 50, 60], [70, 80, 90]])
t_stack_0 = torch.stack([t1, t2], dim=0)
print("t_stack_0:{}".format(t_stack_0))
print("t_stack_0.shape:{}
".format(t_stack_0.shape))
t_stack_1 = torch.stack([t1, t2], dim=1)
print("t_stack_1:{}".format(t_stack_1))
print("t_stack_1.shape:{}
".format(t_stack_1.shape))
t_stack_2 = torch.stack([t1, t2], dim=2)
print("t_stack_2:{}".format(t_stack_2))
print("t_stack_2.shape:{}
".format(t_stack_2.shape))
t_stack_3 = torch.stack([t1, t2], dim=3)
print("t_stack_3.shape:{}".format(t_stack_3.shape))

 

【结果】:

 

t_stack_0:tensor([[[ 1,  2,  3],
         [ 4,  5,  6],
         [ 7,  8,  9]],
        [[10, 20, 30],
         [40, 50, 60],
         [70, 80, 90]]])
t_stack_0.shape:torch.Size([2, 3, 3])
t_stack_1:tensor([[[ 1,  2,  3],
         [10, 20, 30]],
        [[ 4,  5,  6],
         [40, 50, 60]],
        [[ 7,  8,  9],
         [70, 80, 90]]])
t_stack_1.shape:torch.Size([3, 2, 3])
t_stack_2:tensor([[[ 1, 10],
         [ 2, 20],
         [ 3, 30]],
        [[ 4, 40],
         [ 5, 50],
         [ 6, 60]],
        [[ 7, 70],
         [ 8, 80],
         [ 9, 90]]])
t_stack_2.shape:torch.Size([3, 3, 2])
IndexError: Dimension out of range (expected to be in range of [-3, 2], but got 3)

 

【结果说明】:当dim=3时,此时dim= len(out)=3,所以溢出报错。

 

1.2 切分

 

torch.chunk()

 

torch.chunk(input, chunks, dim=0) → List of Tensors

 

【功能】:将张量按照维度 dim 进行平均切分。若不能整除,则最后一份张量小于其他张量。

 

∙ \bullet ∙ input: 待切分的张量。

 

∙ \bullet ∙ chunks: 要切分的份数。

 

∙ \bullet ∙ dim: 要切分的维度。

 

【代码】:

 

import torch
a = torch.ones((2, 7))
print("a = {}".format(a))
list_of_tensors_1 = torch.chunk(a, chunks=3, dim=1)
for idx, t in enumerate(list_of_tensors_1):
    print("第{}个张量: {},shape is {}".format(idx+1, t, t.shape))
print("
")
b = torch.arange(11)
print("b = {}".format(b))
list_of_tensors_2 = b.chunk(6)
for idx, t in enumerate(list_of_tensors_2):
    print("第{}个张量: {},shape is {}".format(idx+1, t, t.shape))
print("
")
c = torch.arange(12)
print("c = {}".format(c))
list_of_tensors_3 = c.chunk(6)
for idx, t in enumerate(list_of_tensors_3):
    print("第{}个张量: {},shape is {}".format(idx+1, t, t.shape))

 

【结果】:

 

a = tensor([[1., 1., 1., 1., 1., 1., 1.],
        [1., 1., 1., 1., 1., 1., 1.]])
第1个张量: tensor([[1., 1., 1.],
        [1., 1., 1.]]),shape is torch.Size([2, 3])
第2个张量: tensor([[1., 1., 1.],
        [1., 1., 1.]]),shape is torch.Size([2, 3])
第3个张量: tensor([[1.],
        [1.]]),shape is torch.Size([2, 1])
b = tensor([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10])
第1个张量: tensor([0, 1]),shape is torch.Size([2])
第2个张量: tensor([2, 3]),shape is torch.Size([2])
第3个张量: tensor([4, 5]),shape is torch.Size([2])
第4个张量: tensor([6, 7]),shape is torch.Size([2])
第5个张量: tensor([8, 9]),shape is torch.Size([2])
第6个张量: tensor([10]),shape is torch.Size([1])
c = tensor([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])
第1个张量: tensor([0, 1]),shape is torch.Size([2])
第2个张量: tensor([2, 3]),shape is torch.Size([2])
第3个张量: tensor([4, 5]),shape is torch.Size([2])
第4个张量: tensor([6, 7]),shape is torch.Size([2])
第5个张量: tensor([8, 9]),shape is torch.Size([2])
第6个张量: tensor([10, 11]),shape is torch.Size([2])

 

【结果说明】:由于 7 不能整除 3,7/3 再向上取整是 3,因此前两个维度是 [2, 3],所以最后一个切分的张量维度是 [2,1]。

 

torch.split()

 

torch.split(tensor, split_size_or_sections, dim=0)

 

【功能】:将张量按照维度 dim 进行平均切分。可以指定每一个分量的切分长度。

 

∙ \bullet ∙ tensor: 待切分的张量。

 

∙ \bullet ∙ split_size_or_sections: 为 int 时,表示每一份的长度,如果不能被整除,则最后一份张量小于其他张量;为 list 时,按照 list 元素作为每一个分量的长度切分。如果 list 元素之和不等于切分维度 (dim) 的值,就会报错。

 

∙ \bullet ∙ dim: 要切分的维度。

 

【代码】:

 

import torch
a = torch.ones((2, 5))
print("a = {}".format(a))
list_of_tensors_1 = torch.split(a, [2, 1, 2], dim=1)
for idx, t in enumerate(list_of_tensors_1):
    print("第{}个张量:{}, shape is {}".format(idx + 1, t, t.shape))
print("
")
b = torch.arange(10).reshape(5, 2)
print("b = {}".format(b))
list_of_tensors_2 = torch.split(b, 2)
for idx, t in enumerate(list_of_tensors_2):
    print("第{}个张量:{}, shape is {}".format(idx + 1, t, t.shape))
print("
")
c = torch.arange(10).reshape(5, 2)
print("c = {}".format(c))
list_of_tensors_3 = torch.split(c, [1, 4])
for idx, t in enumerate(list_of_tensors_3):
    print("第{}个张量:{}, shape is {}".format(idx + 1, t, t.shape))

 

【结果】:

 

a = tensor([[1., 1., 1., 1., 1.],
        [1., 1., 1., 1., 1.]])
第1个张量:tensor([[1., 1.],
        [1., 1.]]), shape is torch.Size([2, 2])
第2个张量:tensor([[1.],
        [1.]]), shape is torch.Size([2, 1])
第3个张量:tensor([[1., 1.],
        [1., 1.]]), shape is torch.Size([2, 2])
b = tensor([[0, 1],
        [2, 3],
        [4, 5],
        [6, 7],
        [8, 9]])
第1个张量:tensor([[0, 1],
        [2, 3]]), shape is torch.Size([2, 2])
第2个张量:tensor([[4, 5],
        [6, 7]]), shape is torch.Size([2, 2])
第3个张量:tensor([[8, 9]]), shape is torch.Size([1, 2])
c = tensor([[0, 1],
        [2, 3],
        [4, 5],
        [6, 7],
        [8, 9]])
第1个张量:tensor([[0, 1]]), shape is torch.Size([1, 2])
第2个张量:tensor([[2, 3],
        [4, 5],
        [6, 7],
        [8, 9]]), shape is torch.Size([4, 2])

 

1.3 索引

 

torch.index_select()

 

torch.index_select(input, dim, index, *, out=None) → Tensor

 

【功能】:在维度 dim 上,按照 index 索引取出数据拼接为张量返回。

 

∙ \bullet ∙ input: 待索引的张量。

 

∙ \bullet ∙ dim: 要索引的维度。

 

∙ \bullet ∙ index(IntTensor or LongTensor): 要索引数据的序号,1D张量。

 

【代码】:

 

import torch
t = torch.randint(0, 9, size=(3, 3))                    # 创建均匀分布
idx = torch.tensor([0, 2], dtype=torch.long)            # 注意 idx 的 dtype 不能指定为 torch.float
print("idx: {}".format(idx))
t_select_1 = torch.index_select(t, dim=0, index=idx)    # 取出第 0 行和第 2 行
print("t: {}
t_select_1:
{}
".format(t, t_select_1))
x = torch.randn(3, 4)
indices = torch.tensor([0, 2])
t_select_2 = torch.index_select(x, 0, indices)          # 取出第 0 行和第 2 行
print("x: {}".format(x))
print("t_select_2:
{}".format(t_select_2))
t_select_3 = torch.index_select(x, 1, indices)          # 取出第 0 列和第 2 列    
print("t_select_3:
{}".format(t_select_3))

 

【结果】:

 

idx: tensor([0, 2])
t: tensor([[5, 3, 6],
        [4, 6, 2],
        [8, 7, 4]])
t_select_1:
tensor([[5, 3, 6],
        [8, 7, 4]])
x: tensor([[-0.7894, -0.9907, -1.9858,  0.5365],
        [ 1.7030, -2.0950, -0.9801,  0.2507],
        [-0.1537,  0.9861,  0.0340, -1.5576]])
t_select_2:
tensor([[-0.7894, -0.9907, -1.9858,  0.5365],
        [-0.1537,  0.9861,  0.0340, -1.5576]])
t_select_3:
tensor([[-0.7894, -1.9858],
        [ 1.7030, -0.9801],
        [-0.1537,  0.0340]])

 

torch.masked_select()

 

torch.masked_select(input, mask, *, out=None) → Tensor

 

【功能】:按照 mask 中的 True 进行索引拼接得到一维张量返回。

 

∙ \bullet ∙ input(Tensor): 待索引的张量。

 

∙ \bullet ∙ mask(BoolTensor): 与 input 同形状的布尔类型张量。

 

【代码】:

 

import torch
t = torch.randint(0, 9, size=(3, 3))
print("t: {}".format(t))
mask = t.le(5)
print("mask: {}".format(mask))
t_select = torch.masked_select(t, mask)     # 取出小于等于 5 的数
print("t_select: {}
".format(t_select))
x = torch.randn(3, 4)
print("x: {}".format(x))
mask = x.ge(0.5)
print("mask: {}".format(mask))
x_select = torch.masked_select(x, mask)
print("x_select: {}
".format(x_select))    # 取出大于等于 0.5 的数

 

【结果】:

 

t: tensor([[5, 3, 6],
        [4, 6, 2],
        [8, 7, 4]])
mask: tensor([[ True,  True, False],
        [ True, False,  True],
        [False, False,  True]])
t_select: tensor([5, 3, 4, 2, 4])
x: tensor([[-0.7894, -0.9907, -1.9858,  0.5365],
        [ 1.7030, -2.0950, -0.9801,  0.2507],
        [-0.1537,  0.9861,  0.0340, -1.5576]])
mask: tensor([[False, False, False,  True],
        [ True, False, False, False],
        [False,  True, False, False]])
x_select: tensor([0.5365, 1.7030, 0.9861])

 

【注意】:最后返回的是一维张量。

 

1.4 变换

 

torch.reshape()

 

torch.reshape(input, shape) → Tensor

 

【功能】:变换张量的形状。当张量在内存中是连续时,返回的张量和原来的张量共享数据内存,改变一个变量时,另一个变量也会被改变。

 

∙ \bullet ∙ input(Tensor): 待变换的张量。

 

∙ \bullet ∙ shape(tuple of python:ints): 新张量的形状。

 

【代码】:

 

import torch
t = torch.randperm(8)   # 生成 0 到 8 的随机排列
print("t: {}".format(t))
t_reshape = torch.reshape(t, (-1, 2, 2))
print("t_reshape: {}".format(t_reshape))
print("t_reshape_shape: {}
".format(t_reshape.shape))
a = torch.arange(4)
print("a: {}".format(a))
a_reshape = torch.reshape(a, (2, 2))
print("a_reshape: {}".format(a_reshape))
print("a_reshape_shape: {}
".format(a_reshape.shape))
b = torch.tensor([[0, 1], [2, 3]])
print("b: {}".format(b))
b_reshape = torch.reshape(b, (-1, ))
print("b_reshape: {}".format(b_reshape))
print("b_reshape_shape: {}
".format(b_reshape.shape))
# 修改原来的张量的一个元素,新张量也会被改变
t[0] = 1024
print("t: {}".format(t))
print("t_reshape: {}".format(t_reshape))
print("t.data内存地址: {}".format(id(t.data)))
print("t_reshape.data内存地址: {}".format(id(t_reshape.data)))

 

【结果】:

 

t: tensor([1, 0, 2, 5, 4, 7, 3, 6])
t_reshape: tensor([[[1, 0],
         [2, 5]],
        [[4, 7],
         [3, 6]]])
t_reshape_shape: torch.Size([2, 2, 2])
a: tensor([0, 1, 2, 3])
a_reshape: tensor([[0, 1],
        [2, 3]])
a_reshape_shape: torch.Size([2, 2])
b: tensor([[0, 1],
        [2, 3]])
b_reshape: tensor([0, 1, 2, 3])
b_reshape_shape: torch.Size([4])
t: tensor([1024,    0,    2,    5,    4,    7,    3,    6])
t_reshape: tensor([[[1024,    0],
         [   2,    5]],
        [[   4,    7],
         [   3,    6]]])
t.data内存地址: 140396336572672
t_reshape.data内存地址: 140396336572672

 

torch.transpose()

 

torch.transpose(input, dim0, dim1) → Tensor

 

【功能】:交换张量的两个维度。常用于图像的变换,比如把 c ∗ h ∗ w c\ast h\ast w 变换为 h ∗ w ∗ c h\ast w\ast c 。

 

∙ \bullet ∙ input(Tensor): 待交换的张量。

 

∙ \bullet ∙ dim0(int): 要交换的第一个维度。

 

∙ \bullet ∙ dim1(int): 要交换的第二个维度。

 

【代码】:

 

import torch
t = torch.rand((2, 3, 4))
print("t: {}".format(t))
print("t shape: {}
".format(t.shape))
t_transpose = torch.transpose(t, dim0=1, dim1=2)
print("t_transpose: {}".format(t_transpose))
print("t_transpose shape: {}".format(t_transpose.shape))

 

【结果】:

 

t: tensor([[[0.4581, 0.4829, 0.3125, 0.6150],
         [0.2139, 0.4118, 0.6938, 0.9693],
         [0.6178, 0.3304, 0.5479, 0.4440]],
        [[0.7041, 0.5573, 0.6959, 0.9849],
         [0.2924, 0.4823, 0.6150, 0.4967],
         [0.4521, 0.0575, 0.0687, 0.0501]]])
t shape: torch.Size([2, 3, 4])
t_transpose: tensor([[[0.4581, 0.2139, 0.6178],
         [0.4829, 0.4118, 0.3304],
         [0.3125, 0.6938, 0.5479],
         [0.6150, 0.9693, 0.4440]],
        [[0.7041, 0.2924, 0.4521],
         [0.5573, 0.4823, 0.0575],
         [0.6959, 0.6150, 0.0687],
         [0.9849, 0.4967, 0.0501]]])
t_transpose shape: torch.Size([2, 4, 3])

 

torch.t()

 

torch.t(input) → Tensor

 

【功能】:2 维张量转置,对于 2 维矩阵而言,等价于 torch.transpose(input, 0, 1)

 

∙ \bullet ∙ input(Tensor): 待转置的张量。

 

【代码】:

 

import torch
x1 = torch.randn(())
print("x1: {}".format(x1))
x1_t = torch.t(x1)
print("x1_t: {}
".format(x1_t))
x2 = torch.randn(3)
print("x2: {}".format(x2))
x2_t = torch.t(x2)
print("x2_t: {}
".format(x2_t))
x3 = torch.randn(2, 3)
print("x3: {}".format(x3))
x3_t = torch.t(x3)
print("x3_t: {}".format(x3_t))

 

【结果】:

 

x1: -0.6013928055763245
x1_t: -0.6013928055763245
x2: tensor([-1.0122, -0.3023, -1.2277])
x2_t: tensor([-1.0122, -0.3023, -1.2277])
x3: tensor([[ 0.9198, -0.3485, -0.8692],
        [-0.9582, -1.1920,  1.9050]])
x3_t: tensor([[ 0.9198, -0.9582],
        [-0.3485, -1.1920],
        [-0.8692,  1.9050]])

 

【结果说明】:对于0-D和1-D张量,输出结果仍然为输入本身。

 

torch.squeeze()

 

torch.squeeze(input, dim=None, *, out=None) → Tensor

 

【功能】:压缩长度为 1 的维度。

 

∙ \bullet ∙ input(Tensor): 待压缩的张量。

 

∙ \bullet ∙ dim(int, optional): 若为 None,则移除所有长度为 1 的维度;若指定维度,则当且仅当该维度长度为 1 时可以移除。

 

【代码】:

 

import torch
t = torch.rand((1, 2, 3, 1))    # 维度 0 和 3 的长度是 1
t_sq = torch.squeeze(t)         # 可以移除维度 0 和 3
t_0 = torch.squeeze(t, dim=0)   # 可以移除维度 0
t_1 = torch.squeeze(t, dim=1)   # 不能移除维度 1
print("t.shape: {}".format(t.shape))
print("t_sq.shape: {}".format(t_sq.shape))
print("t_0.shape: {}".format(t_0.shape))
print("t_1.shape: {}".format(t_1.shape))

 

【结果】:

 

t.shape: torch.Size([1, 2, 3, 1])
t_sq.shape: torch.Size([2, 3])
t_0.shape: torch.Size([2, 3, 1])
t_1.shape: torch.Size([1, 2, 3, 1])

 

torch.unsqueeze()

 

torch.unsqueeze(input, dim) → Tensor

 

【功能】:根据 dim 扩展维度,长度为 1。

 

∙ \bullet ∙ input(Tensor): 待扩展的张量。

 

∙ \bullet ∙ dim(int): 指定扩展的维度。

 

【代码】:

 

import torch
t = torch.tensor([1, 2, 3, 4])
t_0 = torch.unsqueeze(t, 0)     # 在维度 0 扩展
t_1 = torch.unsqueeze(t, 1)     # 在维度 1 扩展
print("t.shape: {}".format(t.shape))
print("t_0.shape: {}".format(t_0.shape))
print("t_1.shape: {}".format(t_1.shape))

 

【结果】:

 

t.shape: torch.Size([4])
t_0.shape: torch.Size([1, 4])
t_1.shape: torch.Size([4, 1])

 

2 张量的数学运算

 

2.1 加法运算

 

torch.add()

 

torch.add(input, other, *, alpha=1, out=None) → Tensor

 

【功能】:逐元素相加,计算公式为: o u t i = i n p u t i + a l p h a × o t h e r i out_{i} = input_{i} + alpha \times other_{i} o u t i ​ = i n p u t i ​ + a l p h a × o t h e r i ​ 。

 

∙ \bullet ∙ input(Tensor): 输入的张量。

 

∙ \bullet ∙ other(Tensor or Number): 与input相加的张量或数值。

 

∙ \bullet ∙ alpha(Number): 乘项因子。

 

【代码】:

 

import torch
a = torch.randn(4)
print("a: {}".format(a))
a_add = torch.add(a, 20)
print("a_add: {}
".format(a_add))
b = torch.randn(4)
print("b: {}".format(b))
c = torch.randn(4, 1)
print("c: {}".format(c))
b_add_c = torch.add(b, c, alpha=10)
print("b_add_c: {}
".format(b_add_c))
d = b + 10 * c
print("d: {}
".format(d))

 

【结果】:

 

a: tensor([-0.6014, -1.0122, -0.3023, -1.2277])
a_add: tensor([19.3986, 18.9878, 19.6977, 18.7723])
b: tensor([ 0.9198, -0.3485, -0.8692, -0.9582])
c: tensor([[-1.1920],
        [ 1.9050],
        [-0.9373],
        [-0.8465]])
b_add_c: tensor([[-11.0006, -12.2689, -12.7896, -12.8786],
        [ 19.9698,  18.7015,  18.1808,  18.0918],
        [ -8.4535,  -9.7218, -10.2425, -10.3315],
        [ -7.5448,  -8.8131,  -9.3339,  -9.4228]])
d: tensor([[-11.0006, -12.2689, -12.7896, -12.8786],
        [ 19.9698,  18.7015,  18.1808,  18.0918],
        [ -8.4535,  -9.7218, -10.2425, -10.3315],
        [ -7.5448,  -8.8131,  -9.3339,  -9.4228]])

 

【结果说明】: torch.add(b, c, alpha=10) 等价于 b + 10 * c

 

2.2 减法运算

 

torch.sub()

 

torch.sub(input, other, *, alpha=1, out=None) → Tensor

 

【功能】:逐元素相减,计算公式为: o u t i = i n p u t i − a l p h a × o t h e r i out_{i} = input_{i} – alpha \times other_{i} o u t i ​ = i n p u t i ​ − a l p h a × o t h e r i ​ 。

 

∙ \bullet ∙ input(Tensor): 输入的张量。

 

∙ \bullet ∙ other(Tensor or Number): 与input相减的张量或数值。

 

∙ \bullet ∙ alpha(Number): 乘项因子。

 

【代码】:

 

import torch
a = torch.tensor((1, 2))
print("a: {}".format(a))
b = torch.tensor((0, 1))
print("b: {}".format(b))
a_sub_b = torch.sub(a, b, alpha=2)
print("a_sub_b: {}
".format(a_sub_b))
c = a - 2 * b
print("c: {}".format(c))

 

【结果】:

 

a: tensor([1, 2])
b: tensor([0, 1])
a_sub_b: tensor([1, 0])
c: tensor([1, 0])

 

【结果说明】: torch.sub(a, b, alpha=2) 等价于 a - 2 * b

 

2.3 哈达玛积运算(element wise,对应元素相乘)

 

torch.mul()

 

torch.mul(input, other, *, out=None) → Tensor

 

【功能】:逐元素相乘,计算公式为: o u t i = i n p u t i × o t h e r i out_{i} = input_{i} \times other_{i} o u t i ​ = i n p u t i ​ × o t h e r i ​ 。

 

∙ \bullet ∙ input(Tensor): 输入的张量。

 

∙ \bullet ∙ other(Tensor or Number): 与input相乘的张量或数值。

 

【代码】:

 

import torch
a = torch.randn(3)
print("a: {}".format(a))
a_mul1 = torch.mul(a, 100)
a_mul2 = a * 100
print("a_mul1: {}".format(a_mul1))
print("a_mul2: {}
".format(a_mul2))
b = torch.randn(4, 1)
c = torch.randn(1, 4)
print("b: {}".format(b))
print("c: {}".format(c))
b_mul_c_1 = torch.mul(b, c)
b_mul_c_2 = b * c
print("b_mul_c_1: {}".format(b_mul_c_1))
print("b_mul_c_2: {}".format(b_mul_c_2))

 

【结果】:

 

a: tensor([-0.6014, -1.0122, -0.3023])
a_mul1: tensor([ -60.1393, -101.2210,  -30.2269])
a_mul2: tensor([ -60.1393, -101.2210,  -30.2269])
b: tensor([[-1.2277],
        [ 0.9198],
        [-0.3485],
        [-0.8692]])
b: tensor([[-0.9582, -1.1920,  1.9050, -0.9373]])
b_mul_c_1: tensor([[ 1.1763,  1.4635, -2.3387,  1.1508],
        [-0.8814, -1.0965,  1.7523, -0.8622],
        [ 0.3339,  0.4154, -0.6638,  0.3266],
        [ 0.8328,  1.0361, -1.6558,  0.8147]])
b_mul_c_2: tensor([[ 1.1763,  1.4635, -2.3387,  1.1508],
        [-0.8814, -1.0965,  1.7523, -0.8622],
        [ 0.3339,  0.4154, -0.6638,  0.3266],
        [ 0.8328,  1.0361, -1.6558,  0.8147]])

 

【结果说明】: torch.mul(b, c) 等价于 b * c

 

2.4 除法运算

 

torch.div()

 

torch.div(input, other, *, rounding_mode=None, out=None) → Tensor

 

【功能】:逐元素相除,计算公式为: o u t i = i n p u t i o t h e r i out_{i} = \frac{input_{i}}{other_{i}} o u t i ​ = o t h e r i ​ i n p u t i ​ ​ 。

 

∙ \bullet ∙ input(Tensor): 输入的张量。

 

∙ \bullet ∙ other(Tensor or Number): 与input相除的张量或数值。

 

∙ \bullet ∙ rounding_mode(str, optional): “None”– 不执行舍入,等价于Python中的 / 运算,或者 np.true_divide ;“trunc”– 将除法结果四舍五入到零,等价于C风格的整除运算;“floor”– 向下舍入除法的结果,等价于Python中的 // 运算,或者 np.floor_divide

 

【代码】:

 

import torch
x = torch.tensor([0.3810, 1.2774, -0.2972, -0.3719, 0.4637])
print("x: {}".format(x))
x_div = torch.div(x, 0.5)
print("x_div: {}".format(x_div))
a = torch.tensor([[-0.3711, -1.9353, -0.4605, -0.2917],
                  [0.1815, -1.0111, 0.9805, -1.5923],
                  [0.1062, 1.4581, 0.7759, -1.2344],
                  [-0.1830, -0.0313, 1.1908, -1.4757]])
b = torch.tensor([0.8032, 0.2930, -0.8113, -0.2308])
a_div_b1 = torch.div(a, b)
print("a_div_b1: {}".format(a_div_b1))
a_div_b2 = a / b
print("a_div_b2: {}".format(a_div_b2))

 

【结果】:

 

x: tensor([ 0.3810,  1.2774, -0.2972, -0.3719,  0.4637])
x_div: tensor([ 0.7620,  2.5548, -0.5944, -0.7438,  0.9274])
a_div_b1: tensor([[-0.4620, -6.6051,  0.5676,  1.2639],
        [ 0.2260, -3.4509, -1.2086,  6.8990],
        [ 0.1322,  4.9764, -0.9564,  5.3484],
        [-0.2278, -0.1068, -1.4678,  6.3938]])
a_div_b2: tensor([[-0.4620, -6.6051,  0.5676,  1.2639],
        [ 0.2260, -3.4509, -1.2086,  6.8990],
        [ 0.1322,  4.9764, -0.9564,  5.3484],
        [-0.2278, -0.1068, -1.4678,  6.3938]])

 

【结果说明】: torch.div(a, b) 等价于 a / b

 

2.5 特殊运算 torch.addcdiv

 

torch.addcdiv()

 

torch.addcdiv(input, tensor1, tensor2, *, value=1, out=None) → Tensor

 

【功能】:计算公式为: o u t i = i n p u t i + v a l u e × t e n s o r 1 i t e n s o r 2 i out_{i} = input_{i} + value\times \frac{tensor1_{i}}{tensor2_{i}} o u t i ​ = i n p u t i ​ + v a l u e × t e n s o r 2 i ​ t e n s o r 1 i ​ ​

 

∙ \bullet ∙ input(Tensor): 输入的张量。

 

∙ \bullet ∙ tensor1(Tensor): 分子张量。

 

∙ \bullet ∙ tensor2(Tensor): 分母张量。

 

∙ \bullet ∙ value(Number, optional): 乘积因子。

 

【代码】:

 

import torch
t = torch.randn(1, 3)
print("t: {}".format(t))
t1 = torch.randn(3, 1)
print("t1: {}".format(t1))
t2 = torch.randn(1, 3)
print("t2: {}".format(t2))
result = torch.addcdiv(t, t1, t2, value=0.5)
print("result: {}".format(result))

 

【结果】:

 

t: tensor([[-0.6014, -1.0122, -0.3023]])
t1: tensor([[-1.2277],
        [ 0.9198],
        [-0.3485]])
t2: tensor([[-0.8692, -0.9582, -1.1920]])
result: tensor([[ 0.1048, -0.3716,  0.2127],
        [-1.1305, -1.4922, -0.6881],
        [-0.4009, -0.8304, -0.1561]])

 

2.6 特殊运算 torch.addcmul

 

torch.addcmul()

 

torch.addcmul(input, tensor1, tensor2, *, value=1, out=None) → Tensor

 

【功能】:计算公式为: o u t i = i n p u t i + v a l u e × t e n s o r 1 i × t e n s o r 2 i out_{i} = input_{i} + value\times tensor1_{i}\times tensor2_{i} o u t i ​ = i n p u t i ​ + v a l u e × t e n s o r 1 i ​ × t e n s o r 2 i ​

 

∙ \bullet ∙ input(Tensor): 输入的张量。

 

∙ \bullet ∙ tensor1(Tensor): 张量,乘子1。

 

∙ \bullet ∙ tensor2(Tensor): 张量,乘子2。

 

∙ \bullet ∙ value(Number, optional): 乘积因子。

 

【代码】:

 

import torch
t = torch.randn(1, 3)
print("t: {}".format(t))
t1 = torch.randn(3, 1)
print("t1: {}".format(t1))
t2 = torch.randn(1, 3)
print("t2: {}".format(t2))
result = torch.addcmul(t, t1, t2, value=0.1)
print("result: {}".format(result))

 

【结果】:

 

t: tensor([[-0.6014, -1.0122, -0.3023]])
t1: tensor([[-1.2277],
        [ 0.9198],
        [-0.3485]])
t2: tensor([[-0.8692, -0.9582, -1.1920]])
result: tensor([[-0.4947, -0.8946, -0.1559],
        [-0.6813, -1.1003, -0.4119],
        [-0.5711, -0.9788, -0.2607]])

 

2.7 幂函数

 

torch.pow()

 

torch.pow(input, exponent, *, out=None) → Tensor

 

【功能】:当 exponent 为标量时,计算公式为: o u t i = i n p u t i e x p o n e n t out_{i} = input_{i}^{exponent} o u t i ​ = i n p u t i e x p o n e n t ​ ;当 exponent 为张量时,计算公式为: o u t i = i n p u t i e x p o n e n t i out_{i} = input_{i}^{exponent_{i} } o u t i ​ = i n p u t i e x p o n e n t i ​ ​ 。

 

∙ \bullet ∙ input(Tensor): 输入的张量。

 

∙ \bullet ∙ exponent(float or tensor): 指数值。

 

【代码】:

 

import torch
t = torch.tensor([1, 2, 3, 4])
print("t: {}".format(t))
t1 = torch.pow(t, 2)
print("t1: {}".format(t1))
exp = torch.arange(1, 5)
print("exp: {}".format(exp))
t2 = torch.pow(t, exp)
print("t2: {}".format(t2))

 

【结果】:

 

t: tensor([1, 2, 3, 4])
t1: tensor([ 1,  4,  9, 16])
exp: tensor([1, 2, 3, 4])
t2: tensor([  1,   4,  27, 256])

 

torch.pow()

 

torch.pow(self, exponent, *, out=None) → Tensor

 

【功能】:计算公式为: o u t i = s e l f e x p o n e n t i out_{i} = self^{exponent_{i} } o u t i ​ = s e l f e x p o n e n t i ​ 。

 

∙ \bullet ∙ self(float): 幂运算的标量基值。

 

∙ \bullet ∙ exponent(tensor): 指数张量。

 

【代码】:

 

import torch
exp = torch.arange(1, 5)
print("exp: {}".format(exp))
base = 2
result = torch.pow(base, exp)
print("result: {}".format(result))

 

【结果】:

 

exp: tensor([1, 2, 3, 4])
result: tensor([ 2,  4,  8, 16])

 

2.7 指数函数

 

torch.exp()

 

torch.exp(input, *, out=None) → Tensor

 

【功能】:计算公式为: o u t i = e i n p u t i out_{i} = e^{input_{i} } o u t i ​ = e i n p u t i ​ 。

 

∙ \bullet ∙ input(Tensor): 输入的张量。

 

【代码】:

 

import torch
import math
t = torch.tensor([0, math.log(2.)])
print("t: {}".format(t))
result = torch.exp(t)
print("result: {}".format(result))

 

【结果】:

 

t: tensor([0.0000, 0.6931])
result: tensor([1., 2.])

 

2.8 对数函数

 

torch.log()

 

torch.log(input, *, out=None) → Tensor

 

【功能】:计算公式为: o u t i = l o g e ( i n p u t i ) out_{i} = log_{e}^{(input_{i}) } o u t i ​ = l o g e ( i n p u t i ​ ) ​ 。

 

∙ \bullet ∙ input(Tensor): 输入的张量。

 

【代码】:

 

import torch
import math
t = torch.tensor([math.e, math.exp(2), math.exp(3)])
print("t: {}".format(t))
result = torch.log(t)
print("result: {}".format(result))

 

【结果】:

 

t: tensor([ 2.7183,  7.3891, 20.0855])
result: tensor([1., 2., 3.])

 

torch.log2()

 

torch.log2(input, *, out=None) → Tensor

 

【功能】:计算公式为: o u t i = l o g 2 ( i n p u t i ) out_{i} = log_{2}^{(input_{i}) } o u t i ​ = l o g 2 ( i n p u t i ​ ) ​ 。

 

∙ \bullet ∙ input(Tensor): 输入的张量。

 

【代码】:

 

import torch
t = torch.tensor([2., 4., 8.])
print("t: {}".format(t))
result = torch.log2(t)
print("result: {}".format(result))

 

【结果】:

 

t: tensor([2., 4., 8.])
result: tensor([1., 2., 3.])

 

torch.log10()

 

torch.log10(input, *, out=None) → Tensor

 

【功能】:计算公式为: o u t i = l o g 10 ( i n p u t i ) out_{i} = log_{10}^{(input_{i}) } o u t i ​ = l o g 1 0 ( i n p u t i ​ ) ​ 。

 

∙ \bullet ∙ input(Tensor): 输入的张量。

 

【代码】:

 

import torch
t = torch.tensor([10., 100., 1000.])
print("t: {}".format(t))
result = torch.log10(t)
print("result: {}".format(result))

 

【结果】:

 

t: tensor([  10.,  100., 1000.])
result: tensor([1., 2., 3.])

 

2.9 三角函数

 

torch.sin()

 

torch.sin(input, *, out=None) → Tensor

 

【功能】:计算公式为: o u t i = s i n ( i n p u t i ) out_{i} = sin\left ( input_{i} \right ) o u t i ​ = s i n ( i n p u t i ​ ) 。

 

∙ \bullet ∙ input(Tensor): 输入的张量。

 

【代码】:

 

import torch
import math
t = torch.tensor([0., 1 / 6 * math.pi, 1 / 3 * math.pi, 1 / 2 * math.pi])
print("t: {}".format(t))
result = torch.sin(t)
print("result: {}".format(result))

 

【结果】:

 

t: tensor([0.0000, 0.5236, 1.0472, 1.5708])
result: tensor([0.0000, 0.5000, 0.8660, 1.0000])

 

同理可以得到其他的三角函数如下所示:

 

torch.cos(input, *, out=None) → Tensor

 

torch.tan(input, *, out=None) → Tensor

 

torch.asin(input, *, out=None) → Tensor # 反正弦函数

 

torch.acos(input, *, out=None) → Tensor # 反余弦函数

 

torch.atan(input, *, out=None) → Tensor # 反正切函数

 

torch.atan2(input, other, *, out=None) → Tensor # input/other对应元素的反正切函数

 

torch.sinh(input, *, out=None) → Tensor # 双曲正弦

 

torch.cosh(input, *, out=None) → Tensor # 双曲余弦

 

torch.tanh(input, *, out=None) → Tensor # 双曲正切

 

2.10 矩阵乘法

 

torch.mm()

 

torch.mm(input, mat2, *, out=None) → Tensor

 

【功能】:如果 input 为 ( n × m ) \left ( n\times m \right ) ( n × m ) 张量,mat2 为 ( m × p ) \left ( m\times p \right ) ( m × p ) 张量,则输出一个 ( n × p ) \left ( n\times p \right ) ( n × p ) 张量。

 

∙ \bullet ∙ input(Tensor): 输入的张量。

 

∙ \bullet ∙ mat2(Tensor): 与 input 执行相乘运算的张量。

 

【代码】:

 

import torch
mat1 = torch.tensor([[1, 2, 3], [0, 1, 4]])
print("mat1: {}".format(mat1))
mat2 = torch.tensor([[3, 2, 1], [2, 1, 4], [0, 0, 1]])
print("mat2: {}".format(mat2))
result = torch.mm(mat1, mat2)
print("result: {}".format(result))

 

【结果】:

 

mat1: tensor([[1, 2, 3],
        [0, 1, 4]])
mat2: tensor([[3, 2, 1],
        [2, 1, 4],
        [0, 0, 1]])
result: tensor([[ 7,  4, 12],
        [ 2,  1,  8]])

 

【注意】: t o r c h . m m {\color{Red} torch.mm} t o r c h . m m 函数不支持 Python 的广播机制,所以必须保证维度能够相乘。

 

torch.bmm()

 

torch.bmm(input, mat2, *, out=None) → Tensor

 

【功能】:实现矩阵乘法的批量运算。input 和 mat2 必须为 3D 张量,并且必须包含相同数量的矩阵。如果 input 为 ( b × n × m ) \left ( b \times n \times m \right ) ( b × n × m ) 张量,mat2 为 ( b × m × p ) \left ( b \times m\times p \right ) ( b × m × p ) 张量,则输出一个 ( b × n × p ) \left ( b \times n\times p \right ) ( b × n × p ) 张量。

 

∙ \bullet ∙ input(Tensor): 输入的张量。

 

∙ \bullet ∙ mat2(Tensor): 与 input 执行相乘运算的张量。

 

【代码】:

 

import torch
tensor1 = torch.rand(2, 3, 4)
tensor2 = torch.rand(2, 4, 3)
print("tensor1: {} tensor1 size: {}".format(tensor1, tensor1.shape))
print("tensor2: {} tensor2 size: {}".format(tensor2, tensor2.shape))
result = torch.bmm(tensor1, tensor2)
print("result: {} result size: {}".format(result, result.shape))

 

【结果】:

 

tensor1: tensor([[[0.4581, 0.4829, 0.3125, 0.6150],
         [0.2139, 0.4118, 0.6938, 0.9693],
         [0.6178, 0.3304, 0.5479, 0.4440]],
        [[0.7041, 0.5573, 0.6959, 0.9849],
         [0.2924, 0.4823, 0.6150, 0.4967],
         [0.4521, 0.0575, 0.0687, 0.0501]]]) tensor1 size: torch.Size([2, 3, 4])
tensor2: tensor([[[0.0108, 0.0343, 0.1212],
         [0.0490, 0.0310, 0.7192],
         [0.8067, 0.8379, 0.7694],
         [0.6694, 0.7203, 0.2235]],
        [[0.9502, 0.4655, 0.9314],
         [0.6533, 0.8914, 0.8988],
         [0.3955, 0.3546, 0.5752],
         [0.4787, 0.5782, 0.7536]]]) tensor2 size: torch.Size([2, 4, 3])
result: tensor([[[0.6924, 0.7355, 0.7807],
         [1.2310, 1.2996, 1.0725],
         [0.7621, 0.8103, 0.8333]],
        [[1.7798, 1.6407, 2.2993],
         [1.0740, 1.0713, 1.4340],
         [0.5183, 0.3150, 0.5500]]]) result size: torch.Size([2, 3, 3])

 

torch.matmul()

 

torch.matmul(input, other, *, out=None) → Tensor

 

【功能】:矩阵的乘法运算,输入可以是高维的。

 

∙ \bullet ∙ input(Tensor): 输入的张量。

 

∙ \bullet ∙ mat2(Tensor): 与 input 执行相乘运算的张量。

 

(1)若两个都是1D的张量,则返回两个张量的点积,结果为标量。

 

【代码】:

 

import torch
tensor1 = torch.tensor([2])
tensor2 = torch.tensor([3])
result = torch.matmul(tensor1, tensor2)
print("result: {}".format(result))

 

【结果】:

 

result: 6

 

【注意】:对于两个输入都是1D的张量, t o r c h . m a t m u l {\color{Red} torch.matmul} t o r c h . m a t m u l 函数不包含 o u t {\color{Red} out} o u t 参数。

 

(2)若两个都是2D的张量,则按照矩阵相乘规则返回2D张量,此时和 t e n s o r . m m {\color{Red} tensor.mm} t e n s o r . m m 函数用法相同。

 

【代码】:

 

import torch
tensor1 = torch.tensor([[1, 2, 3], [0, 1, 4]])
tensor2 = torch.tensor([[3, 2, 1], [2, 1, 4], [0, 0, 1]])
result = torch.matmul(tensor1, tensor2)
print("result: {}".format(result))

 

【结果】:

 

result: tensor([[ 7,  4, 12],
        [ 2,  1,  8]])

 

(3)若input维度1D,other维度2D,则先将1D的维度扩充到2D(1D的维数前面+1),然后得到结果后再将此维度去掉,得到的输出与input的维度相同。即使作广播处理,input的维度也要和other维度做对应关系。

 

【代码】:

 

import torch
tensor1 = torch.rand(4)
tensor2 = torch.rand(4, 3)
print("tensor1: {} tensor1 size: {}".format(tensor1, tensor1.shape))
print("tensor2: {} tensor2 size: {}".format(tensor2, tensor2.shape))
result = torch.matmul(tensor1, tensor2)
print("result: {} result size: {}".format(result, result.shape))

 

【结果】:

 

tensor1: tensor([0.4581, 0.4829, 0.3125, 0.6150]) tensor1 size: torch.Size([4])
tensor2: tensor([[0.2139, 0.4118, 0.6938],
        [0.9693, 0.6178, 0.3304],
        [0.5479, 0.4440, 0.7041],
        [0.5573, 0.6959, 0.9849]]) tensor2 size: torch.Size([4, 3])
result: tensor([1.0800, 1.0537, 1.3031]) result size: torch.Size([3])

 

(4)若input维度2D,other维度1D,则返回两者的点积结果。

 

【代码】:

 

import torch
tensor1 = torch.rand(3, 4)
tensor2 = torch.rand(4)
print("tensor1: {} tensor1 size: {}".format(tensor1, tensor1.shape))
print("tensor2: {} tensor2 size: {}".format(tensor2, tensor2.shape))
result = torch.matmul(tensor1, tensor2)
print("result: {} result size: {}".format(result, result.shape))

 

【结果】:

 

tensor1: tensor([[0.4581, 0.4829, 0.3125, 0.6150],
        [0.2139, 0.4118, 0.6938, 0.9693],
        [0.6178, 0.3304, 0.5479, 0.4440]]) tensor1 size: torch.Size([3, 4])
tensor2: tensor([0.7041, 0.5573, 0.6959, 0.9849]) tensor2 size: torch.Size([4])
result: tensor([1.4148, 1.8176, 1.4376]) result size: torch.Size([3])

 

(5)如果一个维度至少是1D,另外一个大于2D,则返回的是一个批矩阵乘法( a batched matrix multiply)

 

∙ \bullet ∙ 若input是1D,other是大于2D的,则类似于规则(3)。

 

【代码】:

 

import torch
tensor1 = torch.rand(3)
tensor2 = torch.rand(2, 3, 4)
print("tensor1: {} tensor1 size: {}".format(tensor1, tensor1.shape))
print("tensor2: {} tensor2 size: {}".format(tensor2, tensor2.shape))
result = torch.matmul(tensor1, tensor2)
print("result: {} result size: {}".format(result, result.shape))

 

【结果】:

 

tensor1: tensor([0.4581, 0.4829, 0.3125]) tensor1 size: torch.Size([3])
tensor2: tensor([[[0.6150, 0.2139, 0.4118, 0.6938],
         [0.9693, 0.6178, 0.3304, 0.5479],
         [0.4440, 0.7041, 0.5573, 0.6959]],
        [[0.9849, 0.2924, 0.4823, 0.6150],
         [0.4967, 0.4521, 0.0575, 0.0687],
         [0.0501, 0.0108, 0.0343, 0.1212]]]) tensor2 size: torch.Size([2, 3, 4])
result: tensor([[0.8885, 0.6163, 0.5223, 0.7998],
        [0.7067, 0.3556, 0.2594, 0.3528]]) result size: torch.Size([2, 4])

 

∙ \bullet ∙ 若other是1D,input是大于2D的,则类似于规则(4)。

 

【代码】:

 

import torch
tensor1 = torch.rand(2, 3, 4)
tensor2 = torch.rand(4)
print("tensor1: {} tensor1 size: {}".format(tensor1, tensor1.shape))
print("tensor2: {} tensor2 size: {}".format(tensor2, tensor2.shape))
result = torch.matmul(tensor1, tensor2)
print("result: {} result size: {}".format(result, result.shape))

 

【结果】:

 

tensor1: tensor([[[0.4581, 0.4829, 0.3125, 0.6150],
         [0.2139, 0.4118, 0.6938, 0.9693],
         [0.6178, 0.3304, 0.5479, 0.4440]],
        [[0.7041, 0.5573, 0.6959, 0.9849],
         [0.2924, 0.4823, 0.6150, 0.4967],
         [0.4521, 0.0575, 0.0687, 0.0501]]]) tensor1 size: torch.Size([2, 3, 4])
tensor2: tensor([0.0108, 0.0343, 0.1212, 0.0490]) tensor2 size: torch.Size([4])
result: tensor([[0.0895, 0.1481, 0.1062],
        [0.1593, 0.1186, 0.0176]]) result size: torch.Size([2, 3])

 

∙ \bullet ∙ 若input和other都是3D的,则与 t o r c h . b m m {\color{Red} torch.bmm} t o r c h . b m m 函数功能一样。

 

∙ \bullet ∙ 如果input中某一维度满足可以广播,那幺也是可以进行相乘操作的。如果 input 为 ( j × 1 × n × m ) \left ( j \times 1 \times n \times m \right ) ( j × 1 × n × m ) 张量,other 为 ( k × m × p ) \left ( k \times m\times p \right ) ( k × m × p ) 张量,则输出一个 ( j × k × n × p ) \left ( j \times k \times n\times p \right ) ( j × k × n × p ) 张量。

 

【代码】:

 

import torch
tensor1 = torch.rand(2, 1, 2, 4)
tensor2 = torch.rand(2, 4, 3)
print("tensor1: {} tensor1 size: {}".format(tensor1, tensor1.shape))
print("tensor2: {} tensor2 size: {}".format(tensor2, tensor2.shape))
result = torch.matmul(tensor1, tensor2)
print("result: {} result size: {}".format(result, result.shape))

 

【结果】:

 

tensor1: tensor([[[[0.4581, 0.4829, 0.3125, 0.6150],
          [0.2139, 0.4118, 0.6938, 0.9693]]],
        [[[0.6178, 0.3304, 0.5479, 0.4440],
          [0.7041, 0.5573, 0.6959, 0.9849]]]]) tensor1 size: torch.Size([2, 1, 2, 4])
tensor2: tensor([[[0.2924, 0.4823, 0.6150],
         [0.4967, 0.4521, 0.0575],
         [0.0687, 0.0501, 0.0108],
         [0.0343, 0.1212, 0.0490]],
        [[0.0310, 0.7192, 0.8067],
         [0.8379, 0.7694, 0.6694],
         [0.7203, 0.2235, 0.9502],
         [0.4655, 0.9314, 0.6533]]]) tensor2 size: torch.Size([2, 4, 3])
result: tensor([[[[0.4164, 0.5295, 0.3430],
          [0.3481, 0.4416, 0.2103]],
         [[0.9301, 1.3436, 1.3915],
          [1.3026, 1.5286, 1.7407]]],
        [[[0.3976, 0.5286, 0.4266],
          [0.5643, 0.7458, 0.5208]],
         [[0.8973, 1.2344, 1.5302],
          [1.4484, 2.0080, 2.2456]]]]) result size: torch.Size([2, 2, 2, 3])

 

3 线性回归

 

最后,利用上面介绍的张量的基本操作来实现一个简单的线性回归分析,因为线性回归是深度学习的基础,所以有必要先来学习一下如何利用pytorch来实现一个简单的线性回归。

 

线性回归是分析一个变量 ( y y y ) 与另外一 (多) 个变量 ( x x x ) 之间的关系的方法。一般可以写成 y = w x + b y=wx+b w x + b 。线性回归的目的就是求解参数 w , b w, b w , b 。

 

线性回归的求解可以分为 3 步:

 

(1)确定模型: y = w x + b y=wx+b w x + b

 

(2)选择损失函数,一般使用均方误差 MSE: 1 m ∑ i = 1 m ( y i − y ^ i ) 2 \frac{1}{m}\sum_{i=1}^{m} \left ( y_{i}-\hat{y} _{i} \right ) ^{2} ∑ i = 1 m ​ ( y i ​ − y ^ ​ i ​ ) 2 。其中 y ^ i \hat{y} _{i} y ^ ​ i ​ 是预测值, y i y_{i} y i ​ 是真实值。

 

(3)使用梯度下降法求解梯度 (其中 α \alpha α 是学习率),并更新参数: w = w − α ∗ w . g r a d w = w – \alpha * w.grad w . g r a d b = b − α ∗ b . g r a d b = b -\alpha * b.grad b . g r a d

 

【代码】:

 

import torch
import numpy as np
import matplotlib.pyplot as plt
torch.manual_seed(10)
r = 0.05    # 学习率
x = torch.rand(20, 1) * 10              # 创建训练数据
y = 2 * x + (5 + torch.randn(20, 1))    # torch.randn(20, 1) 用于添加噪声
# 构建线性回归参数
w = torch.randn(1, requires_grad=True)
b = torch.zeros(1, requires_grad=True)
# 迭代训练 1000 次
for iteration in range(2000):
    # 前向传播,计算预测值
    wx = torch.mul(w, x)
    y_hat = torch.add(wx, b)
    # 计算 MSE loss
    loss = (1 / 20 * (y - y_hat) ** 2).mean()
    # 反向传播
    loss.backward()
    # 更新参数
    b.data.sub_(r * b.grad)
    w.data.sub_(r * w.grad)
    # 每次更新参数之后,都要清零张量的梯度
    w.grad.zero_()
    b.grad.zero_()
    # 绘图,每隔 20 次重新绘制直线
    if iteration % 20 == 0:
        plt.scatter(x.data.numpy(), y.data.numpy())
        plt.plot(x.data.numpy(), y_hat.data.numpy(), 'r-', lw=3)
        plt.xlim(1.5, 10)
        plt.ylim(8, 28)
        plt.title("Iteration: {}
w: {} b: {} 
loss: {}"
                  .format(iteration, w.data.numpy(), b.data.numpy(), loss.data.numpy()))
        plt.pause(0.5)

 

【结果】:

 

迭代0次的结果如下:

 

 

迭代2000次的结果如下:

 

 

迭代20000次的结果如下:

 

Be First to Comment

发表回复

您的电子邮箱地址不会被公开。 必填项已用*标注