回顾总结深度学习中,常用的部分矩阵操作,并给出numpy、paddle、torch代码实现。 个人能力有限,如果错误烦请指正!
在阅读论文过程中度一些矩阵乘积操作有些不了解,在此总结各种矩阵乘积操作并给出numpy、torch、paddle实现过程
ref:
矩阵的各种乘积 ——CSDN
NumPy 线性代数 ——菜鸟教程
dot ——paddle API
tensordot ——paddle API
dot ——torch API
向量积 ——百度百科
matmul ——paddle API
mm ——paddle API
matmul ——torch API
mm ——torch API
如何在Numpy中求元素矩阵乘法(Hadamard积)
multiply ——paddle API
multipy——torch API
mul ——torch API
克罗内克积与笛卡尔积 ——CSDN
矩阵直积 ——百度文库
两个数组的克罗内科乘积 ——CSDN
kron ——paddle API
kron ——torch API
张量积 ——百度百科
Preliminaries
numpy==1.19.2
paddlepaddle==2.2.2
torch==1.7.0
定义矩阵A、B、C,向量a、c
其中A.shape = [ma, na] B.shape = [mb, nb] C.shape = [ma, na] ,a.shape=[m] c.shape=[m]
内积/inner product
数量积/scalar product;
点积/dot product;
输入shape必须相同
向量内积——>标量数值,shape = [1]
矩阵内积,使用A与C进行内积:
-
numpy:np.inner, 所有行向量两两做内积,shape = [ma, ma]
-
paddle:
-
paddle.dot, 对应行的行向量两两做内积,shape = [ma, 1]
-
paddle.tensordot,
-
axes=[0], 所有列向量两两做内积,shape = [na, na]
-
axes=[1], 所有行向量两两做内积,shape = [ma, ma]
-
axes=[0, 1], 对应行的行向量两两做内积,求和,shape = [1]
-
-
-
torch: torch.dot,仅支持1维向量内积
外积/outer product
向量积/corss product
乘积
叉乘
矩阵乘法/矩阵积
第1个输入矩阵的第2维shape必须与第2个输入矩阵的第1维shape相同
矩阵外积,使用A与B进行外积:
-
numpy:
-
np.inner,shape = [ma, nb]
-
np.dot,shape = [ma, nb]
-
-
paddle:
-
paddle.matmul,shape = [ma, nb]
-
paddle.mm,shape = [ma, nb]
-
-
torch:
-
torch.matmul,shape = [ma, nb]
-
torch.mm,shape = [ma, nb]
-
元素积/element-wise product
点乘/point-wise product
哈达玛积/Hadamard product
输入shape必须相同
矩阵元素,使用A与C进行内积
-
numpy:
-
*, shape = [ma, na]
-
np.multiply, shape = [ma, na]
-
-
paddle:
-
*, shape = [ma, na]
-
paddle.multiply, shape = [ma, na]
-
-
torch: torch.dot
-
*, shape = [ma, na]
-
torch.multiply, shape = [ma, na]
-
torch.mul, shape = [ma, na]
-
直积/direct product
克罗内克积/Kronecker product
笛卡尔积/Cartesian product
两矩阵中所有元素对应相乘
输入任意shape的两矩阵
矩阵元素,使用A与B进行内积
-
numpy:np.kron, shape = [ma * mb, na * nb]
-
paddle:paddle.kron, shape = [ma * mb, na * nb]
-
torch(version>=1.8.0):torch.kron, shape = [ma * mb, na * nb]
张量积/tensor product
代码:
# -*- coding: utf-8 -*- # @File : matirx_product.py # @Author : zhaoHL # @Contact : huilin16@qq.com # @Time Create First: 2022/3/2 15:51 # @Contributor : zhaoHL # @Time Modify Last : 2022/3/2 15:51 ''' @File Description: ''' from __future__ import print_function from __future__ import division import paddle import torch import numpy as np A = [[1, 2], [3, 4], [5, 6]] B = [[1, 2, 3], [4, 5, 6]] C = A.copy() a = [1, 2, 3] c = a.copy() def matrix_inner_product(A, B): A_np = np.array(A) B_np = np.array(B) inner_pd_np = np.inner(A_np, B_np) print('numpy demo'.center(50, '-')) print('A_np.shape', A_np.shape) print('B_np.shape', B_np.shape) print('inner_pd_np.shape', inner_pd_np.shape) print('inner_pd_np\n', inner_pd_np, '\n') A_paddle = paddle.to_tensor(A, dtype=paddle.float32) B_paddle = paddle.to_tensor(B, dtype=paddle.float32) inner_pd_paddle = paddle.dot(A_paddle, B_paddle) inner_pd_paddle0 = paddle.tensordot(A_paddle, B_paddle, axes=[0]) inner_pd_paddle1 = paddle.tensordot(A_paddle, B_paddle, axes=[1]) inner_pd_paddle01 = paddle.tensordot(A_paddle, B_paddle, axes=[0, 1]) print('paddle demo'.center(50, '-')) print('A_paddle.shape', A_paddle.shape) print('B_paddle.shape', B_paddle.shape) print('inner_pd_paddle\n', inner_pd_paddle) print(inner_pd_paddle0) print(inner_pd_paddle1) print(inner_pd_paddle01, '\n') def vector_inner_product(a, b): A_torch = torch.tensor(a) B_torch = torch.tensor(b) inner_pd_torch = torch.dot(A_torch, B_torch) print('torch demo'.center(50, '-')) print('A_torch.shape', A_torch.shape) print('B_torch.shape', B_torch.shape) print('inner_pd_torch\n', inner_pd_torch, '\n') def matrix_outer_product(A, B): A_np = np.array(A) B_np = np.array(B) outer_pd_np = np.matmul(A_np, B_np) outer_pd_np0 = np.dot(A_np, B_np) print('numpy demo'.center(50, '-')) print('A_np.shape', A_np.shape) print('B_np.shape', B_np.shape) print('outer_pd_np.shape', outer_pd_np.shape) print('outer_pd_np\n', outer_pd_np, '\n', outer_pd_np0, '\n') A_paddle = paddle.to_tensor(A, dtype=paddle.float32) B_paddle = paddle.to_tensor(B, dtype=paddle.float32) outer_pd_paddle = paddle.matmul(A_paddle, B_paddle) outer_pd_paddle0 = paddle.mm(A_paddle, B_paddle) print('paddle demo'.center(50, '-')) print('A_paddle.shape', A_paddle.shape) print('B_paddle.shape', B_paddle.shape) print('outer_pd_paddle\n', outer_pd_paddle, '\n', outer_pd_paddle0, '\n') A_torch = torch.tensor(A) B_torch = torch.tensor(B) outer_pd_torch = torch.matmul(A_torch, B_torch) outer_pd_torch0 = torch.mm(A_torch, B_torch) print('torch demo'.center(50, '-')) print('A_torch.shape', A_torch.shape) print('B_torch.shape', B_torch.shape) print('outer_pd_torch\n', outer_pd_torch, '\n', outer_pd_torch0, '\n') def matrix_ele_product(A, B): A_np = np.array(A) B_np = np.array(B) ele_pd_np = A_np * B_np ele_pd_np0 = np.multiply(A_np, B_np) print('numpy demo'.center(50, '-')) print('A_np.shape', A_np.shape) print('B_np.shape', B_np.shape) print('ele_pd_np.shape', ele_pd_np.shape) print('ele_pd_np\n', ele_pd_np, '\n', ele_pd_np0, '\n') A_paddle = paddle.to_tensor(A, dtype=paddle.float32) B_paddle = paddle.to_tensor(B, dtype=paddle.float32) ele_pd_paddle = A_paddle * B_paddle ele_pd_paddle0 = paddle.multiply(A_paddle, B_paddle) print('paddle demo'.center(50, '-')) print('A_paddle.shape', A_paddle.shape) print('B_paddle.shape', B_paddle.shape) print('ele_pd_paddle\n', ele_pd_paddle, '\n', ele_pd_paddle0, '\n') A_torch = torch.tensor(A) B_torch = torch.tensor(B) ele_pd_torch = A_torch * B_torch ele_pd_torch0 = torch.multiply(A_torch, B_torch) ele_pd_torch1 = torch.mul(A_torch, B_torch) print('torch demo'.center(50, '-')) print('A_torch.shape', A_torch.shape) print('B_torch.shape', B_torch.shape) print('ele_pd_torch\n', ele_pd_torch, '\n', ele_pd_torch0, '\n', ele_pd_torch1, '\n') def matrix_direct_product(A, B): A_np = np.array(A) B_np = np.array(B) dir_pd_np = np.kron(A_np, B_np) print('numpy demo'.center(50, '-')) print('A_np.shape', A_np.shape) print('B_np.shape', B_np.shape) print('dir_pd_np.shape', dir_pd_np.shape) print('dir_pd_np\n', dir_pd_np, '\n') A_paddle = paddle.to_tensor(A, dtype=paddle.float32) B_paddle = paddle.to_tensor(B, dtype=paddle.float32) dir_pd_paddle = paddle.kron(A_paddle, B_paddle) print('paddle demo'.center(50, '-')) print('A_paddle.shape', A_paddle.shape) print('B_paddle.shape', B_paddle.shape) print('dir_pd_paddle\n', dir_pd_paddle, '\n') A_torch = torch.tensor(A) B_torch = torch.tensor(B) dir_pd_torch = torch.kron(A_torch, B_torch) print('torch demo'.center(50, '-')) print('A_torch.shape', A_torch.shape) print('B_torch.shape', B_torch.shape) print('dir_pd_torch\n', dir_pd_torch, '\n') if __name__ == '__main__': pass vector_inner_product(a, c) matrix_inner_product(A, C) matrix_outer_product(A, B) matrix_ele_product(A, C) matrix_direct_product(A, B)