上传整体流程
首次上传
initiateMultipartUpload初始化分段上传获取uploadId
uploadPart 上传段
completeMultipartUpload 合并段
二次上传
getObjectMetadata获取上传完成的文件
listMultipartUploads获取上传信息,返回最近一次上传的uploadId
listParts 根据uploadId获取已上传的分片信息
uploadPart 上传未完成的段
completeMultipartUpload 合并段
安装华为云OBS
npm install esdk-obs-browserjs
使用OBS
import ObsClient from "esdk-obs-browserjs/src/obs";
获取秘钥
几乎所有的云平台管理都需要获取秘钥,秘钥通常调用后端接口获取
const secret = await getHwSecret();
//创建OBS对象
const hwClient = new ObsClient({
access_key_id: secret.ak,
secret_access_key: secret.sk,
server: secret.endPoint,
timeout: 3000,
});
Bucket、Key、Prefix作用
在上传时的配置参数,Bucket可以理解为文件夹名称,用于区分文件存放的路径;Key可以理解为文件名称,区分唯一的文件,Prefix可以理解为查找文件的匹配项,通常可以与key相同。
const params = {
Bucket: secret.bucketName,
Key: key,
Prefix: key,
};
初始化分段上传
function initMultiPartUpload(hwClient: any, params: any) {
return new Promise((resolve, reject) => {
hwClient.initiateMultipartUpload(params, (err: any, result: any) => {
if (err) {
reject(err);
} else if (result.CommonMsg.Status < 300 && result.InterfaceResult) {
const uploadId = result.InterfaceResult.UploadId;
resolve(uploadId);
}
});
});
}
分段上传示例
单个分片上传,并获取上传完成的ETag信息,用于手动合并分片时的校验。
//单段上传,成功上传时返回ETag值
function uploadNextPart(
n: number,
hwClient: any,
uploadId: any,
file: any,
params: any
) {
const count = Math.ceil(file.size / PartSize);
const lastPartSize = file.size % PartSize;
return new Promise((resolve, reject) => {
hwClient.uploadPart(
{
...params,
PartNumber: n,
UploadId: uploadId,
SourceFile: file,
PartSize: count === n ? lastPartSize : PartSize,
Offset: (n - 1) * PartSize,
},
(err: any, result: any) => {
if (err) {
reject(err);
} else {
if (result.CommonMsg.Status < 300 && result.InterfaceResult) {
resolve(result.InterfaceResult.ETag);
}
}
}
);
});
}
分片上传,parts不为空时实现续传功能
这里提供了一种分段上传续传逻辑,parts在后面会介绍怎么获取的,它是同一个文件已经上传的分片信息。这里的逻辑是过滤已上传分片,通过PartNubmer来过滤。通过Promise.all方法确保所有分片都执行完,进行手动合并
//分片上传,parts不为空时实现续传功能
async function uploadPart(
fileState: FileState,
file: File,
uploadId: any,
parts: any,
hwClient: any,
params: any
) {
const completeParts = [...parts];
const partNumbers = parts?.map((_: any) => _.PartNumber) || [];
const count = Math.ceil(file.size / PartSize);
if (partNumbers.length) {
fileState.status = FileStatus.processing;
fileState.percent = parseInt((completeParts.length * 100) / count);
}
let startTime = null as any;
const uploadPromises = []; // 存储所有分片上传的 Promise 对象
for (let n = 1; n <= count; n++) {
if (!partNumbers.includes(n)) {
if (!startTime) {
startTime = new Date();
}
const promise = uploadNextPart(n, hwClient, uploadId, file, params)
.then((data: any) => {
completeParts.push({
PartNumber: n,
ETag: data,
});
//按分片数量计算上传速度
const currentTime = new Date();
const elapsedTime =
(currentTime.getTime() - startTime.getTime()) / 1000;
startTime = currentTime;
//上传速度MB/s
fileState.uploadSpeed = (10 / elapsedTime).toFixed(2);
fileState.percent = parseInt((completeParts.length * 100) / count);
})
.catch((err: any) => {
fileState.status = FileStatus.fail;
throw err;
});
uploadPromises.push(promise);
}
}
//所有promise上传结束,调用分片合并校验接口
Promise.all(uploadPromises)
.then(async () => {
checkMultiUpload(uploadId, completeParts, fileState, hwClient, params);
})
.catch(() => {
fileState.status = FileStatus.fail;
});
}
合并分片校验
华为云对合并分片的校验规则是PartNumber必须是按序排列的,因此需要对分片进行排序
//校验分片是否全部上传成功
function completeMultiUpload(
uploadId: any,
parts: any,
fileState: any,
hwClient: any,
params: any
) {
const newParts = parts.sort((a: any, b: any) => a.PartNumber - b.PartNumber);
hwClient.completeMultipartUpload(
{
...params,
UploadId: uploadId,
Parts: newParts,
},
function (err: any, result: any) {
if (err) {
fileState.status = FileStatus.fail;
} else {
if (result.CommonMsg.Status < 300 && result.InterfaceResult) {
fileState.status = FileStatus.success;
fileState.percent = 100;
}
}
}
);
}
检测文件是否上传完成
如果需要断点续传,可以先使用listObjects方法,校验文件是否上传完成。传入首次上传时key信息,查找的关键字Prefix信息
const params = {
Bucket: secret.bucketName,
Key: key,
Prefix: key,
};
function checkIsCompleted(hwClient: any, params: any): Promise<any> {
return new Promise(async (resolve, reject) => {
try {
await hwClient.listObjects(params, (err: any, result: any) => {
if (err) {
reject(err);
} else {
if (
result.CommonMsg.Status < 300 &&
result.InterfaceResult.Contents.length
) {
resolve(true);
} else {
resolve(false);
}
}
});
} catch (err: any) {
reject(err);
}
});
}
获取上传上传的uploadId
如果文件没有上传完成,通过listMultipartUploads方法校验文件是否部分上传,并获取最近上传的uploadId
async function getHwCheckpoint(hwClient: any, params: any) {
return new Promise((resolve, reject) => {
hwClient.listMultipartUploads(params, (err: any, result: any) => {
if (err) {
reject(err);
} else if (
result.CommonMsg.Status < 300 &&
result.InterfaceResult &&
result.InterfaceResult.Uploads.length
) {
const uploads = result.InterfaceResult.Uploads;
resolve(uploads[uploads.length - 1].UploadId);
} else {
resolve("");
}
});
});
}
获取已上传分片信息
根据返回的最近一次上传的uploadId信息,使用listParts方法获取详细的分片信息。当分片数量过大时,使用递归方式获取详细的分片信息。
//根据uploadId获取已上传分片信息
async function listAllUploadParts(
uploadId: any,
hwClient: any,
params: any
): Promise<any> {
let completeParts = [] as any;
const listAll = async function (partNumberMarker = null) {
return new Promise((resolve, reject) => {
hwClient.listParts(
{
...params,
UploadId: uploadId,
PartNumberMarker: partNumberMarker,
},
(err: any, result: any) => {
if (err) {
reject(err);
} else {
if (result.CommonMsg.Status < 300 && result.InterfaceResult) {
const parts = result.InterfaceResult.Parts.map((_: any) => {
return {
PartNumber: parseInt(_.PartNumber),
ETag: _.ETag,
};
});
completeParts = [...completeParts, ...parts];
if (result.InterfaceResult.IsTruncated === "true") {
resolve(listAll(result.InterfaceResult.NextPartNumberMarker));
} else {
resolve(completeParts);
}
} else {
reject(new Error("Failed to list parts"));
}
}
}
);
});
};
await listAll();
return completeParts;
}
主体完整代码示例
const PartSize = 5 * 1024 * 1024;
async function hwRequest(
fileState: FileState,
file: File,
key: string,
mode: UploadMode
) {
const secret = await getHwSecret();
fileState.url = `${secret.domain}${key}`;
//创建OBS对象
const hwClient = new ObsClient({
access_key_id: secret.ak,
secret_access_key: secret.sk,
server: secret.endPoint,
timeout: 3000,
});
const params = {
Bucket: secret.bucketName,
Key: key,
Prefix: key,
};
const isCompleted = await checkIsCompleted(hwClient, params);
if (isCompleted) {
//已全部上传
fileState.percent = 100;
fileState.status = FileStatus.success;
} else {
//检查是否部分上传
const uploadId = await getHwCheckpoint(hwClient, params);
if (!uploadId) {
//首次上传
const uploadId = await initMultiPartUpload(hwClient, params);
const completeParts = [] as any;
uploadPart(fileState, file, uploadId, completeParts, hwClient, params);
} else {
//二次上传续传
const completeParts = await listAllUploadParts(
uploadId,
hwClient,
params
);
uploadPart(fileState, file, uploadId, completeParts, hwClient, params);
}
}
}