详解ASP.NET Core实现的文件断点上传下载-51CTO.COM

场景分析

文件的断点续传功能可以应用于需要上传或下载大文件的场景。在网络状况不佳或者文件较大时，一次性上传或下载整个文件可能会耗费大量时间和带宽，并且可能会导致因中断而失败的情况发生。通过实现文件的断点续传功能，可以将大文件分割成小块，分别上传或下载，即使在网络出现问题时也可以通过上传或下载已经完成的文件块来继续未完成的操作，减少耗时和带宽消耗，提高用户体验。常见的应用场景如：视频、音频文件的上传或下载，大型软件的更新或下载，云盘等文件存储服务等等。

设计思路

在asp.net core 中实现文件的断点续传功能，需要进行以下步骤：

客户端与服务端建立连接。
客户端上传文件时，将文件切割成小块并分别传输到服务端。
服务端接收到文件块后进行校验和整合。
客户端下载文件时，请求服务端指定位置开始的文件块，并按照顺序拼接成完整文件。

具体实现思路如下：

断点上传

客户端上传文件时，服务端接收到请求，在服务端创建对应的文件，并记录上传时间、文件大小等信息。
将接收到的文件块保存到物理文件中，每次上传块的时候都应该压缩这个块，以减少带宽的消耗。
服务端返回给客户端成功接收的文件块编号和文件处理情况。

断点下载

客户端下载文件时，服务端接收到请求，读取文件对应的块信息，判断客户端需要下载的文件块范围是否存在于服务端的文件块信息中。
如果该文件块不存在，则返回错误信息。
如果该文件块存在，则读取对应文件块并返回给客户端。

实现流程

断点上传

创建上传控制器

在 ASP.NET Core 应用程序中，创建一个名为 UploadController 的控制器。在该控制器中，使用 ApiControllerAttribute 特性声明该控制器为 Web API 控制器。

[ApiController]
[Route("[controller]")]
public class UploadController : ControllerBase

实现上传方法

在控制器中实现下面的代码。

[HttpPost]
public async Task<IActionResult> Upload(IFormFile file, int chunkIndex, int totalChunks)
{
    if (file == null || file.Length == 0)
    {
        return BadRequest("This request does not have any body");
    }

    // create the folder if it doesn't exist yet
    string folderPath = Path.Combine(Directory.GetCurrentDirectory(), "Uploads");
    if (!Directory.Exists(folderPath))
    {
        Directory.CreateDirectory(folderPath);
    }

    string filePath = Path.Combine(folderPath, file.FileName);

    using (FileStream stream = new FileStream(filePath, FileMode.Create, FileAccess.Write, FileShare.Write))
    {
        await file.CopyToAsync(stream);
    }

    return Ok();
}

实现断点上传逻辑

在上面的代码中，我们只是将整个文件保存到了服务器上。现在，我们需要实现断点上传的逻辑。断点上传指的是，将大文件分成多个小块，并逐个上传。

using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using System;
using System.IO;
using System.Threading.Tasks;

namespace ResumeTransfer
{
    [Route("api/[controller]")]
    [ApiController]
    public class UploadController : ControllerBase
    {
        private const string UploadsFolder = "Uploads";

        [HttpPost]
        public async Task<IActionResult> Upload(IFormFile file, int? chunkIndex, int? totalChunks)
        {
            if (file == null || file.Length == 0)
            {
                return BadRequest("This request does not have any body");
            }

            // Check if the chunk index and total chunks are provided.
            if (!chunkIndex.HasValue || chunkIndex.Value < 0 || !totalChunks.HasValue || totalChunks.Value <= 0)
            {
                return BadRequest("Invalid chunk index or total chunks");
            }

            // Create folder for upload files if not exists
            string folderPath = Path.Combine(Directory.GetCurrentDirectory(), UploadsFolder);
            if (!Directory.Exists(folderPath))
            {
                Directory.CreateDirectory(folderPath);
            }

            string filePath = Path.Combine(folderPath, file.FileName);

            // Check if chunk already exists
            if (System.IO.File.Exists(GetChunkFileName(filePath, chunkIndex.Value)))
            {
                return Ok();
            }

            using (FileStream stream = new FileStream(GetChunkFileName(filePath, chunkIndex.Value), FileMode.Create, FileAccess.Write, FileShare.Write))
            {
                await file.CopyToAsync(stream);
            }

            if (chunkIndex == totalChunks - 1)
            {
                // All the chunks have been uploaded, merge them into a single file
                MergeChunks(filePath, totalChunks.Value);
            }

            return Ok();
        }

        private void MergeChunks(string filePath, int totalChunks)
        {
            using (var finalStream = new FileStream(filePath, FileMode.CreateNew))
            {
                for (int i = 0; i < totalChunks; i++)
                {
                    var chunkFileName = GetChunkFileName(filePath, i);
                    using (var stream = new FileStream(chunkFileName, FileMode.Open))
                    {
                        stream.CopyTo(finalStream);
                    }
                    System.IO.File.Delete(chunkFileName);
                }
            }
        }

        private string GetChunkFileName(string filePath, int chunkIndex)
        {
            return $"{filePath}.part{chunkIndex.ToString().PadLeft(5, '0')}";
        }
    }
}

客户端上传

客户端使用 axios 库进行文件上传。使用下面的代码，将文件分块并管理每个文件块的大小和数量。

// get file size and name
const fileSize = file.size;
const fileName = file.name;

// calculate chunk size
const chunkSize = 10 * 1024 * 1024; // 10MB

// calculate total chunks
const totalChunks = Math.ceil(fileSize / chunkSize);

// chunk upload function
const uploadChunk = async (chunkIndex) => {
  const start = chunkIndex * chunkSize;
  const end = Math.min((chunkIndex + 1) * chunkSize, fileSize);

  const formData = new FormData();
  formData.append("file", file.slice(start, end));
  formData.append("chunkIndex", chunkIndex);
  formData.append("totalChunks", totalChunks);

  await axios.post("/upload", formData);
};

for (let i = 0; i < totalChunks; i++) {
  await uploadChunk(i);
}

断点下载

创建下载控制器

创建一个名为 DownloadController 的控制器，使用 ApiControllerAttribute 特性声明该控制器为 Web API 控制器。

[ApiController]
[Route("[controller]")]
public class DownloadController : ControllerBase

实现断点下载逻辑

在控制器中实现下面的代码。

using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using System;
using System.IO;

namespace ResumeTransfer
{
    [Route("api/[controller]")]
    [ApiController]
    public class DownloadController : ControllerBase
    {
        private const string UploadsFolder = "Uploads";

        [HttpGet("{fileName}")]
        public IActionResult Download(string fileName, long? startByte = null, long? endByte = null)
        {
            if (string.IsNullOrEmpty(fileName))
            {
                return BadRequest("Invalid file name");
            }

            string filePath = Path.Combine(Directory.GetCurrentDirectory(), UploadsFolder, fileName);

            if (!System.IO.File.Exists(filePath))
            {
                return NotFound();
            }

            long contentLength = new System.IO.FileInfo(filePath).Length;

            // Calculate the range to download.
            if (startByte == null)
            {
                startByte = 0;
            }
            if (endByte == null)
            {
                endByte = contentLength - 1;
            }

            // Adjust the startByte and endByte to be within the range of the file size.
            if (startByte.Value < 0 || startByte.Value >= contentLength || endByte.Value < startByte.Value || endByte.Value >= contentLength)
            {
                Response.Headers.Add("Content-Range", $"bytes */{contentLength}");
                return new StatusCodeResult(416); // Requested range not satisfiable
            }

            // Set the Content-Disposition header to enable users to save the file.
            Response.Headers.Add("Content-Disposition", $"inline; filename={fileName}");

            Response.StatusCode = 206; //Partial Content

            Response.Headers.Add("Accept-Ranges", "bytes");

            long length = endByte.Value - startByte.Value + 1;
            Response.Headers.Add("Content-Length", length.ToString());

            // Send the file data in a range of bytes, if requested
            byte[] buffer = new byte[1024 * 1024];
            using (FileStream stream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read))
            {
                stream.Seek(startByte.Value, SeekOrigin.Begin);
                int bytesRead;
                while (length > 0 && (bytesRead = stream.Read(buffer, 0, (int)Math.Min(buffer.Length, length))) > 0)
                {
                    // Check if the client has disconnected.
                    if (!Response.HttpContext.Response.Body.CanWrite)
                    {
                        return Ok();
                    }

                    Response.Body.WriteAsync(buffer, 0, bytesRead);
                    length -= bytesRead;
                }
            }

            return new EmptyResult();
        }
    }
}

客户端下载

客户端使用 axios 库进行文件下载。使用下面的代码，将要下载的文件拆分成小块，并按照顺序下载。

const CHUNK_SIZE = 1024 * 1024 * 5; // 5MB

const downloadChunk = async (chunkIndex) => {
  const res = await axios.get(`/download?fileName=${fileName}&chunkIndex=${chunkIndex}`, {
    responseType: "arraybuffer",
  });
  const arrayBuffer = res.data;

  const start = chunkIndex * CHUNK_SIZE;
  const end = start + CHUNK_SIZE;
  const blob = new Blob([arrayBuffer]);

  const url = URL.createObjectURL(blob);

  const a = document.createElement("a");
  a.style.display = "none";
  a.href = url;
  a.download = fileName;
  document.body.appendChild(a);
  a.click();
  URL.revokeObjectURL(url);
};

for (let i = 0; i < totalChunks; i++) {
  await downloadChunk(i);
}

单元测试

在我们的代码中实现单元测试可以确保代码的正确性，并且可以减少手工测试的负担。我们可以使用
Microsoft.VisualStudio.TestTools.UnitTesting（在 .NET Core 中，也可以使用 xUnit 或 NUnit 进行单元测试）进行单元测试。

下面是一个简单的上传控制器单元测试示例：

using ResumeTransfer;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using System.IO;
using System.Threading.Tasks;
using Xunit;
using Microsoft.Extensions.Primitives;
using Moq;

namespace ResumeTransfer.Tests
{
    public class UploadControllerTests
    {
        [Fact]
        public async Task Upload_ReturnsBadRequest_WhenNoFileIsSelected()
        {
            // Arrange
            var formCollection = new FormCollection(new Dictionary<string, StringValues>(), new FormFileCollection());
            var context = new Mock<HttpContext>();
            context.SetupGet(x => x.Request.Form).Returns(formCollection);
            var controller = new UploadController { ControllerContext = new ControllerContext { HttpContext = context.Object } };

            // Act
            var result = await controller.Upload(null, 0, 1);

            // Assert
            Assert.IsType<BadRequestObjectResult>(result);
        }

        [Fact]
        public async Task Upload_ReturnsBadRequest_WhenInvalidChunkIndexOrTotalChunksIsProvided()
        {
            // Arrange
            var fileName = "test.txt";
            var fileStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes("This is a test file."));
            var formFile = new FormFile(fileStream, 0, fileStream.Length, "Data", fileName);
            var formCollection = new FormCollection(new Dictionary<string, StringValues>
            {
                { "chunkIndex", "0" },
                { "totalChunks", "0" }
            }, new FormFileCollection { formFile });
            var context = new Mock<HttpContext>();
            context.SetupGet(x => x.Request.Form).Returns(formCollection);
            var controller = new UploadController { ControllerContext = new ControllerContext { HttpContext = context.Object } };

            // Act
            var result = await controller.Upload(formFile, 0, 0);

            // Assert
            Assert.IsType<BadRequestObjectResult>(result);
        }

        [Fact]
        public async Task Upload_UploadsChunk_WhenChunkDoesNotExist()
        {
            // Arrange
            var fileName = "test.txt";
            var fileStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes("This is a test file."));
            var formFile = new FormFile(fileStream, 0, fileStream.Length, "Data", fileName);
            var formCollection = new FormCollection(new Dictionary<string, StringValues>
            {
                { "chunkIndex", "0" },
                { "totalChunks", "1" }
            }, new FormFileCollection { formFile });
            var context = new Mock<HttpContext>();
            context.SetupGet(x => x.Request.Form).Returns(formCollection);
            var uploadsFolder = Path.Combine(Directory.GetCurrentDirectory(), "uploads");
            var controller = new UploadController { ControllerContext = new ControllerContext { HttpContext = context.Object }, UploadsFolder = uploadsFolder };

            // Act
            var result = await controller.Upload(formFile, 0, 1);
            var uploadedFilePath = Path.Combine(uploadsFolder, fileName);

            // Assert
            Assert.IsType<OkResult>(result);
            Assert.True(File.Exists(uploadedFilePath));

            using (var streamReader = new StreamReader(File.OpenRead(uploadedFilePath)))
            {
                var content = await streamReader.ReadToEndAsync();
                Assert.Equal("This is a test file.", content);
            }

            File.Delete(uploadedFilePath);
        }

        [Fact]
        public async Task Upload_UploadsChunk_WhenChunkExists()
        {
            // Arrange
            var fileName = "test.txt";
            var fileStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes("This is a test file."));
            var formFile = new FormFile(fileStream, 0, fileStream.Length, "Data", fileName);
            var formCollection1 = new FormCollection(new Dictionary<string, StringValues>
            {
                { "chunkIndex", "0" },
                { "totalChunks", "2" }
            }, new FormFileCollection { formFile });
            var context = new Mock<HttpContext>();
            context.SetupGet(x => x.Request.Form).Returns(formCollection1);
            var uploadsFolder = Path.Combine(Directory.GetCurrentDirectory(), "uploads");
            var controller = new UploadController { ControllerContext = new ControllerContext { HttpContext = context.Object }, UploadsFolder = uploadsFolder };

            // Act
            var result1 = await controller.Upload(formFile, 0, 2);

            // Assert
            Assert.IsType<OkResult>(result1);

            // Arrange
            var formCollection2 = new FormCollection(new Dictionary<string, StringValues>
            {
                { "chunkIndex", "1" },
                { "totalChunks", "2" }
            }, new FormFileCollection { formFile });
            context.SetupGet(x => x.Request.Form).Returns(formCollection2);

            // Act
            var result2 = await controller.Upload(formFile, 1, 2);
            var uploadedFilePath = Path.Combine(uploadsFolder, fileName);

            // Assert
            Assert.IsType<OkResult>(result2);
            Assert.True(File.Exists(uploadedFilePath));

            using (var streamReader = new StreamReader(File.OpenRead(uploadedFilePath)))
            {
                var content = await streamReader.ReadToEndAsync();
                Assert.Equal("This is a test file.This is a test file.", content);
            }

            File.Delete(uploadedFilePath);
        }
    }
}

性能测试

为了实现更好的性能和响应时间，我们可以使用 BenchmarkDotNet 进行性能测试，以便找到性能瓶颈并对代码进行优化。

下面是一个简单的上传控制器性能测试示例：

using BenchmarkDotNet.Attributes;
using ResumeTransfer;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Http.Internal;

namespace ResumeTransfer.Benchmarks
{
    [MemoryDiagnoser]
    public class UploadControllerBenchmarks
    {
        private readonly UploadController _controller;
        private readonly IFormFile _testFile;

        public UploadControllerBenchmarks()
        {
            _controller = new UploadController();
            _testFile = new FormFile(new MemoryStream(System.Text.Encoding.UTF8.GetBytes("This is a test file")), 0, 0, "TestFile", "test.txt");
        }

        [Benchmark]
        public async Task<IActionResult> UploadSingleChunk()
        {
            var formCollection = new FormCollection(new System.Collections.Generic.Dictionary<string, Microsoft.Extensions.Primitives.StringValues>
            {
                { "chunkIndex", "0" },
                { "totalChunks", "1" }
            }, new FormFileCollection { _testFile });
            var request = new DefaultHttpContext();
            request.Request.Form = formCollection;
            _controller.ControllerContext = new ControllerContext { HttpContext = request };

            return await _controller.Upload(_testFile, 0, 1);
        }

        [Benchmark]
        public async Task<IActionResult> UploadMultipleChunks()
        {
            var chunkSizeBytes = 10485760; // 10 MB
            var totalFileSizeBytes = 52428800; // 50 MB
            var totalChunks = (int)Math.Ceiling((double)totalFileSizeBytes / chunkSizeBytes);

            for (var i = 0; i < totalChunks; i++)
            {
                var chunkStartByte = i * chunkSizeBytes;
                var chunkEndByte = Math.Min(chunkStartByte + chunkSizeBytes - 1, totalFileSizeBytes - 1);

                var chunkFileContent = new byte[chunkEndByte - chunkStartByte + 1];
                using (var memoryStream = new MemoryStream(chunkFileContent))
                {
                    using (var binaryWriter = new BinaryWriter(memoryStream))
                    {
                        binaryWriter.Write(chunkStartByte);
                        binaryWriter.Write(chunkEndByte);
                    }
                }

                var chunkFileName = $"{_testFile.FileName}.part{i.ToString().PadLeft(5, '0')}";
                var chunkFilePath = Path.Combine(Directory.GetCurrentDirectory(), ChunkUploadController.UploadsFolder, chunkFileName);

                using (var fileStream = new FileStream(chunkFilePath, FileMode.Create, FileAccess.Write, FileShare.Write))
                {
                    await fileStream.WriteAsync(chunkFileContent);
                    await fileStream.FlushAsync();
                }
            }

            var formCollection = new FormCollection(new System.Collections.Generic.Dictionary<string, Microsoft.Extensions.Primitives.StringValues>
            {
                { "chunkIndex", "0" },
                { "totalChunks", totalChunks.ToString() }
            }, new FormFileCollection { _testFile });
            var request = new DefaultHttpContext();
            request.Request.Form = formCollection;
            _controller.ControllerContext = new ControllerContext { HttpContext = request };

            return await _controller.Upload(_testFile, 0, totalChunks);
        }
    }
}

运行上面的代码后，将会输出详细的性能测试结果。

根据测试结果，我们可以找到性能瓶颈，并对代码进行优化，以达到更高的性能和更快的响应时间。