Various ways, principles and problems encountered in uploading Spring Boot files from building development scaffolding

Various ways, principles and problems encountered in uploading Spring Boot files from building development scaffolding

Article Directory

File Upload

Overview

Spring supports pluggable

MultipartResolver
The subject uploads files. There are currently 2 implementations;

  • Before Servlet 2.5 and earlier versions, file uploads need to be realized with the help of commons-fileupload component.
  • After the Servlet 3.0 specification , native support for file upload is provided, which further simplifies the implementation of the application.

commons-fileupload

To use commons-fileupload 's CommonsMultipartResolver to handle file uploads, we need to add the following dependencies:

< Dependency > < the groupId > Commons-FileUpload </the groupId > < the artifactId > Commons-FileUpload </the artifactId > </dependency > copy the code

Configuration definition

CommonsMultipartResolver
bean.

@Bean(name = "multipartResolver") public CommonsMultipartResolver multipartResolver () { CommonsMultipartResolver multipartResolver = new CommonsMultipartResolver(); multipartResolver.setMaxUploadSize( 100000 ); return multipartResolver; } Copy code

Servlet 3.0

See

MultipartAutoConfiguration.java
Class, automatically configured by default
StandardServletMultipartResolver
, We don't need to do anything

@Configuration(proxyBeanMethods = false) @ConditionalOnClass({ Servlet.class, StandardServletMultipartResolver.class, MultipartConfigElement.class }) @ConditionalOnProperty(prefix = "spring.servlet.multipart", name = "enabled", matchIfMissing = true) @ConditionalOnWebApplication( type = Type.SERVLET) @EnableConfigurationProperties(MultipartProperties.class) public class MultipartAutoConfiguration { private final MultipartProperties multipartProperties; public MultipartAutoConfiguration (MultipartProperties multipartProperties) { this .multipartProperties = multipartProperties; } @Bean @ConditionalOnMissingBean({ MultipartConfigElement.class, CommonsMultipartResolver.class }) public MultipartConfigElement multipartConfigElement () { return this .multipartProperties.createMultipartConfig(); } @Bean(name = DispatcherServlet.MULTIPART_RESOLVER_BEAN_NAME) @ConditionalOnMissingBean(MultipartResolver.class) public StandardServletMultipartResolver multipartResolver () { StandardServletMultipartResolver multipartResolver = new StandardServletMultipartResolver(); multipartResolver.setResolveLazily( this .multipartProperties.isResolveLazily()); return multipartResolver; } } Copy code

Common file upload related requirements, I summarized and summarized as follows:

Single file upload

Front-end core code

< form method = "POST" action = "/upload-file" enctype = "multipart/form-data" > < table > < tr > < td > < input type = "file" name = "file"/> </td > </tr > < tr > < td > < input type = "submit" value = "Submit"/> </td > </tr > </table > </form > Copy code

Back-end core code

@RequestMapping(value = "/upload-file", method = RequestMethod.POST) public String submit ( @RequestParam("file") MultipartFile file) { return "ok" ; } Copy code

Multiple file upload

Front-end core code

< form method = "POST" action = "/upload-files" enctype = "multipart/form-data" > < table > < tr > < td > Select a file to upload </td > < td > < input type = "file" name = "files"/> </td > </tr > < tr > < td >Select a file to upload </td > < td > < input type = "file" name = "files"/> </td > </tr > < tr > < td > Select a file to upload </td > < td > < input type = " file" name = "files"/> </td > </tr > < tr > < td > <INPUT type = "Submit" value = "the Submit"/> </TD > </TR > </Table > </form > copy the code

Back-end core code

We need to note that each input field has the same name so that it can be accessed as a MultipartFile array:

@RequestMapping(value = "/upload-files", method = RequestMethod.POST) public String submit ( @RequestParam("files") MultipartFile[] files) { return "ok" ; } Copy code

File upload with other parameters

Front-end core code

< form method = "POST" action = "/upload-files-with-data" enctype = "multipart/form-data" > < table > < tr > < td > Name </td > < td > < input type = "text" name = "name"/> </td > </tr > < tr > < td >Email </td >< td > < input type = "text" name = "email"/> </td > </tr > < tr > < td > Select a file to upload </td > < td > < input type = "file" name = "file"/> </td > </tr > < tr > <td > < input type= "Submit" value = "the Submit"/> </TD > </TR > </Table > </form > copy the code

Back-end core code

In the controller, we can use the @RequestParam annotation to get all form data, or we can get it without @RequestParam

@PostMapping("/upload-files-with-data") public String submit ( @RequestParam MultipartFile file, @RequestParam String name, String email) { return "ok" ; } Copy code

Elegant back-end implementation

We can also encapsulate all form fields in a class, which is very convenient when there are many other fields in the file.

public class FormDataWithFile { private String name; private String email; private MultipartFile file; } @PostMapping("/upload-files-with-data") public String submit (FormDataWithFile formDataWithFile) { return "ok" ; } Copy code

Multiple (file + parameters) upload

The functional requirements are similar to uploading the following requests:

[ { "name" : "a" , "emainl" : "b" , "file" : }, { "name" : "a" , "emainl" : "" , "file" : } ] Copy code

But this is not working, the solution is as follows:

Solution 1: Upload file Base64

Convert the file to a base64 string, but the size of the converted string is 3 times the size of the original image. (Use with caution)

[ { "name" : "a" , "emainl" : "" , "fileBase64" : "xxxxx" }, { "name" : "b" , "emainl" : "" , "fileBase64" : "xxxxx" } ] Copy code

Option 2: upload file url

Upload the picture to the server first , get the file url, and then upload the file URL and other parameters to the backend

[ { "name" : "a" , "emainl" : "" , "fileUrl" : "xxxxx.png" }, { "name" : "b" , "emainl" : "" , "fileUrl" : "xxxxx.png" } ] Copy code

File upload principle

Usually the format of the request content for a file upload is as follows:

POST/upload HTTP/1.1 Host :xxx.org Content-type: multipart/form-data, boundary= "boundaryStr" --boundaryStr content-disposition: form-data; name= "name" Name Of Picture --boundaryStr Content-disposition: attachment; name= "picfile" ; filename= "picfile.gif" Content-type: image/gif Content-Transfer-Encoding: binary ...contents of picfile.gif... copy code

Where boundary specifies the boundary string of content segmentation;

Content-dispostion specifies that this is an attachment (file), including the parameter name and file name;

Content-type specifies the file type;

Content-Transfer-Encoding specifies the content transfer encoding;

Tomcat implements the Servlet3.0 specification and encapsulates the file upload stream through ApplicationPart . DiskFileItem describes the upload file entity, which is generated when the request is parsed. It should be noted that DiskFileItem declares a temporary file for temporary storage. The content of the uploaded file, SpringMVC encapsulates the upper request entity again, and finally is constructed as a MultipartFile and passed to the application.
Examples are as follows:

The temporary files generated are as follows:

This is the directory of temporary files, which can be configured


Open the temporary file and check its contents as follows:

  • Parameters: name
  • Parameters: After the file

    upload is complete, the temporary file will be deleted

As you can see, parameters that are not file types are also written to the temporary file.

Capture packets through Fiddler :

POST http://localhost:8080/upload-files-with-data HTTP/1.1 cache-control : no-cache Accept : */* Host : localhost:8080 accept-encoding : gzip, deflate content-type : multipart/form -data; boundary=--------------------------895818005136536360125479 content-length : 268707 Connection : keep-alive ----------------------------895818005136536360125479 Content-Disposition: form-data; name = "name" 123 ----------------------------895818005136536360125479 Content-Disposition: form-data; name = "file" ; filename = "test.txt " Content-Type: text/plain abc123 ----------------------------895818005136536360125479 Content-Disposition: form-data; name = "file" ; filename = "1114289-20190110120111312- 1475461850.png" Content-Type: image/png ... contents of png... ----------------------------895818005136536360125479-- copy code

At this point, we probably know the principle of HTTP uploading files. HTTP stores all the data of the form that needs to be uploaded in the request body in a certain format, and the same is true for files.

  • Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryqj67FUBQUHXZj78G
    Indicates that you want to upload an attachment,
  • among them
    boundary
    Indicates the separator, if there are multiple items in the form, use
    boundary
    Is separated, and each form item is
    ------FormBoundary
    Start with
    ------FormBoundary
    end. For example:
------FormBoundary Content-Disposition: form-data; name="param1" value1 ------FormBoundary Copy code

This one

boundary
The value of is generated by the browser, and the browser guarantees that it is not duplicated with the uploaded content.

  • In each division, we need to focus on
    Content-Disposition
    Message header, the first parameter is always fixed
    form-data
    , Name represents the attribute name of the form element, and the content after the carriage return and line feed is the value of the element. and also
    Content-Type
    Indicates the MIME type of the file we upload, and we need to distinguish the file based on this on the server side.
  • the last one
    boundary
    Will end with two more
    -

According to this format, HTTP encapsulates the data in the form into a request and sends it to the server. The server parses the received request according to this rule to complete the file upload function.

The following is an example of back-end analysis found on the Internet. You can DEBUG tracking code to analyze.

@WebServlet(urlPatterns = "/lakerfile") public class FileUploadDemo extends HttpServlet { @Override public void doPost (HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { DiskFileItemFactory fac = new DiskFileItemFactory(); ServletFileUpload upload = new ServletFileUpload(fac); upload.setFileSizeMax( 10 * 1024 * 1024 ); upload.setSizeMax( 20 * 1024 * 1024 ); if (ServletFileUpload.isMultipartContent(request)) { //Only process Multipart requests List<FileItem> list = upload.parseRequest( new ServletRequestContext(request)); //parse the message for (FileItem item: list) { if (item.isFormField()) { String fileName = item.getFieldName(); String value = item.getString( "UTF-8" ); } else { File file = new File(realPath, name); item.write(file); ... } } } Copy code

Problems encountered

Spring Boot upload file size limit

spring: servlet: multipart: # Maximum file size (single) max-file-size: 10MB # When the file is larger than the threshold, it will be written to the disk, supporting B/KB/MB unit file-size-threshold: 0B #//Max requested size (overall) max-size-request: 100MB copy the code

These parameters are controlled by SpringMVC and used to inject the file upload configuration of Servlet3.0. The associated classes are as follows:

public class MultipartConfigElement { Private Final String LOCATION; //= ""; Private Final Long the maxFileSize; //= -1; Private Final Long maxRequestSize; //= -1; Private Final int fileSizeThreshold; //= 0; duplicated code

The upload file is too large to be blocked

@ExceptionHandler(MaxUploadSizeExceededException.class) public Response handleMaxSizeException (MaxUploadSizeExceededException e) { log.error(e.getMessage(), e); return Response.error( 500 , "File too large!" ); } Copy code

Custom tomcat working directory

Custom temporary file generation directory

Server: Tomcat: basedir: /Laker/tmp copy the code

Uploading files using swagger does not work

  • allowMultiple=true
    : Indicates that it is an array format parameter
  • dataType = "__file"
    : Indicates the type of the parameter in the array
@ApiOperation(value = "Upload", notes = "Upload") @ApiImplicitParams({ @ApiImplicitParam(paramType = "form", name = "file", value = "file object", required = true, dataType = "__file"), @ApiImplicitParam(paramType = "form", name = "files", value = "file array", allowMultiple = true, dataType = "__file") }) public void test ( @RequestParam("file") MultipartFile file, @RequestParam(value = "files", required = false) MultipartFile[] files) throws Exception { } Copy code

reference:

QQ Group [837324215]
Follow my public account [ Java Interviewer ], learn together