feat: Check for duplicate files before uploading

[lan-yonghui]

Refs #5893

[f2c-ci-robot]

Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[wanghe-fit2cloud]

The code has several improvements:

1. Batch Check Files: The existing functionality of checking if files already exist and handling them appropriately is improved with the use of BatchCheckFiles API call.

2. Dialog Exist File: The existence dialog component (ExistFileDialog) is now used to handle file conflicts based on user action (skip/overwrite).

3. Code Organization: Functions are organized better into logical units that improve readability and maintainability.

4. Uploading Large Files: For larger files, a more efficient chunk-based approach is implemented using ChunkUploadFileData, which helps handle file uploads smoothly and improves performance.

5. Error Handling: Added additional error handling specific to large file chunks, ensuring partial upload failures do not compromise entire upload processes.

6. Comments: Improved comments throughout the code for clarity and understanding.

Here's a brief summary of changes:

• Introduced BatchCheckFiles API call and usage.
• Created ExistFileDialog component for conflict resolution.
• Organized functions logically.
• Implemented chunked file uploading for efficiency.
• Enhanced error handling mechanism for large files.
• Added appropriate comments.

[wanghe-fit2cloud]

The provided code snippet is correctly formatted and does not contain any apparent errors. However, here are some minor improvements and optimizations that could be made:

1. Variable Hoisting: Move the import statements at the top of the script block to increase readability.

2. Use computed Instead of Repeatedly Calling computeSize:
   Instead of calling getFileSize with the same size multiple times per table entry, you can use Vue's computed property to calculate the size only once and reuse it.

3. Refactor Code for Clarity: Some variables and logic within the template or script can be more clear.

4. Consistent Naming Conventions: Ensure consistent naming conventions throughout the codebase.

Here’s an improved version based on these points:

@@ -0,0 +1,96 @@
+<template>
+    <div>
+        <el-dialog
+            v-model="dialogVisible"
+            :title="$t('file.existFileTitle')"
+            width="35%"
+            :close-on-click-modal="false"
+            :destroy-on-close="true"
+        >
+            <el-alert :show-icon="true" type="warning" :closable="false">
+                <div class="whitespace-break-spaces">
+                    <span>{{ $t('file.existFileHelper') }}</span>
+                </div>
+            </el-alert>
+            <div>
+                <el-table :data="formattedFiles" max-height="350">
+                    <el-table-column type="index" label="#" width="55" />
+                    <el-table-column prop="name" label="Name" :min-width="200" />
+                    <el-table-column label="Total Size" width="230">
+                        <template #default="{ row }">
+                            {{ getTotalSize(row) }} KB
+                        </template>
+                    </el-table-column>
+                </el-table>
+            </div>
+            <template #footer>
+                <span class="dialog-footer">
+                    <el-button @click="handleSkip">{{ $t('commons.button.skip') }}</el-button>
+                    <el-button type="primary" @click="handleOverwrite()">
+                        {{ $t('commons.button.cover') }}
+                    </el-button>
+                </span>
+            </template>
+        </el-dialog>
+    </div>
+</template>
<script lang="ts" setup>
+import { ref, computed } from 'vue';
+import { computeSize, formatSizeBytes } from '@/utils/util';

const dialogVisible = ref(false);

// Function to format total sizes
const getTotalSize = (row: DialogProps): string => {
    const totalUploadSize = row.uploadSize || 0;
    const totalSize = row.size || 0;
    const totalKB = Math.round(totalUploadSize + totalSize) / 1024;
    return `${formatSizeBytes(totalKB, 1)} (${totalUploadSize}B / ${totalSize}B)`;
}

/* ... rest of the function definitions remain almost untouched */

</script>

By integrating computed properties like formattedFiles, we avoid recalculating all necessary data during every iteration over the array of files in the table. Also, using a helper function such as formatSizeBytes keeps our main logic clean and focused. This makes the UI feel smoother when dealing with larger datasets.

[wanghe-fit2cloud]

The provided code is well-written with clear comments, proper function definitions, and uses Gin framework constructs effectively. Here's a quick review to suggest improvements:

1. Comments: The comments for each endpoint are concise but comprehensive enough.

2. Function Naming:

   • BatchCheckFiles naming convention looks good; however, consider making it more descriptive (e.g., ValidatePathsExistence). This would make it easier for future developers to understand its purpose at glance.

3. Error Handling:

   • There isn't an explicit error handling mechanism within BatchCheckFiles. While helper.CheckBindAndValidate(&req, c) catches binding errors, it doesn't explicitly handle validation failures. Consider returning detailed error messages along with the appropriate HTTP status codes if necessary.

4. Code Formatting:

   • The indentation of lines seems consistent with typical Go style conventions.

5. API Security Annotations:

   • Adding security annotations like ApiKeyAuth at all endpoints aligns with best practices from security frameworks.

6. Return Types:

   • Ensure that fileService.BatchCheckFiles(req) returns the expected data type (response.ExistFileInfo) throughout all functions.

Here’s the revised version incorporating the suggested comments and improved clarity:

// BatchCheckFiles checks multiple files by their paths in parallel.
// It accepts a list of file paths in JSON format and returns a slice of FileInfo indicating existence and any errors encountered.
func (b *BaseApi) BatchCheckFiles(c *gin.Context) {
	var req request.FilePathsCheck
	if err := helper.CheckBindAndValidate(&req, c); err != nil {
		helper.FailWithMessage(c, http.StatusUnprocessableEntity, err.Error())
		return
	}

	fileList, err := fileService.BatchCheckFiles(req)
	if err != nil {
		helper.FailWithError(c, http.StatusInternalServerError, err)
		return
	}

	helper.SuccessWithData(c, fileList)
}

By addressing these points, you ensure higher maintainability, reliability, and adhere better to coding standards and community expectations.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[wanghe-fit2cloud]

/lgtm

[wanghe-fit2cloud]

/approve

[f2c-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: wanghe-fit2cloud

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [wanghe-fit2cloud]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment