Syncing latest changes from master for noobaa-core #18
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Bucket Notifications - phase 1 Signed-off-by: Amit Prinz Setter <[email protected]>
Signed-off-by: Utkarsh Srivastava <[email protected]> add configurable for bucket name length Signed-off-by: Utkarsh Srivastava <[email protected]>
… usage Signed-off-by: Utkarsh Srivastava <[email protected]> cleanup code Signed-off-by: Utkarsh Srivastava <[email protected]> fix reset method and rename config params Signed-off-by: Utkarsh Srivastava <[email protected]> handle overflow case Signed-off-by: Utkarsh Srivastava <[email protected]> Co-authored-by: Guy Margalit <[email protected]>
…der-partial-encoded [NSFS] Fix Newline Reader to work with partial reads and improve its memory usage
Signed-off-by: Ashish Pandey <[email protected]>
bucket_api: Pagination support for list_bucket API
Signed-off-by: Romy <[email protected]>
…resigned-url Fix pre-signed url issues
…latest version Signed-off-by: nadav mizrahi <[email protected]>
…ile-version use delete_version src file instead of move_to_dest dest_file to get latest version
Update the Dockerfiles Signed-off-by: liranmauda <[email protected]>
Update the Dockerfiles
…or concurrecy retries Signed-off-by: nadav mizrahi <[email protected]>
…onditions NSFS | versioning | remove seperation between GPFS and POSIX errors for concurrency retries
noobaa#8501) Signed-off-by: Amit Prinz Setter <[email protected]>
Signed-off-by: naveenpaul1 <[email protected]>
NSFS | Healthcheck is not reporting error for buckets with out access
Signed-off-by: Romy <[email protected]>
…resigned-url Presigned URL - Expiry limit from milliseconds to seconds and epoch fixes + automatic tests
In last PR for pagination - noobaa#8475, I had added one empty file by mistake. Removing it now. Signed-off-by: Ashish Pandey <[email protected]>
noobaa-core: Remove empty file
`unique` was removed from `shared_ptr` This PR removes `unique` and the unused `NAN_SET_BUF_DETACH` Signed-off-by: liranmauda <[email protected]>
…d-c-function Removing `unique` and the unused `NAN_SET_BUF_DETACH`
Signed-off-by: Romy <[email protected]>
…-upgrade-script NC | Online Upgrade | Config directory restructure upgrade script
Signed-off-by: shirady <[email protected]>
Add S3 GetObjectAttributes API Implementation
Signed-off-by: shirady <[email protected]>
NC | NSFS | Versioning | Avoid Errors On Put Object of Directory Content
…nkedContentDecoder)
1. In http_utils.js accept more types of content sha256 headers (STREAMING-UNSIGNED-PAYLOAD-TRAILER, STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER), as without those headers on clients that add the checksum headers with trailing we would fail.
2. Change the state of the machine and add more states to support the trailing headers: STATE_READ_TRAILER (like we have STATE_READ_CHUNK_HEADER), STATE_WAIT_NL_TRAILER (like we have STATE_WAIT_NL_DATA) and STATE_WAIT_NL_END.
3. Set the following constants to limit the request (avoid the client from abuse):
- MAX_CHUNK_SIZE - we want to have a lower number than Number.MAX_SAFE_INTEGER (reference), we expect lower number.
- MAX_CHUNK_HEADER_SIZE - we don't expect that long (as it is saved in memory during the parsing).
- MAX_TRAILER_SIZE - same, in the example we saw it was about ~30 (x-amz-checksum-crc32:uOMGCw==).
- MAX_TRAILERS - currently we saw the trailer of checksum (x-amz-checksum-crc32:uOMGCw==), we expect to have a few trailers in a request.
4. Refactor and organize - add comments with explanations about the state machine, add helper functions, add separation between the parts, rename chunk_header_str to chunk_header, add members related to trailers, add the member this.stream_pos which we use for validation.
5. Improve building the string (either this.chunk_header, and this.trailer) so we won't build it byte by byte, but only after we find the CR ('\r`).
6. Replace buffer slice function with subarray as the function slice was deprecated (see reference).
Co-authored-by: Guy Margalit <[email protected]>
Signed-off-by: shirady <[email protected]>
FIx Request Handling on Requests with Trailing Headers (Mainly in `ChunkedContentDecoder`)
Signed-off-by: jackyalbo <[email protected]>
Update AWS_API_Compatibility.md - Adding CORS
Signed-off-by: nadav mizrahi <[email protected]>
…ructure NSFS | versioning | add direcory content versioning PUT action
… Phase 2 - Code maintenance: Clean dead code from net_utils Signed-off-by: liranmauda <[email protected]>
Code maintenance | Clean dead code from net_utils | nope-ip removal - Phase 2
…gram Signed-off-by: shirady <[email protected]>
Docs | Add `ChunkedContentDecoder` Documentation with State Machine Diagram
Signed-off-by: shirady <[email protected]>
NC | NSFS | CLI | Separate Bucket and Account List Functions
Signed-off-by: Romy <[email protected]>
NC | CLI | Fix for unsettable flags issues
…llocation Signed-off-by: nadav mizrahi <[email protected]>
…guration NSFS | add configuration flag to enable dynamic supplemental groups allocation
…en deleted and also for other operations Signed-off-by: Aayush Chouhan <[email protected]>
NC | CLI | Updated the CLI response to have more info when account/bucket has been deleted
`stun.js`:
- Replaced `node-ip` with native `net_utils` calls
`http_utils.js`:
- Refractored `no_proxy_list`
- Replaced `node-ip.cidr` with native `net_utils.is_cidr` call
`net_utils.js`:
- Fixed a bug where `is_fqdn("loclhost")` returned falsely true
- Add `is_cidr`, `ip_toString`, and `ip_toBuffer`
- Added helper functions for `ip_toBuffer`
- Adding `test_net_utils.test.js`
Signed-off-by: liranmauda <[email protected]>
nope-ip removal - Phase 3
Signed-off-by: shirady <[email protected]>
NC | Test Files | Remove `nsfs` From File Name and Update Docs
Signed-off-by: jackyalbo <[email protected]>
Setting active_root_key parallel to loading key
Signed-off-by: Romy <[email protected]>
…nt-api
Add http://{endpoint_address}:{s3_port}/_/version api to the endpoint
…optional Signed-off-by: Romy <[email protected]>
…ional NC | Online upgrade | convert --expected_hosts flag from required to optional
…T if there were no failures (noobaa#8765) Signed-off-by: Amit Prinz Setter <[email protected]>
…oad version-id Signed-off-by: nadav mizrahi <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR containing the latest commits from master branch