3.7

Highlights

• Reading, writing, and listing archives
• Multi-object transactions; multi-object ETL (in addition to previously supported full bucket-to-bucket transforms)
• "Easy URL"
• New and improved CLI
• New documentation website
• Bug fixes and performance improvements

TAR, TGZ, and ZIP archives

• unify/reuse list-range and archiving logic !4483
• revise xaction renewal process; propagate archival UUID back to client !4488
• unify archiving with existing multi-object operations !4484
• CLI: extend PUT to support archive creation !4485
• archive: support multiple mime types; add zip test !4491
• naming iniside archives: by default, do not include source bucket name !4495
• archive: ref count targets on a per archival transaction basis !4497
• archive: error processing; end-of-iteration !4500
• list objects API: new option to list archived content !4536
• archive: support TGZ !4541
• support full path via archpath query parameter !4543
• archive: nest (tar, tgz) writers !4544
• archive: add checksumming for all supported types !4547
• add checksum-sizer; archive: add basic stats; eliminate fseek !4548
• multi-obj archiving: transactions vs reusable xaction !4550
• archive: add a file to a TAR archive !4553
• archive: API naming, message flags !4556
• archive CLI: make it work; docs: add multi-object operations !4561
• CLI: add top level archive command !4570
• Blog: an article on how to properly append to an existing TAR !4571
• archive: finalize and cleanup logic; consistent renames !4575
• archive: transactions, control flags, and error handling !4576
• archive: allow multiobject operation to append to existing archive !4580

Multi-object Transform and Copy

• multi-object transform & copy and archive: ref-count targets, unify quiescence logic !4511
• multi-object transform (initial); consistent renames !4555
• add multi-object (list | range) transformation !4565
• multi-object copy & transform: multiple transactions, single xaction !4569
• transform & copy, bucket & multi-obj: consistent naming !4572
• transform & copy, bucket & multi-obj: continued ref !4573
• transform & copy multi-obj: ref-count transactions !4574
• multi-object copy & transform: refactor API, add stress !4581
• multi-object copy & transform: add utils !4583
• ETL: forward Init call to the primary !4591

Multi-object operations (common)

• move/rename list and range xops (ref) !4459
• list and range xops (refactoring part two) !4461
• revise list and range operations !4464
• list and range ops: continued refactoring !4469
• more rigorous idleness check; unique transport endpoint (major) !4595

eXtended Actions (xactions)

• xaction registry: remove finished (ref) !4442
• enforce unique xaction ID !4449
• consolidate xactions (ref) !4462
• consolidate xactions (part two) !4463
• remove/simplify cluster.XactID (ref) !4474
• remove xaction.Args - simplify (ref) !4475
• xactions: continued refactoring !4476
• renewable xaction (w/ code reduction) !4515
• renewable xaction (part two) !4516
• renewable xaction (part three: WPR) !4517
• rename on-demand xaction !4518
• renewable xaction (part four) !4520
• renewable xaction (part five) !4521
• renewable xaction (part six) !4522
• renewable xaction (part seven) !4523
• renewable xaction (part eight) !4524
• renewable xaction (part nine) !4526
• rename/clarify xaction scope !4527
• renewable xaction (part ten) !4528
• simplify initialization of on-demand xactions !4529
• EC: xaction renewals !4533
• renewable xaction (part eleven) !4534
• improve synchronicity when starting to run (distributed) xactions !4535
• wait for rebalance to start (in re: improve synchronicity) !4538
• xactions: improve synchronicity (LRU, resilver) !4546

Intra-cluster Transport

• data mover stages; reduce unreg-recv delay !4545
• consistency in copying obj attrs to/from http and transport headers !4452
• transport streams: send completion vs freeing obj headers !4496
• transport/streams: reduce/optimize header size, add SID/opcode; archive: send-done !4505
• transport: refactor receive logic; use slab for headers; monotime !4507
• transport Rx: simplify eof-ok !4508

Documentation

• docs: add a section on increasing disk priority for aisnode !4470
• docs: setting CPU governor to performance value !4471
• CLI docs: update bucket props command !4506
• docs: major restructuring to place all documentation in docs directory !4514
• docs: bucket properties and property inheritance !4551
• docs: make documentation a website !4554
• website: replace images links with embedded youtube videos !4557
• docs: move 'info for developers' out from user cli documentation !4558
• website: various visual improvements !4559
• website: remove posts header !4560
• website: refactore and fix redirections and images !4563
• website: add author field to blog post !4564
• website: fix tables and enumerations !4568
• docs: revamp RESTful API doc !4590

Tests

• tests: allow larger number of idle connections so we can reuse them !4445
• make archive test more stressful !4562
• tests: fix PrefetchList and PrefetchRange !4466
• add multi-objects test !4567
• update archiveListRange test, make generated obj names more predictable !4468
• fix object props test for aws and a bucket with disabled versioning !4478
• test: remove hidden object path augmentation !4486
• archive stress-test improvements !4499
• add functional test for transforming object with GET request !4531
• fix LRU unit test; check capacity consistently with existing code !4539

Bug fixes and improvements

• revise/refactor LRU; xaction-being-renewed; misc renames !4537
• revise housekeeper !4443
• GFN: remove locks, revise, simplify !4444
• unify object metadata, transport and ETL headers (part one) !4446
• unify object metadata, transport and ETL headers (part two) !4447
• checksum utils: add accessors, optimize !4451
• object custom MD: send/receive and store upon migration (part one) !4453
• not-found error: add source node for context !4454
• revise head-object: target (incl. S3) and api package !4455
• cleanup metafile if a bucket is deleted while EC is saving metadata !4456
• s3 compatibility: reuse target.getObject, eliminate copy-paste !4457
• set custom object properties (part one) !4458
• move EC file type constants to fs package !4460
• add lom.HrwTarget helper (ref) !4465
• handle bucket-already-exists when creating one !4467
• consistency in handling unmarshal errors !4473
• gcp client singleton !4479
• aws client per region !4480
• backend API changes: context (part one) !4481
• backend API changes: context (part two) !4482
• CLI: new options for PUT object operation !4487
• assorted TODO fixes !4490
• xreg args by value !4492
• lint/revive: enable unused-parameter !4493
• "easy URL" - aka alternative AIS API mapping (feature) !4494
• tweak initializing remote backend (ref, race) !4498
• ETL executor implementation (initial) !4501
• ETL: revise bucket copy&transform (major) !4502
• CLI: add subcommand 'ais bucket prop' !4503
• follow-up: aborting prior to starting to run !4504
• slab-allocator: add AllocSize !4509
• CLI: TAB-TAB to list only active cloud providers !4510
• EC: use Hdr.SID instead of internal EC.intraReq.sender !4512
• EC: use transport.Hdr.Opcode instead of intraReq.action !4519
• dSort: remove redundant field in remoteRequest msg !4525
• CVE fix: replace JWT library !4530
• LRU: remove on-demand logic; archive: no need to quiesce data mover !4540
• notifications: getting status and err vs rlocks !4542
• CLI: minor refactoring and cleanup; error handling !4549
• CLI: add bucket lru subcommand; show bucket props by default !4552
• EC recv meta: fix error handling !4566
• debug modules; logs; persistent markers !4577
• node name !4578
• glog not to flush in a separate goroutine; minor ref !4579
• CI: add newest Go version to be used in GitHub actions !4586
• revise quiescence logic; on-demand stats !4587
• build: upgrade fuse pkg !4588
• CLI ais bucket cp: support --list and --template options !4589
• CLI ETL: support --list and --template options !4593