v0.3.0

A release that adds support for using clp-json to both compress logs from object storage and store archives on object storage (docs). This release also includes some other features and bug fixes.

NOTE: We are bumping the minor version due to a breaking format change in clp-text/clp-json’s jobs table format and in clp-json/clp-s’ archive format. If this affects you, reach out and we can help with version migration.

The CLP release includes two tars:

• clp-json for compressing and searching JSON logs
• clp-text for compressing and searching unstructured text logs

Docs

The docs for this release are available here.

New features

• clp-json: Add support for compressing logs from S3. (#651)
• clp-json: Add support for storing, decompressing, and searching archives from S3. (#634, #674, #683)
• clp-json: Add support for viewing logs from S3. (#662, #673, #678)
• clp-s: Add support for compressing logs from S3. (#639)
• clp-s: Add support for writing single-file archives. (#563)
• clp-s: Add support for reading and searching single-file archives, including from S3. (#656)
• Add BoundedReader to prevent out-of-bound reads in segmented input streams. (#624)

Bug fixes & improvements

• clp-json: Add option to output search results as raw logs. (#641)
• clp-package: Unify the metadata schema for JSON and IR streams. (#620)
• clp-package: Enable replica set for the MongoDB results cache and configure it when starting the package. (#632)
• clp: Advance to the next message when a message has an out-of-range timestamp when searching archives (fixes #659). (#660)
• clp-s: Unescape string values during ingestion and fix support for search using escape sequences. (#622)
• clp-s: Improve error reporting for directory-creation failure during compression. (#671, #684)
• clp-s: Rename tables section to use segment numbering scheme. (#666)
• Handle 0-byte reads when BufferReader's underlying buffer is fully consumed. (#687)
• Add missing libcurl4 dependency to clp-core and package execution containers. (#670)
• Disable file system translation in checkinstall during dependency installation (fixes #642). (#644)

View the full changelog for more details.

Thanks to @AVMatthews, @Bill-hbrhbr, @Eden-D-Zhang, @LinZhihao-723, @gibber9809, @haiqi96, @jackluo923, @junhaoliao, and @kirkrodrigues for their contributions.