ecs compat default fix #15499
Open
ecs compat default fix #15499
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Update minor plugin versions in gemfile lock Co-authored-by: logstashmachine <[email protected]> Co-authored-by: Kaise Cheng <[email protected]>
(cherry picked from commit 04fd084) Co-authored-by: João Duarte <[email protected]>
The DeadLetterQueueWriter#updateOldestSegmentReference was sorting the existing segments by the lexicographic file names. For example, if there are a few segments on the DQL path with index 8, 9 and 10, it was choosing the 10 as the oldest segment instead of 8. This commit changed it to comparing the segment ID as numbers, so the file deletion will happen following the proper order. (cherry picked from commit 4990918) Co-authored-by: Edmo Vamerlatti Costa <[email protected]>
Co-authored-by: logstashmachine <[email protected]> Co-authored-by: Kaise Cheng <[email protected]> (cherry picked from commit db86da6) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update Puma dependency (cherry picked from commit afc50eb) Co-authored-by: Andrea Selva <[email protected]>
Revert PR #15267 which updated Puma in Gemfile.template and correctly place it in logstash-core.gemspec, beeing a dependency of Logstash core. (cherry picked from commit 58e23fb) Co-authored-by: Andrea Selva <[email protected]>
….0 (#15269) (#15270) (cherry picked from commit fe1b1ed) Co-authored-by: Rob Bavey <[email protected]>
* deps: downgrade jruby, keep updated default-gem dependencies By downgrading JRuby to 9.4.2.0 we avoid the silent global crash of the scheduler backing `Concurrent::TimerTask` that occurs when Jruby 9.4.3.0's invokedynamic promotes a method to run natively, incorrectly. Upstream bug: jruby/jruby#7904 Along with the downgrade of JRuby itself to 9.4.2.0, we cherry-pick the updates to gems that were included in the latest JRuby 9.4.3.0 to ensure we don't back out relevant fixes to stdlib. We also remove a pinned-dependency on `racc` that is no longer relevant. Resolves: #15282 * Imported the licenses for some gems - cgi - date - ffi-binary-libfixposix - io-console - net-http - net-protocol - reline - time - timeout - uri * specs: avoid mocking global ::Gem::Dependency::new * build: remove redundanct dependsOn declaration * deps: notice use of ffi-binary-libfixposix via Ruby license this gem is tri-licensed `Ruby` / `EPL-2.0` / `LGPL-2.1-or-later` and the Ruby license is preferred to EPL when available --------- Co-authored-by: andsel <[email protected]>
* Forward port 8.9.2 release notes to 8.10
Co-authored-by: Karen Metts <[email protected]>
…5296) This commit adds a call to identify serverless cluster before fetching `/_xpack` to verify the license and supported features. When it is serverless, the license checker uses hardcoded xpack info instead of calling xpack API. The internal pipeline of monitoring is disabled and logs error when configured to use legacy monitoring. Fixes: ingest-dev#2303 ingest-dev#2284 Co-authored-by: Rob Bavey <[email protected]> (cherry picked from commit 6b2fa20) Co-authored-by: kaisecheng <[email protected]>
This commit adds null guard to get the native thread when constructing pipeline report Fix: #15300 (cherry picked from commit cd78558) Co-authored-by: kaisecheng <[email protected]>
Co-authored-by: João Duarte <[email protected]>
Since the DRA build for 8.10.1 was made with 82daae8 , this fix didn't get in.
* Update patch plugin versions in gemfile lock * Update Gemfile.jruby-3.1.lock.release Remove `universal-java-11` dependency. Co-authored-by: Ry Biesemeyer <[email protected]> --------- Co-authored-by: logstashmachine <[email protected]> Co-authored-by: Mashhur <[email protected]> Co-authored-by: Ry Biesemeyer <[email protected]>
* Update release notes for 8.10.2 * Update 8.10.2 release notes. --------- Co-authored-by: logstashmachine <[email protected]> Co-authored-by: Mashhur <[email protected]>
…e JVM (#15210) (#15350) Expands the description of memory used by Logstash, dividing the heap and non-heap; describing in details which parts composes the non-heap, how to size it and list the JVM settings that can be used to properly define this space. Co-authored-by: Karen Metts <[email protected]> Co-authored-by: João Duarte <[email protected]> (cherry picked from commit b31c429) Co-authored-by: Andrea Selva <[email protected]>
* Update patch plugin versions in gemfile lock * Apply suggestions from code review Remove unnecessary `universal-java-11` dependency. --------- Co-authored-by: logstashmachine <[email protected]> Co-authored-by: Mashhur <[email protected]>
* Update patch plugin versions in gemfile lock * Apply suggestions from code review Remove unnecessary universal-java-11 dependency --------- Co-authored-by: logstashmachine <[email protected]> Co-authored-by: Mashhur <[email protected]>
* Add known imap and email plugin issues section to Logstash 8.10+ versions. Co-authored-by: Mashhur <[email protected]> Co-authored-by: Karen Metts <[email protected]>
PR#15392 (which was a backport to `8.10` branch for the Buildkite DRA scripts) had a bug. This commit moves the dra scripts under the right directory as used in the other branches.
(cherry picked from commit c0e8121) Co-authored-by: Andrea Selva <[email protected]>
This is a backport of the initial Pull Request pipeline for Buildkite. While currently we haven't migrated all PR jobs from Jenkins, this is needed so PRs against non `main` branches don't fail this step (also giving us the possibility to test functionality against non `main` branches). Relates: - #15402 - #15413 - #15415 - #15421 - elastic/ingest-dev#1721 ## Release notes [rn:skip]
This commit adds Java unit tests (including sonar scans) to the Buildkite pull request pipeline. Relates: elastic/ingest-dev#1721 (cherry picked from commit e285425) Co-authored-by: Dimitrios Liappis <[email protected]>
This commit adds integration tests to the Buildkite PR pipeline. Relates: - #15437 - elastic/ingest-dev#1721 - #15279 (cherry picked from commit 816d7e6) Co-authored-by: Dimitrios Liappis <[email protected]>
This commit adds the remaining steps for the Buildkite pull request pipeline i.e. - `IT Persistent Queues / part 1` - `IT Persistent Queues / part 2` - `x-pack unit tests` - `x-pack integration` Once merged we will be able to retire the corresponding Logstash pull request multijob. Relates: - #15438 - #15437 - elastic/ingest-dev#1721 - #15279 (cherry picked from commit 555a8ff) Co-authored-by: Dimitrios Liappis <[email protected]>
Fix a few typos in the Logstash README.md (cherry picked from commit 784ca02) Co-authored-by: Dimitrios Liappis <[email protected]>
* Update release notes for 8.10.4 Co-authored-by: Rob Bavey <[email protected]> Co-authored-by: Karen Metts <[email protected]>
This commit adds a skeleton Buildkite pipeline for aarch64 exhaustive tests. - #15459 - elastic/ingest-dev#1724 (cherry picked from commit 0083738) Co-authored-by: Dimitrios Liappis <[email protected]>
This commit is the first part of migrating away the aarch64 Jenkins jobs to Buildkite. It adds a group of exhaustive test steps in the aarch64 pipeline. The java unit tests are temporarily disabled as they run SonarQube scans which need to be associated with pull requests. Relates: elastic/ingest-dev#1724 (cherry picked from commit 36656de) Co-authored-by: Dimitrios Liappis <[email protected]>
Current (8.10) docs incorrect for default value of pipeline.ecs_compatibility and supplemental text. v8.4 docs are correct, but all versions after are incorrect. Updated to reflect 8.4 doc.
Hythloday-zero
changed the title
💚 Build Succeeded
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Release notes
[rn:skip]
What does this PR do?
This PR updates the documentation for pipeline.ecs_compatibility setting to reflect the new default setting of 'v8' in Logstash 8.x.
Why is it important/What is the impact to the user?
This PR clarifies a default setting changes between Logstash 7 and 8. Users from Logstash 7 had pipeline.ecs_compatibility: disabled set as the default, as the v1 and v8 values were Beta features. This can cause pipeline logic to no longer work as intended when the default in Logstash 8 is set to 'v8'.
By making users aware of this change via documentation, they are able to account for the setting either globally in the logstash.yml file or on a per-pipeline/filter processor basis.
Checklist
- [ ] My code follows the style guidelines of this project- [ ] I have commented my code, particularly in hard-to-understand areas- [ ] I have added tests that prove my fix is effective or that my feature worksAuthor's Checklist
How to test this PR locally
Create a Logstash 8 instance. Create a pipeline with a filter phase that has the following conditional:
clone {
clones => [ "compat-check" ]
}
if [type] == "compat-check" {
mutate {
add_field => { "ecs_check" => "true"
}
}
In the Input phase, simulate/read any document, and Output to Elasticsearch.
Using the default setting for pipeline.ecs_compatibility, you will receive 2 documents to the index, neither of which will contain the added field of 'ecs_check'.
Set pipeline.ecs_compatibility: disabled in the logstash.yml and restart the Logstash instance. Run the document through the pipeline again, and you will have 2 documents in the index. This time, one will have the 'ecs_check' field set to 'true' and the other doc will not have the field.
Related issues
Use cases
Screenshots
Logs