Skip to content

Commit

Permalink
Split out dashboard vs stack monitoring
Browse files Browse the repository at this point in the history
  • Loading branch information
robbavey committed Nov 2, 2023
1 parent 1930f04 commit b8bbd03
Show file tree
Hide file tree
Showing 7 changed files with 575 additions and 423 deletions.
19 changes: 19 additions & 0 deletions docs/static/monitoring/monitoring-confirm.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
[float]
[[confirm-enrollment]]
//tag::confirm-enrollment-widget[]

* Once enrollment is confirmed and incoming data is received, click **View assets** to access dashboards related to the Logstash integration.
* For traditional Stack Monitoring UI, the dashboards marked **[Logs Logstash]** are used to visualize the logs
produced by your Logstash Instances, with those marked **[Metrics Logstash]** for the technical preview metrics
dashboards.
** Note that these will only be populated with data if the **Metrics (Technical Preview)** checkbox was
selected.
--
[role="screenshot"]
image::images/integration-assets-dashboards.png[Integration assets]
--

A number of dashboards are included to view Logstash as a whole, and dashboards that allow you to drill-down
into how Logstash is performing on a node, pipeline and plugin basis.
//end::confirm-enrollment-widget[]
132 changes: 132 additions & 0 deletions docs/static/monitoring/monitoring-ea-dashboards.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
[[monitoring-with-elastic-agent-dashboards]]
=== Collect {ls} monitoring data with {agent} for dashboards
[subs="attributes"]
++++
<titleabbrev>{agent} collection</titleabbrev>
++++

You can use {agent} to collect data about {ls} and ship it to the monitoring
cluster. The benefit of {agent} collection is that the monitoring agent remains
active even if the {ls} instance does not. Plus you can manage all your
monitoring agents from a central location in {fleet}.

To collect and ship monitoring data:

. <<disable-default-include,Disable default collection of monitoring metrics>>
. <<define-cluster__uuid-include,Specify the target `cluster_uuid` (optional)>>
. <<configure-metricbeat,Install and configure {agent} to collect monitoring data>>

[float]
[[disable-default-include]]
//include section about disabling default collection from the metricbeat topic
include::monitoring-mb.asciidoc[tag=disable-default-collection]

[float]
[[define-cluster__uuid-include]]
//include section about defining cluster_uuid from the metricbeat topic
include::monitoring-mb.asciidoc[tag=define-cluster-uuid]

[float]
[[configure-elastic-agent-logstash-dashboards]]

== Install and configure {agent}


[discrete]
[[add-logstash-integration]]
=== Step 1: Add the {agent} Logstash integration

{agent} is a single, unified way to add monitoring for logs, metrics, and other
types of data to a host. Each agent has a single policy you can update to add
integrations for new data sources, security protections, and more.

In this step, add the {ls} integration to monitor host logs and metrics.

. Go to the {kib} home page and click **Add integrations**.
+
--
[role="screenshot"]
image::images/kibana-home.png[{kib} home page]
--

. In the query bar, search for **Logstash** and select the integration to see more
details about it.

. Click **Add Logstash**.

. Configure the integration name and optionally add a description.

[[configure-logstash-logs]]
==== Configure {agent} to collect {ls} Logs

Make sure that **Logs** is turned on.

--
[role="screenshot"]
image::images/integration-configure-logs.png[{kib} home page]
--

. Make sure you configure all required settings:
* Under **Logs**, modify the log paths
to match your {ls} environment.

[[configure-logstash-dashboards]]
==== Configure {agent} with Logstash Dashboards

Make sure that **Metrics (Technical Preview)** is turned on, and **Metrics (Stack Monitoring)** is turned off.


--
[role="screenshot"]
image::images/integration-configure-tech-preview.png[Configuration page for adding log paths to the {agent} System integration]
--


* Under **Metrics (Technical Preview)**, make sure the Logstash URL setting
points to your {ls} instance URLs. By default, the integration collects {ls}
monitoring metrics from `https://localhost:9600`. If that host and port number are not
correct, update the `Logstash URL` setting. If you configured {ls} to use encrypted
communications and/or a username and password, you must access it via HTTPS, and expand the **Advanced Settings** options, and fill in with the appropriate values for your {ls} instance.


. Click **Save and continue**. This step takes a minute or two to complete. When
it's done, you'll have an agent policy that contains a system integration policy
for the configuration you just specified.
+
--
[role="screenshot"]
image::images/integration-configure-save.png[Configuration page for adding the {agent} System integration]
--

. In the popup, click **Add {agent} to your hosts** to open the **Add agent**
flyout.
+
TIP: If you accidentally close the popup, go to **{fleet} > Agents**, then click
**Add agent** to access the flyout.



[discrete]
[[add-agent-to-fleet]]

=== Step 2: Install and run an {agent} on your machine
// Include section about installing agents
include::monitoring-install.asciidoc[tag=install-agent-widget]

==== Step 3: Confirm Enrollment
// Include section about confirming enrollment
include::monitoring-confirm.asciidoc[tag=confirm-enrollment-widget]

[discrete]
[[view-data-dashboard]]
=== Step 4: Monitor Logstash logs and metrics (Logstash Metrics Dashboard Technical Preview)


. From the list of assets, open the **[Metrics Logstash] Logstash overview** dashboard to view overall performance. Then follow the navigation panel to further drill down into Logstash performance.
+
[role="screenshot"]
image::images/integration-dashboard-overview.png[The Logstash Overview dashboard in {kib} with various metrics from your monitored Logstash]

You can hover over any visualization to adjust its settings, or click the
**Edit** button to make changes to the dashboard. To learn more, refer to
{kibana-ref}/dashboard.html[Dashboard and visualizations].
79 changes: 5 additions & 74 deletions docs/static/monitoring/monitoring-ea.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -89,89 +89,20 @@ communications, you must access it via HTTPS. For example, use a `hosts` setting
like
`https://localhost:9600`.

[[configure-logstash-dashboards]]
==== Configure {agent} with Logstash Dashboards (Technical Preview)

Make sure that **Metrics (Technical Preview)** is turned on, and **Metrics (Stack Monitoring)** is turned off.


--
[role="screenshot"]
image::images/integration-configure-tech-preview.png[Configuration page for adding log paths to the {agent} System integration]
--


* Under **Metrics (Technical Preview)**, make sure the Logstash URL setting
points to your {ls} instance URLs. By default, the integration collects {ls}
monitoring metrics from `https://localhost:9600`. If that host and port number are not
correct, update the `Logstash URL` setting. If you configured {ls} to use encrypted
communications and/or a username and password, you must access it via HTTPS, and expand the **Advanced Settings** options, and fill in with the appropriate values for your {ls} instance.


. Click **Save and continue**. This step takes a minute or two to complete. When
it's done, you'll have an agent policy that contains a system integration policy
for the configuration you just specified.
+
--
[role="screenshot"]
image::images/integration-configure-save.png[Configuration page for adding the {agent} System integration]
--

. In the popup, click **Add {agent} to your hosts** to open the **Add agent**
flyout.
+
TIP: If you accidentally close the popup, go to **{fleet} > Agents**, then click
**Add agent** to access the flyout.


[discrete]
[[add-agent-to-fleet]]
=== Step 2: Install and run an {agent} on your machine

The **Add agent** flyout has two options: **Enroll in {fleet}** and
**Run standalone**. The default is to enroll the agents in {fleet}, as this
reduces the amount of work on the person managing the hosts by providing
a centralized management tool in {kib}.

include::{tab-widget-dir}/install-agent-widget.asciidoc[]

// Include section about installing agents
include::monitoring-install.asciidoc[tag=install-agent-widget]

==== Step 3: Confirm Enrollment

* Once enrollment is confirmed and incoming data is received, click **View assets** to access dashboards related to the Logstash integration.
* For traditional Stack Monitoring UI, the dashboards marked **[Logs Logstash]** are used to visualize the logs
produced by your Logstash Instances, with those marked **[Metrics Logstash]** for the technical preview metrics
dashboards.
** Note that these will only be populated with data if the **Metrics (Technical Preview)** checkbox was
selected.

--
[role="screenshot"]
image::images/integration-assets-dashboards.png[Integration assets]
--

A number of dashboards are included to view Logstash as a whole, and dashboards that allow you to drill-down
into how Logstash is performing on a node, pipeline and plugin basis.
// Include section about confirming enrollment
include::monitoring-confirm.asciidoc[tag=confirm-enrollment-widget]


[discrete]
[[view-data-stack-monitoring]]
=== Step 4: Monitor Logstash logs and metrics (Stack Monitoring)

{kibana-ref}/monitoring-data.html[View the monitoring data in {kib}], and navigate to the <<logstash-monitoring-ui, {monitoring} UI>>

[discrete]
[[view-data-dashboard]]
=== Step 4: Monitor Logstash logs and metrics (Logstash Metrics Dashboard Technical Preview)




. From the list of assets, open the **[Metrics Logstash] Logstash overview** dashboard to view overall performance. Then follow the navigation panel to further drill down into Logstash performance.
+
[role="screenshot"]
image::images/integration-dashboard-overview.png[The Logstash Overview dashboard in {kib} with various metrics from your monitored Logstash]

You can hover over any visualization to adjust its settings, or click the
**Edit** button to make changes to the dashboard. To learn more, refer to
{kibana-ref}/dashboard.html[Dashboard and visualizations].
{kibana-ref}/monitoring-data.html[View the monitoring data in {kib}], and navigate to the <<logstash-monitoring-ui, {monitoring} UI>>
11 changes: 11 additions & 0 deletions docs/static/monitoring/monitoring-install.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
[float]
[[install-agent]]
//tag::install-agent-widget[]

The **Add agent** flyout has two options: **Enroll in {fleet}** and
**Run standalone**. The default is to enroll the agents in {fleet}, as this
reduces the amount of work on the person managing the hosts by providing
a centralized management tool in {kib}.

include::{tab-widget-dir}/install-agent-widget.asciidoc[]
//end::install-agent-widget[]
21 changes: 18 additions & 3 deletions docs/static/monitoring/monitoring-overview.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,31 @@ For an introduction to monitoring your Elastic stack, see

[float]
[[configuring-logstash-xpack]]
=== Configuring monitoring for {ls}
=== Configuring monitoring for {ls} (Beta)

Make sure that monitoring is enabled on your {es} cluster. Then configure *one* of
these methods to collect {ls} metrics:

* <<monitoring-with-elastic-agent-dashboards, {agent} collection>>. {agent} collects
monitoring data from your {ls} instance and sends it directly to your monitoring
cluster, and shows the data in Logstash Dashboards.
The benefit of {agent} collection is that the monitoring agent remains
active even if the {ls} instance does not, you can manage all your
monitoring agents from a central location in {fleet}.
Plus, you can monitor an expanded set of metrics, includng


[float]
[[configuring-logstash-xpack-legacy]]
=== Configuring monitoring for {ls} (legacy)
* <<monitoring-with-elastic-agent, {agent} collection>>. {agent} collects
monitoring data from your {ls} instance and sends it directly to your monitoring
cluster. The benefit of {agent} collection is that the monitoring agent remains
active even if the {ls} instance does not. Plus you can mange all your
cluster, and shows the data in Logstash Dashboards.
The benefit of {agent} collection is that the monitoring agent remains
active even if the {ls} instance does not, you can manage all your
monitoring agents from a central location in {fleet}.
Plus, you can monitor an expanded set of metrics, includng


* <<monitoring-with-metricbeat, {metricbeat} collection>>. Metricbeat collects
monitoring data from your {ls} instance and sends it directly to your monitoring
Expand Down
Loading

0 comments on commit b8bbd03

Please sign in to comment.