Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add interactive plots with Plotly #163

Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Added

- Improve security on multi-user systems. Dashboard now generates a login token when started. Users
must login with the token to view project and job data in the dashboard.
- PlotViewer module for showing interactive plots in the dashboard (#162, #163).

Changed
+++++++
Expand Down
1 change: 1 addition & 0 deletions doc/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ Dashboard Modules
modules.FlowStatus
modules.ImageViewer
modules.Notes
modules.PlotViewer
modules.Schema
modules.StatepointList
modules.TextDisplay
Expand Down
File renamed without changes.
3 changes: 3 additions & 0 deletions examples/plots-matplotlib/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Matplotlib Plots Example

To run this example, call `python init.py` and then `python dashboard.py run`.
File renamed without changes.
File renamed without changes.
3 changes: 3 additions & 0 deletions examples/plots-plotly/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
signac.rc
signac_project_document.json
workspace/
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Plots Example
# Plotly Plots Example

To run this example, call `python init.py` and then `python dashboard.py run`.
78 changes: 78 additions & 0 deletions examples/plots-plotly/dashboard.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
#!/usr/bin/env python3
# Copyright (c) 2019 The Regents of the University of Michigan
# All rights reserved.
# This software is licensed under the BSD 3-Clause License.
from scipy.signal import coherence

from signac_dashboard import Dashboard
from signac_dashboard.modules import PlotViewer, StatepointList, TextDisplay


class PlotDashboard(Dashboard):
def job_sorter(self, job):
return job.sp.get("coherence_time", -1)

def job_title(self, job):
return f"Coherence time: {job.sp.coherence_time}"


def correlation_text(job):
return "Correlation coefficient: {:.5f}".format(job.doc["correlation"])


def plotly_args(job):
# Visualization adapted from:
# https://matplotlib.org/gallery/lines_bars_and_markers/cohere.html

# It's necessary to cast to list because the list elements of the job
# document are BufferedJSONAttrList, which is not serializable
signals_traces = [
{
"x": list(job.doc["t"]),
"y": list(job.doc["s1"]),
"name": "s1",
},
{
"x": list(job.doc["t"]),
"y": list(job.doc["s2"]),
"name": "s2",
},
]
signals_layout = {
"xaxis": {
"title": "time",
"range": [0, 2],
},
"height": 200,
"margin": dict(t=30, b=40, l=40, r=0),
}

dt = job.doc["t"][1] - job.doc["t"][0]
coherence_x, coherence_y = coherence(
job.doc["s1"], job.doc["s2"], nfft=256, fs=1.0 / dt
)
coherence_traces = [
{
"x": coherence_x.tolist(),
"y": coherence_y.tolist(),
}
]
coherence_layout = {
"title": f"Coherence time = {job.sp.coherence_time}",
"xaxis": {"title": "frequency"},
"yaxis": {"title": "coherence", "range": [0, 1]},
"height": 200,
"margin": dict(t=30, b=40, l=40, r=0),
}
return [
("Signals", signals_traces, signals_layout),
("Coherence", coherence_traces, coherence_layout),
]


if __name__ == "__main__":
modules = []
modules.append(StatepointList())
modules.append(PlotViewer(plotly_args=plotly_args))
modules.append(TextDisplay(name="Correlation", message=correlation_text))
PlotDashboard(modules=modules).main()
40 changes: 40 additions & 0 deletions examples/plots-plotly/init.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
#!/usr/bin/env python3
# Copyright (c) 2019 The Regents of the University of Michigan
# All rights reserved.
# This software is licensed under the BSD 3-Clause License.
import numpy as np
import signac

project = signac.init_project("plots")


def plot_coherence(job):
# Data generation adapted from:
# https://matplotlib.org/gallery/lines_bars_and_markers/cohere.html

print(f"Generating signals for coherence time {job.sp.coherence_time}, job {job}")
# Fixing random state for reproducibility
np.random.seed(job.sp.seed)

dt = 0.01
t = np.arange(0, 30, dt)
nse1 = np.random.randn(len(t)) # white noise 1
nse2 = np.random.randn(len(t)) # white noise 2

# Two signals with a coherent part and a random part
s1 = np.sin(2 * np.pi * job.sp.coherence_time * t) + nse1
s2 = np.sin(2 * np.pi * job.sp.coherence_time * t) + nse2

# Save the signal data
job.doc["t"] = t.tolist()
job.doc["s1"] = s1.tolist()
job.doc["s2"] = s2.tolist()

# Save correlation coefficient
job.doc["correlation"] = np.corrcoef(s1, s2)[0, 1]


for i in range(30):
job = project.open_job({"coherence_time": i, "seed": 42})
job.init()
plot_coherence(job)
2 changes: 2 additions & 0 deletions signac_dashboard/modules/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
from .flow_status import FlowStatus
from .image_viewer import ImageViewer
from .notes import Notes
from .plot_viewer import PlotViewer
from .schema import Schema
from .statepoint_list import StatepointList
from .text_display import TextDisplay
Expand All @@ -16,6 +17,7 @@
"FlowStatus",
"ImageViewer",
"Notes",
"PlotViewer",
"Schema",
"StatepointList",
"TextDisplay",
Expand Down
112 changes: 112 additions & 0 deletions signac_dashboard/modules/plot_viewer.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# Copyright (c) 2022 The Regents of the University of Michigan
# All rights reserved.
# This software is licensed under the BSD 3-Clause License.
from typing import Callable, Dict, Iterable, List, Tuple, Union

import flask_login
from flask import abort, render_template
from jinja2.exceptions import TemplateNotFound
from signac import Project
from signac.contrib.job import Job

from signac_dashboard.dashboard import Dashboard
from signac_dashboard.module import Module


class PlotViewer(Module):
"""Displays a plot associated with the job.

The PlotViewer module can display an interactive plot by using the
Plotly JavaScript library. For information on the different accepted
parameters for the data and layout, refer to the `Plotly JS documentation
<https://plotly.com/javascript/>`_.

Example:

.. code-block:: python

from signac_dashboard.modules import PlotViewer

def plotly_args_function(project):
return [
("Card title", # if empty, the "name" parameter will be used
# each element on the data list is a different trace
[{
"x": [1, 2, 3, 4, 5], # x coordinates of the trace
"y": [1, 2, 4, 8, 16] # y coordinates of the trace
}],
{"margin": {"t": 0}} # layout specification for the whole plot
)
]

plot_module = PlotViewer(plotly_args=plotly_args_function, context="ProjectContext")

:param name: Default name for the card. Ignored if the :code:`plotly_args`
callable provides one for each card.
:type name: str
:param plotly_args: A callable that accepts a job (in the :code:`'JobContext'`)
or a project (in the :code:`'ProjectContext'`) and returns an iterable. Each
element will constitute a new card and will be composed of a tuple of three
elements: the card title, the plotly data and the plotly layout specification.
:type plotly_args: callable
:param context: Supports :code:`'JobContext'` and :code:`'ProjectContext'`.
:type context: str
"""

_supported_contexts = {"JobContext", "ProjectContext"}

def __init__(
self,
name="Plot Viewer",
plotly_args: Callable[
[Union[Job, Project]], Iterable[Tuple[str, List[Dict], Dict]]
] = lambda _: [],
context="JobContext",
template="cards/plot_viewer.html",
**kwargs,
):

super().__init__(
name=name,
context=context,
template=template,
**kwargs,
)
self.plotly_args = plotly_args

def get_cards(self, job_or_project):
return [
{
"name": title if title else self.name,
"content": render_template(
self.template,
jobid=job_or_project.id,
plotly_data=data,
plotly_layout=layout,
),
}
for title, data, layout in self.plotly_args(job_or_project)
]

def register(self, dashboard: Dashboard):
# Register routes
@dashboard.app.route("/module/plot_viewer/<path:filename>")
@flask_login.login_required
def plot_viewer_asset(filename):
try:
return render_template(f"plot_viewer/{filename}")
except TemplateNotFound:
abort(404, "The file requested does not exist.")

# Register assets
assets = ["js/plot_viewer.js"]
for assetfile in assets:
dashboard.register_module_asset(
{
"url": f"/module/plot_viewer/{assetfile}",
}
)

cdn_assets = ["https://cdn.plot.ly/plotly-2.16.1.min.js"]
for asseturl in cdn_assets:
dashboard.register_module_asset({"url": asseturl})
4 changes: 4 additions & 0 deletions signac_dashboard/templates/cards/plot_viewer.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
<div class="plot_viewer"
data-plotly-data='{{ plotly_data | tojson }}'
data-plotly-layout='{{ plotly_layout | tojson }}'>
</div>
12 changes: 12 additions & 0 deletions signac_dashboard/templates/plot_viewer/js/plot_viewer.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
function draw_plot(element) {
data = JSON.parse(element.getAttribute("data-plotly-data"));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe the card HTML can contain a REST endpoint, and fetch the data with a second call? That way we don't have to generate and include all the raw data in the HTML page. This should help a lot with page responsiveness. You can see examples of REST calls in other modules:

If you need help with this, let me know.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have implemented this in 226111e. The page loading is way more responsive now.

Still, I'm a total newbie with Flask, so a thorough review is needed.

My main concerns are:

  • I have had to create a module identifier so that a different endpoint is created for each instance of the PlotlyView module. I decided to hash the id(self) which should work, but has the disadvantage that each time the dashboard is launched the URLs change. I don't think this is a huge problem, but I feel like there is a better way to do this.
  • For each added instance of the module, the register method is called. However, the assets URLs should only be added once. If you try to do it multiple times, the second time around the module is not registered. I have solved this with a class attribute PlotlyViewer._assets_url_registered that avoids this duplication. Still, I feel like this solution is somewhat hacky.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@javierbg
Flask Blueprints are the way to go. I started experimenting with them in #182 to allow multiple Notes modules to be added.

layout = JSON.parse(element.getAttribute("data-plotly-layout"));

Plotly.newPlot(element, data, layout);
}

$(document).on('turbolinks:load', function() {
$('.plot_viewer').each((index, element) => {
draw_plot(element);
});
})