Add test results visualization #360

mw-kapilg · 2024-09-18T08:53:51Z

This feature creates an in-platform view for MATLAB test results (whenever a default test runner or pre-made TestTask is used) if the user has a MATLAB Test license.

Key highlights:

Each row of the table represents the test file.
All tests for a single file are shown under that file.
The files with same name and different parent directories are shown separately and you can hover over the file name to see its path.
The table in table format allows easy management of test cases for a test file
You see the logs in-place for a failed/incomplete. If there are more than failures within a test case, say multiple verification failures, each verification failure will have its own dropdown and log box.
Each of the passed, failed, etc. buttons act as filters to respective type of tests
The "Open in MATLAB Online" button helps to redirect users to MATLAB Online. If their Jenkins project is connected to a git repository, they are prompted to open the in MATLAB Online for easy debugging.
The help button redirects users to our plugin's configuration guide.

Here's a few snapshots:

mw-hrastega · 2024-10-23T15:20:47Z

src/main/resources/com/mathworks/ci/TestResultsViewAction/index.jelly

+                                Test Name
+                            </th>
+                            <th class="pane-header" style="width: auto; min-width: 10em; text-align: right;">
+                                Duration (seconds)


(s) instead of (seconds)?

Is it better to include + and - signs or drop-down arrows instead?

Updated (seconds) to (s)

Is it better to include + and - signs or drop-down arrows instead?

I didn't get this, could you explain?

Nothing major. Currently, it seems that you use the + and - buttons to expand and collapse sections in the report. I was wondering whether a drop-down arrow could do the same. In the test browser, we resort to the arrow for expansion: https://www.mathworks.com/help/matlab/matlab_prog/run-tests-using-test-browser.html#:~:text=You%20can%20also%20add%20tests%20to%20the%20test%20browser%20by%20clicking%20the%20drop%2Ddown%20arrow

That's true. For reference I had a look at the Jenkins JUnit and TAP plugins which were using + and - symbols to expand and collapse. I don't mind the chevrons either, do you feel using them for dropdowns would be more helpful for the user?

src/main/resources/com/mathworks/ci/TestResultsViewAction/index.jelly

pom.xml

src/main/java/com/mathworks/ci/TestCase.java

src/main/java/com/mathworks/ci/TestDiagnostics.java

src/main/java/com/mathworks/ci/TestFile.java

src/main/java/com/mathworks/ci/TestDiagnostics.java

src/main/java/com/mathworks/ci/TestCase.java

src/main/java/com/mathworks/ci/TestFile.java

src/main/java/com/mathworks/ci/TestResultsViewAction.java

src/main/java/com/mathworks/ci/utilities/MatlabCommandRunner.java

src/test/java/integ/com/mathworks/ci/TestResultsViewActionTest.java

src/main/java/com/mathworks/ci/MatlabTestCase.java

src/main/java/com/mathworks/ci/TestResultsViewAction.java

src/main/resources/+ciplugins/+jenkins/TestResultsViewPlugin.m

src/main/resources/com/mathworks/ci/TestResultsViewAction/index.jelly

src/test/java/integ/com/mathworks/ci/TestResultsViewActionTest.java

src/main/resources/+ciplugins/+jenkins/TestResultsViewPlugin.m

src/main/java/com/mathworks/ci/MatlabTestCase.java

src/main/java/com/mathworks/ci/MatlabTestFile.java

src/main/java/com/mathworks/ci/TestResultsViewAction.java

src/main/java/com/mathworks/ci/MatlabTestFile.java

davidbuzinski

Looking good! I like the changes to MatlabTestCase and MatlabTestFile. A few more things in the MATLAB code for the TestRunnerPlugin then it should be ready to go.

src/main/resources/+ciplugins/+jenkins/TestResultsViewPlugin.m

sameagen-MW

Just some minor comments from me! The only blocker I see is what David mentioned with the cleanup code for the plugin.

src/main/java/com/mathworks/ci/TestResultsViewAction.java

sameagen-MW · 2025-02-03T15:29:21Z

src/main/resources/+ciplugins/+jenkins/TestResultsViewPlugin.m

+    methods (Access=protected)
+        function reportFinalizedSuite(plugin, pluginData)
+            % Checkout MATLAB Test license
+            license('checkout', 'matlab_test');


I'm still slightly worried about this. I think ideally for me we'd do the license check and checkout at the same time, just to avoid any situations where MATLAB Test was available initially at the time of the first check but not at the time of reporting the finalized suite. Probably not a big deal though.

I think it should be fine. Do you have a case in mind where the situation you're saying could happen?

mw-kapilg added 30 commits August 1, 2024 10:57

commit to draft test procedure

b7b737a

update finally block

a65e93f

fix merge conflicts

49e7bee

Update actions

a7bd44b

Change build colors

eb83cb6

fix merge conflicts

dc059ae

update colour scheme

9d79737

add tests

12ebdc3

remove extra comments

f917c1f

try fix for ci test failure

284d2df

try fix for ci test failure

0f4b801

add debug statements

fe6fab8

debug failure

cfb1795

debug

2071d66

update input stream

e50a7c7

update paths

620bceb

update test workspace

1c068e4

more debugging

7725c3a

fix testcase workspace

16f0988

final fix (hopefully)

213cfad

revert filepath

46666c9

remove absolute path

cb2f6e8

update workspace path

fb7793b

add logger

678a12d

update test

7b746fd

update test data

2d81d41

Specify java version in yml

06b2d12

update slave label

c15e013

add null checks

40a7af4

add null check

3517803

mw-hrastega reviewed Oct 23, 2024

View reviewed changes

src/main/resources/com/mathworks/ci/TestResultsViewAction/index.jelly Outdated Show resolved Hide resolved

mw-hrastega reviewed Oct 23, 2024

View reviewed changes

src/main/resources/com/mathworks/ci/TestResultsViewAction/index.jelly Outdated Show resolved Hide resolved

nbhoski reviewed Oct 24, 2024

View reviewed changes

sameagen-MW reviewed Oct 25, 2024

View reviewed changes

Update as per review comments

34c1a5b

mw-kapilg requested review from nbhoski, sameagen-MW and mw-hrastega November 5, 2024 10:36

Base automatically changed from dev_main to 2.15.1-SNAPSHOT-Qualification November 14, 2024 14:10

davidbuzinski requested review from davidbuzinski and removed request for nbhoski December 12, 2024 15:10

resolve conflicts

2a7c908

mw-kapilg changed the base branch from 2.15.1-SNAPSHOT-Qualification to dev_main January 16, 2025 04:36

resolve conflicts

49cbe24

sameagen-MW reviewed Jan 23, 2025

View reviewed changes

davidbuzinski reviewed Jan 23, 2025

View reviewed changes

mw-kapilg added 2 commits January 29, 2025 00:30

update as per review comments

254f0e8

add default case

caa10cb

mw-kapilg requested review from sameagen-MW and davidbuzinski January 28, 2025 19:27

davidbuzinski reviewed Jan 28, 2025

View reviewed changes

src/main/java/com/mathworks/ci/MatlabTestFile.java Outdated Show resolved Hide resolved

mw-kapilg added 2 commits January 29, 2025 10:20

update method accesses and fix decimal addition issue

33ea7c9

remove commented code

3a1bc14

mw-kapilg requested a review from davidbuzinski January 29, 2025 17:52

davidbuzinski reviewed Jan 29, 2025

View reviewed changes

src/main/resources/+ciplugins/+jenkins/TestResultsViewPlugin.m Outdated Show resolved Hide resolved

src/main/resources/+ciplugins/+jenkins/TestResultsViewPlugin.m Outdated Show resolved Hide resolved

sameagen-MW approved these changes Feb 3, 2025

View reviewed changes

address feedback, fix ui elements

6ccc534

mw-kapilg requested a review from davidbuzinski February 5, 2025 07:03

davidbuzinski approved these changes Feb 6, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add test results visualization #360

Add test results visualization #360

mw-kapilg commented Sep 18, 2024

mw-hrastega Oct 23, 2024

mw-hrastega Oct 23, 2024

mw-kapilg Nov 5, 2024

mw-hrastega Nov 5, 2024

mw-kapilg Jan 29, 2025

davidbuzinski left a comment

sameagen-MW left a comment

sameagen-MW Feb 3, 2025

mw-kapilg Feb 5, 2025

Add test results visualization #360

Are you sure you want to change the base?

Add test results visualization #360

Conversation

mw-kapilg commented Sep 18, 2024

mw-hrastega Oct 23, 2024

Choose a reason for hiding this comment

mw-hrastega Oct 23, 2024

Choose a reason for hiding this comment

mw-kapilg Nov 5, 2024

Choose a reason for hiding this comment

mw-hrastega Nov 5, 2024

Choose a reason for hiding this comment

mw-kapilg Jan 29, 2025

Choose a reason for hiding this comment

davidbuzinski left a comment

Choose a reason for hiding this comment

sameagen-MW left a comment

Choose a reason for hiding this comment

sameagen-MW Feb 3, 2025

Choose a reason for hiding this comment

mw-kapilg Feb 5, 2025

Choose a reason for hiding this comment