Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

clean(GH-881): Add Schema Diagram Generation to Github Pages #885

Merged
merged 45 commits into from
Feb 16, 2025

Conversation

ThomasAFink
Copy link
Member

@ThomasAFink ThomasAFink commented Feb 16, 2025

Pull Request Checklist (Feature Branch to next):

  • Ich habe die neuesten Änderungen aus dem next Branch in meinen Feature-Branch gemergt.
  • Das Code-Review wurde abgeschlossen.
  • Fachliche Tests wurden durchgeführt und sind abgeschlossen.

Summary by CodeRabbit

  • New Features

    • Integrated automated generation of schema diagrams into the deployment workflow, ensuring all assets are ready before publishing.
  • Documentation

    • Updated public documentation with new links offering direct access to API references and schema diagrams for enhanced clarity.

Thomas Fink added 30 commits February 16, 2025 12:38
Copy link
Contributor

coderabbitai bot commented Feb 16, 2025

Walkthrough

The pull request integrates schema diagram generation into the CI/CD process. A new job, generate-schema-diagrams, is added to the combined workflow and set as a prerequisite for the deployment job. The deploy workflow now includes steps for downloading the generated schema diagram artifact and updates documentation links. Additionally, a new workflow file automates the generation of schema diagrams using Python, uploading the results as an artifact for further use in deployment.

Changes

File Change Summary
.github/workflows/combined-workflow-with-docs.yaml Added new job generate-schema-diagrams (invokes external workflow) and updated the deploy-to-pages job to require its successful completion.
.github/workflows/deploy-pages.yaml Added contents: read permission; inserted steps to download the schema diagrams artifact and list directory structure; updated index page links for documentation.
.github/workflows/generate-schema-diagrams.yaml New workflow file that defines the generate job to convert JSON Schemas into Mermaid diagrams using a Python script, then uploads the diagrams as an artifact.

Sequence Diagram(s)

sequenceDiagram
  participant Trigger as "Push (next branch)"
  participant Combined as "Combined Workflow"
  participant SchemaGen as "Generate Diagram Job"
  participant Deploy as "Deploy to Pages Job"
  
  Trigger->>Combined: Initiates workflow
  Combined->>SchemaGen: Run generate-schema-diagrams job
  SchemaGen->>SchemaGen: Process JSON schemas & generate diagrams
  SchemaGen->>Deploy: Upload schema diagrams artifact
  Deploy->>Deploy: Download artifact & update docs
  Deploy->>Pages: Deploy updated pages
Loading
sequenceDiagram
  participant Runner as "GitHub Runner"
  participant Checkout as "Checkout Repo"
  participant Setup as "Setup Python 3.12"
  participant Install as "Install Dependencies"
  participant Script as "Run convert_schema.py"
  participant Upload as "Upload Artifact"
  
  Runner->>Checkout: Checkout repository
  Runner->>Setup: Configure Python environment
  Runner->>Install: Install jsonschema & jinja2
  Runner->>Script: Execute schema conversion script
  Script->>Upload: Save diagrams & upload as artifact
Loading

Poem

I'm a hopping rabbit in YAML delight,
With workflows growing under the moonlight.
Schema diagrams sprout from JSON seeds,
Guiding deployments, fulfilling all needs.
I jump with joy at each new code beat,
Happy in pipelines where success and art meet!
🐇✨


🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (5)
.github/workflows/combined-workflow-with-docs.yaml (1)

58-410: Trailing Spaces Cleanup
YAMLlint has flagged several instances of trailing spaces (e.g., lines 58, 65, 72, 80, etc.). Please remove these extraneous spaces to ensure the file meets linting standards.

🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 104-104: no new line character at the end of file

(new-line-at-end-of-file)

.github/workflows/deploy-pages.yaml (1)

176-193: Redundant Artifact Download Steps
After creating the index page, the workflow re-downloads coverage reports, API docs, and security reports. Please confirm whether this repetition is intentional or if the earlier downloads can be reused to reduce redundancy.

.github/workflows/generate-schema-diagrams.yaml (3)

13-31: Job Setup and Dependencies Installation
The job correctly checks out the repository, creates the output directory, sets up Python 3.12, and installs the required dependencies (jsonschema and jinja2). For reproducibility, consider pinning the dependency versions.


32-401: Schema Converter Script Review
The inline Python script is comprehensive and well organized:
• It defines helper functions (e.g., sanitize_name, get_type_name) and encapsulates schema loading and conversion logic.
• The conversion process covers property types, array handling, and nested objects.
• HTML generation via a Jinja2 template is clearly implemented.

Suggestions:
– Consider adding error handling when loading and parsing JSON schema files to better manage malformed inputs.
– Pin dependency versions in the installation step for added reproducibility.

Overall, the script adheres to good practices and effectively generates the desired diagrams.

🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 58-58: trailing spaces

(trailing-spaces)


[error] 65-65: trailing spaces

(trailing-spaces)


[error] 72-72: trailing spaces

(trailing-spaces)


[error] 80-80: trailing spaces

(trailing-spaces)


[error] 90-90: trailing spaces

(trailing-spaces)


[error] 94-94: trailing spaces

(trailing-spaces)


[error] 97-97: trailing spaces

(trailing-spaces)


[error] 101-101: trailing spaces

(trailing-spaces)


[error] 110-110: trailing spaces

(trailing-spaces)


[error] 120-120: trailing spaces

(trailing-spaces)


[error] 136-136: trailing spaces

(trailing-spaces)


[error] 154-154: trailing spaces

(trailing-spaces)


[error] 283-283: trailing spaces

(trailing-spaces)


[error] 290-290: trailing spaces

(trailing-spaces)


[error] 304-304: trailing spaces

(trailing-spaces)


[error] 308-308: trailing spaces

(trailing-spaces)


[error] 313-313: trailing spaces

(trailing-spaces)


[error] 318-318: trailing spaces

(trailing-spaces)


[error] 323-323: trailing spaces

(trailing-spaces)


[error] 330-330: trailing spaces

(trailing-spaces)


[error] 337-337: trailing spaces

(trailing-spaces)


[error] 344-344: trailing spaces

(trailing-spaces)


[error] 349-349: trailing spaces

(trailing-spaces)


[error] 355-355: trailing spaces

(trailing-spaces)


[error] 358-358: trailing spaces

(trailing-spaces)


[error] 364-364: trailing spaces

(trailing-spaces)


[error] 367-367: trailing spaces

(trailing-spaces)


[error] 379-379: trailing spaces

(trailing-spaces)


[error] 382-382: trailing spaces

(trailing-spaces)


[error] 387-387: trailing spaces

(trailing-spaces)


[error] 398-398: trailing spaces

(trailing-spaces)


58-410: Trailing Spaces Cleanup
YAMLlint has identified multiple trailing space issues (e.g., lines 58, 65, 72, etc.) as well as a missing newline at the end of the file. Please remove these formatting issues to keep the file clean and compliant with linting guidelines.

🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 58-58: trailing spaces

(trailing-spaces)


[error] 65-65: trailing spaces

(trailing-spaces)


[error] 72-72: trailing spaces

(trailing-spaces)


[error] 80-80: trailing spaces

(trailing-spaces)


[error] 90-90: trailing spaces

(trailing-spaces)


[error] 94-94: trailing spaces

(trailing-spaces)


[error] 97-97: trailing spaces

(trailing-spaces)


[error] 101-101: trailing spaces

(trailing-spaces)


[error] 110-110: trailing spaces

(trailing-spaces)


[error] 120-120: trailing spaces

(trailing-spaces)


[error] 136-136: trailing spaces

(trailing-spaces)


[error] 154-154: trailing spaces

(trailing-spaces)


[error] 283-283: trailing spaces

(trailing-spaces)


[error] 290-290: trailing spaces

(trailing-spaces)


[error] 304-304: trailing spaces

(trailing-spaces)


[error] 308-308: trailing spaces

(trailing-spaces)


[error] 313-313: trailing spaces

(trailing-spaces)


[error] 318-318: trailing spaces

(trailing-spaces)


[error] 323-323: trailing spaces

(trailing-spaces)


[error] 330-330: trailing spaces

(trailing-spaces)


[error] 337-337: trailing spaces

(trailing-spaces)


[error] 344-344: trailing spaces

(trailing-spaces)


[error] 349-349: trailing spaces

(trailing-spaces)


[error] 355-355: trailing spaces

(trailing-spaces)


[error] 358-358: trailing spaces

(trailing-spaces)


[error] 364-364: trailing spaces

(trailing-spaces)


[error] 367-367: trailing spaces

(trailing-spaces)


[error] 379-379: trailing spaces

(trailing-spaces)


[error] 382-382: trailing spaces

(trailing-spaces)


[error] 387-387: trailing spaces

(trailing-spaces)


[error] 398-398: trailing spaces

(trailing-spaces)


[error] 410-410: no new line character at the end of file

(new-line-at-end-of-file)

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3fb8849 and 07a03c8.

📒 Files selected for processing (3)
  • .github/workflows/combined-workflow-with-docs.yaml (1 hunks)
  • .github/workflows/deploy-pages.yaml (4 hunks)
  • .github/workflows/generate-schema-diagrams.yaml (1 hunks)
🧰 Additional context used
🪛 YAMLlint (1.35.1)
.github/workflows/generate-schema-diagrams.yaml

[error] 58-58: trailing spaces

(trailing-spaces)


[error] 65-65: trailing spaces

(trailing-spaces)


[error] 72-72: trailing spaces

(trailing-spaces)


[error] 80-80: trailing spaces

(trailing-spaces)


[error] 90-90: trailing spaces

(trailing-spaces)


[error] 94-94: trailing spaces

(trailing-spaces)


[error] 97-97: trailing spaces

(trailing-spaces)


[error] 101-101: trailing spaces

(trailing-spaces)


[error] 110-110: trailing spaces

(trailing-spaces)


[error] 120-120: trailing spaces

(trailing-spaces)


[error] 136-136: trailing spaces

(trailing-spaces)


[error] 154-154: trailing spaces

(trailing-spaces)


[error] 283-283: trailing spaces

(trailing-spaces)


[error] 290-290: trailing spaces

(trailing-spaces)


[error] 304-304: trailing spaces

(trailing-spaces)


[error] 308-308: trailing spaces

(trailing-spaces)


[error] 313-313: trailing spaces

(trailing-spaces)


[error] 318-318: trailing spaces

(trailing-spaces)


[error] 323-323: trailing spaces

(trailing-spaces)


[error] 330-330: trailing spaces

(trailing-spaces)


[error] 337-337: trailing spaces

(trailing-spaces)


[error] 344-344: trailing spaces

(trailing-spaces)


[error] 349-349: trailing spaces

(trailing-spaces)


[error] 355-355: trailing spaces

(trailing-spaces)


[error] 358-358: trailing spaces

(trailing-spaces)


[error] 364-364: trailing spaces

(trailing-spaces)


[error] 367-367: trailing spaces

(trailing-spaces)


[error] 379-379: trailing spaces

(trailing-spaces)


[error] 382-382: trailing spaces

(trailing-spaces)


[error] 387-387: trailing spaces

(trailing-spaces)


[error] 398-398: trailing spaces

(trailing-spaces)


[error] 410-410: no new line character at the end of file

(new-line-at-end-of-file)

⏰ Context from checks skipped due to timeout of 90000ms (10)
  • GitHub Check: call-owasp-security-checks / security-scan (zmsclient, 8.0)
  • GitHub Check: call-owasp-security-checks / security-scan (zmsdb, 8.0)
  • GitHub Check: call-php-unit-tests / zmsapi-test
  • GitHub Check: call-owasp-security-checks / security-scan (zmsdldb, 8.0)
  • GitHub Check: call-owasp-security-checks / security-scan (zmsmessaging, 8.0)
  • GitHub Check: call-owasp-security-checks / security-scan (zmscitizenapi, 8.0)
  • GitHub Check: call-owasp-security-checks / security-scan (zmscalldisplay, 8.0)
  • GitHub Check: call-owasp-security-checks / security-scan (zmsapi, 8.0)
  • GitHub Check: call-owasp-security-checks / security-scan (zmsadmin, 8.0)
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (7)
.github/workflows/combined-workflow-with-docs.yaml (2)

83-84: Schema Diagram Generation Job Added
The new generate-schema-diagrams job is correctly integrated by referencing its dedicated workflow file. This modular design improves maintainability.


87-91: Deploy-to-Pages Dependency Update
Adding generate-schema-diagrams into the needs array and checking its result in the if condition ensures that deployment occurs only after successful diagram generation. Please verify that the external workflow reliably reports its status.

.github/workflows/deploy-pages.yaml (2)

23-26: Updated Permissions
The addition of contents: read to the permissions block is appropriate for accessing repository contents during deployment.


41-46: Download Schema Diagrams Step
The newly added step to download the schema-diagrams artifact and place it in public/diagrams is well integrated. This ensures that the generated diagrams become part of the deployed assets.

.github/workflows/generate-schema-diagrams.yaml (3)

1-9: New Workflow for Schema Diagram Generation
This workflow neatly encapsulates the schema diagram generation logic. The use of workflow_call with an output (diagrams_artifact) enhances reusability and integrates well with downstream deployment jobs.


10-12: Permissions Setup
Granting contents: read permission is essential for accessing and processing the JSON schema files. The permission configuration is appropriately minimal and secure.


402-410: Artifact Upload and Workflow Finalization
Executing the schema conversion script and uploading the resulting diagrams artifact (with a retention of one day) is implemented correctly. Ensure that downstream workflows properly utilize this artifact.

🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 410-410: no new line character at the end of file

(new-line-at-end-of-file)

@ThomasAFink ThomasAFink merged commit bea2a98 into next Feb 16, 2025
49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant