Issue/webpack 5 upgrade #106

cayb0rg · 2023-05-25T21:58:01Z

MWDK Upgrade

Upgrades all of the packages in the Materia Widget Development Kit to their latest versions and removes deprecated packages.
Server is now created in express.js and uses webpack-dev-middleware to compile and view assets.
Adds a loading screen for installing widgets to local Materia
Adds a scorescreen page. Custom scorescreens are supported if the author uploads sample score data from Materia.

Requirements:

Node v16 or later (including v18)
Yarn

Local setup

### Materia The current Materia widget-dependencies-package does not support the MWDK. UPDATE: the MWD package has been updated. **These steps are no longer needed**. 1. Checkout this [react/widget-dependencies-package-part-three branch](https://github.com/cayb0rg/Materia/tree/react/widget-dependencies-package-part-three) of Materia. This is required to load the player and creator. 2. Install Materia if you have not. 3. Run ``yarn build`` inside Materia's root folder. 4. Run ``yarn link`` inside Materia/public/dist. 5. Start Materia (run ``docker-compose up`` in Materia/docker and ``yarn dev`` in Materia/src)

Widget

Each widget may need a few small changes to support Webpack v5. Here is an incomprehensive list of the possible changes:

New webpack.config.js entry rules:

Include only entries that have HTML files (except for markdown files), as these will be used by HtmlWebpackPlugin. Include the HTML file as the first element, then include any necessary JS and CSS in the array.
There should be no extension in the entry's name.
CSS and Markdown guides should not have their own entries

It should look similar to:

const entries = {
	'player': [
		path.join(srcPath, 'player.html'),
		path.join(srcPath, 'player.js'),
		path.join(srcPath, 'player.scss')
	],
	'creator': [
		path.join(srcPath, 'creator.html'),
		path.join(srcPath, 'creator.js'),
		path.join(srcPath, 'creator.scss'),
	],
	'scoreScreen': [
		path.join(srcPath, 'scoreScreen.html'),
		path.join(srcPath, 'scoreScreen.js'),
		path.join(srcPath, 'scoreScreen.scss'),
	]
}

Either include CSS inside your JS files for MiniCssExtractPlugin to recognize them, or include them as elements in the entry arrays (see above).
Use absolute urls for assets like loading fonts

Remove the following rules from moduleRules, if applicable: rules.loadAndCompileMarkdown andrules.loadAndPrefixCSS
Update the NPM scripts inside package.json to:

"start": "mwdk-start",
"build": "mwdk-build-prod",
"build-dev": "mwdk-build-dev"

mwdk-start: This starts the Express server and sets the webpack configuration to development mode.
mwdk-build-prod: This runs webpack in production mode. See the output in /build folder.
mwdk-build-dev: This runs webpack in development mode. See the output in /build folder.

Look at an example for the changes that need to be made (ignore the images).

AngularJS Widgets

AngularJS widgets may encounter an angular injection error. This usually only occurs with widgets that have directives, services, and controllers loaded in separate files. There are two ways to fix this. One way is to do what This or That does, and set up the directives, controllers, and services in a single file (for example, see creator.js. The other, more difficult way I've found is to manually bootstrap the angular modules.

Remove ng-app from body or HTML tags.
Move the main scripts such as player.js and creator.js to the end of the body tag
Inside the files declaring the angular modules, remove the variable declaration when initially declaring the module. For example, the matching widget would look like:
At the very end of the last file in the respective webpack entry, manually start the module with the command: angular.bootstrap(document, ['your-module-name-here']). The example above would look like:

Look at an example for the changes that need to be made.

Running this MWDK PR

Checkout this PR.
Run yarn install && yarn build && yarn linkinside the Materia Widget Development Kit. You may also use bun install.
Run rm -rf node_modules inside your widget.
Enter the git url for MWDK dependency: "materia-widget-development-kit": "https://github.com/cayb0rg/Materia-Widget-Dev-Kit.git#issue/webpack-5-upgrade"
Run yarn install or bun install
(Optional) Run yarn link materia-widget-development-kit. This will allow you to make changes to the MWDK without having to reinstall it in your widget.
Run yarn start or bun start

Known Bugs / To-Do

Updated Widgets (PRs made)

…dgets

clpetersonucf

Really great work here so far. With limited issues I was able to get one of your widget PRs running locally and installed to dev Materia.

Some high-level comments:

As commented, MWD has been updated to hopefully (?) include the most recent/missing dependencies. It's worth confirming 0.2.0 works out of the box, and replace the reference to it in the package.json when possible.
I love the little progress bar when installing to local Materia. That's a nice touch.
I haven't tested the score screen feature, but the instructions could be a little clearer as to what's required. If the score screen options aren't ready for prime-time, we could also potentially disable them until they have more time in the oven.
Previously the MWDK would allow you to save draft/publish a qset to test in the player. Right now it looks like the demo is the only qset that works with the player. Is that the case? Are there significant challenges to re-implementing cached qsets?
Would it be possible to render Player and Creator guides, if they exist, within the MWDK? It looks like we already have view templates to support them.

Otherwise, just a little bit of polish based on feedback items, and making sure all current changes on this branch are final, required, and complete, and it looks like it might be good to go.

qsets/.gitkeep

clpetersonucf · 2023-10-09T19:36:49Z

express.js

+	// 2. filter for materia-web image and named xxxx_phpfpm_1 name
+	// 3. pick the first line
+	// 4. pick the container name
+	let targetImage = execSync('docker ps -a --format "{{.Image}} {{.Names}}" | grep -e ".*materia:.* docker_app_.*" | head -n 1 | cut -d" " -f2');


This line continues to give us problems. Some versions of docker-compose use hyphens, others use underscores, and the server does not handle a search failure well. We can modify the search pattern grep is using to: ".*materia:.* docker[-_]app[-_].*" which will catch either option.

Also it'd be nice if the server gave a more well-handled response if the docker install process runs into an issue

.vscode/settings.json

package.json

webpack-widget.js

cayb0rg · 2023-10-10T20:08:58Z

Thanks for the feedback!

New changes:

Updated materia-widget-dependencies package version to 0.2.0
Fixed link to guides.js in guide pages
Updated scorescreen instructions
The kit should support demo and one additional instance. I've added the ability to have multiple widget instances.
Made the widget installation process more verbose so that all errors are forwarded to the user as they are received

clpetersonucf

There might be some edge case issues we haven't encountered yet, but nothing show-stopping. By all accounts the new dev kit does the job required and works great.

Really excellent work. This was a significant undertaking and you knocked it out of the park.

FrenjaminBanklin

Super minor nitpick with some inline spacing, but nothing I'd hold up a release for.

I couldn't get the score screen working with valid JSON, but considering the latest version of the MWDK doesn't have a functional score screen either I'm not sure I'd hold up a release for that, either.

Everything looks like it's working well otherwise.

FrenjaminBanklin · 2023-10-24T15:03:54Z

express.js

+const path            	= require('path');
+const fs              	= require('fs');
+const express         	= require('express')
+const qsets           	= path.join(__dirname, 'qsets');
+const yaml            	= require('yamljs');
+const { execSync }    	= require('child_process');
+const waitUntil       	= require('wait-until-promise').default
+const { v4: uuidv4 }    = require('uuid')
+const sharp           	= require('sharp')
+const util				= require('util');
+const cors 				= require('cors')
+const hbs				= require('hbs');
+
+// common paths used here
+const srcPath 				= path.join(process.cwd(), 'src') + path.sep
+const outputPath 			= path.join(process.cwd(), 'build') + path.sep
+
+// Webpack middleware setup
+const webpack 							= require('webpack');
+const webpackDevMiddleware 	= require('webpack-dev-middleware');


This doesn't really matter, but stick to spaces for inline alignment.

Would prefer to remove all the extraneous white space, personally.

cayb0rg and others added 20 commits May 3, 2023 08:10

stashing changes

0cf22b4

Loads React widgets and most pages

a1c94ea

A little bit of cleanup

849a55f

Renaming files

54e254c

Prevents webpack from loading JS prematurely - a fix for AngularJS wi…

3bc7342

…dgets

Update CSS links

2ba1835

Better support for markdown guides

36f19ab

adds progress bar to installer window

5ece256

removes storage panel and qset panel buttons from header

a6cbfbe

enables template MWDK startup

9700f6e

changing how entry files are handled in htmlwebpackplugin

06f2e63

Update package.json

9ca2908

Merge

066899f

Adds scorescreen and changes static assets path

f744710

Add demo instance option to landing page

e407cee

Cleans up previous demos on start

7ba4f28

Updates the helper annotation tool links

8d02e05

Updates .wigt asset generator plugin to webpack 5

5e0fa79

Update scorepage message and cleanup

692c4ae

update yarn.lock

c0b134a

This was referenced Sep 12, 2023

MWDK update & generation support ucfopen/crossword-materia-widget#56

Open

Issue/mwdk-upgrade ucfopen/enigma-materia-widget#45

Closed

Issue/mwdk upgrade ucfopen/this-or-that-materia-widget#57

Closed

cayb0rg added 7 commits September 19, 2023 16:36

Fixes default scorescreen and adds custom score screen support

29b217a

Remove empty file

2481cb7

Add instructions for finding JSON file

0057f69

Fix log and qset ids and account for different question formatting

7f81262

Update default entries

86a8ee7

Quick fix for response data

961efc7

Update file paths for clarity

2bcd328

cayb0rg marked this pull request as ready for review September 28, 2023 14:13

ColbyZW mentioned this pull request Sep 28, 2023

Update simple survey to work with the new dev kit ucfopen/survey-materia-widget#73

Open

Update materia asset urls

80fdd5e

clpetersonucf requested changes Oct 9, 2023

View reviewed changes

cayb0rg and others added 5 commits October 10, 2023 08:48

Fix link to guides.js

abe68d6

Update scorescreen instructions and links on score page

01c3b49

Add ability to have multiple widget instances

17014cc

Make widget installation process more verbose

08dd9d6

Adds required empty qsets dir

b60927a

cayb0rg and others added 4 commits October 10, 2023 16:12

Fix to player link on creator page

2c7f50e

merge in qsets/.gitkeep

d0cca52

Style adjustments for dev kit pages. Some small code cleanup.

579c08e

Removes deprecated travis config

7ab2b0c

clpetersonucf previously approved these changes Oct 20, 2023

View reviewed changes

fixes issue where qsets dir cleanup removes .gitkeep

b63235e

clpetersonucf dismissed their stale review via b63235e October 20, 2023 20:35

update package version to 3.0.0

e3305af

FrenjaminBanklin previously approved these changes Oct 24, 2023

View reviewed changes

alignment fixes in the express.js file

e3f98b2

clpetersonucf dismissed FrenjaminBanklin’s stale review via e3f98b2 October 24, 2023 17:12

clpetersonucf merged commit 62da49b into ucfopen:master Oct 24, 2023

cayb0rg mentioned this pull request Oct 31, 2023

Issue/mwdk upgrade ucfopen/flash-cards-materia-widget#32

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue/webpack 5 upgrade #106

Issue/webpack 5 upgrade #106

cayb0rg commented May 25, 2023 •

edited

Loading

clpetersonucf left a comment

clpetersonucf Oct 9, 2023

clpetersonucf Oct 9, 2023

cayb0rg commented Oct 10, 2023

clpetersonucf left a comment

FrenjaminBanklin left a comment

FrenjaminBanklin Oct 24, 2023

Issue/webpack 5 upgrade #106

Issue/webpack 5 upgrade #106

Conversation

cayb0rg commented May 25, 2023 • edited Loading

MWDK Upgrade

Requirements:

Local setup

Widget

AngularJS Widgets

Running this MWDK PR

Known Bugs / To-Do

Updated Widgets (PRs made)

clpetersonucf left a comment

Choose a reason for hiding this comment

clpetersonucf Oct 9, 2023

Choose a reason for hiding this comment

clpetersonucf Oct 9, 2023

Choose a reason for hiding this comment

cayb0rg commented Oct 10, 2023

clpetersonucf left a comment

Choose a reason for hiding this comment

FrenjaminBanklin left a comment

Choose a reason for hiding this comment

FrenjaminBanklin Oct 24, 2023

Choose a reason for hiding this comment

cayb0rg commented May 25, 2023 •

edited

Loading