Changes for builtins v1.88.1

Refactored documentation and added scripts to simplify IP checks.

Contributed on behalf of ST Microelectronics

Signed-off-by: Thomas Mäder <[email protected]>

.gitignore

@@ -3,4 +3,7 @@ lib
 *.log
 dist
 external-builtins/
+vscode-builtin-extensions
 dash-licenses.jar
+*.zip
+summary.txt

README.md

@@ -1,99 +1,17 @@
 # Built-in vscode extensions
 
-This extension contributes built-in VS Code extensions to Eclipse Theia applications.
+This extension contains code to build, package and publish the extensions that are included with VS Code.
 
-[![Open in Gitpod](https://gitpod.io/button/open-in-gitpod.svg)](https://gitpod.io/#https://github.com/theia-ide/vscode-builtin-extensions)
+We build/package them ourselves, from the MIT-licensed vscode repository, and then publish them individually to Open VSX. We do not modify the extensions, other than a couple of very minor adaptations, to make them suitable to work as standalone extensions.
 
-## Getting started (locally)
+The "vscode builtins", "vscode built-ins" or "vscode built-in extensions" are a set of extensions whose code resides in the public vscode repository. They are built along and bundled as a group, in the Visual Studio Code product as well as in products based on Code OSS and derivatives such as VSCodium. As such, they are not made available as individual .vsix packages, for use in other IDE applications, outside of the vscode family. This is why we have this repo here - to build, package and individually publish, the various built-in extensions.
 
-Install [nvm](https://github.com/creationix/nvm#install-script).
+Every sub-folder of vscode/extensions/ is one built-in vscode extension (with a couple of exceptions like node_modules after a build)
 
-    curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash
+## Getting started
 
-Install npm and node.
-
-    nvm install 18
-    nvm use 18
-
-NOTE: To re-build older `vscode` extensions requiring node 12, use the branch `node-12` on this repository and follow the corresponding `README.md` on that branch.
-
-Install yarn.
-
-    npm install -g yarn
-
-Install vscode.
-
-    cd vscode-builtin-extensions
-    git submodule init
-    git submodule update
-
-Install vscode prerequisite dependencies.
-
-    https://github.com/Microsoft/vscode/wiki/How-to-Contribute#prerequisites
-
-Pick a specific vscode version (optional)
-
-    cd <repo root>/vscode
-    git checkout 1.58.1
-
-## Build
-
-    yarn
-    yarn build:extensions
-
-## Packaging a built-in vscode extension.
-
-The version of the packaged built-in corresponds to the `version` present in the vscode sub-module's `package.json`. For `next` versions, an appropriate hash suffix is added.
-
-Latest / solid revision example:
-
-    cd vscode; git checkout 1.58.1; cd ..
-    yarn; yarn package-vsix:latest
-
-Next / interim revision example:
-
-    cd vscode; git checkout fe671f300845ca5161885125b1e12d43fc25ccf8; cd ..
-    yarn; yarn package-vsix:next
-
-The generated `.vsix` will be under folder `./dist`
-
-## Packaging built-in vscode extensions in an extension-pack.
-
-The version of the built-in extension-pack corresponds to the `version` present in the vscode sub-module's `package.json`. For `next` versions, an appropriate hash suffix is added.
-
-Latest / solid revision example:
-
-    cd vscode; git checkout 1.58.1; cd ..
-    yarn; yarn create-extension-pack:latest
-
-Next / interim revision example:
-
-    cd vscode; git checkout fe671f300845ca5161885125b1e12d43fc25ccf8; cd ..
-    yarn; yarn create-extension-pack:next
-
-The generated `.vsix` will be under the folder `./dist`
-
-## Publishing individual built-in vscode extensions and builtin-extension-packs to open-vsx
-
-The `ovsx` client is used to publish to an open-vsx registry. The default registry is set to the public instance at https://open-vsx.org.
-
-The environment variable `OVSX_REGISTRY_URL` may be set to configure publishing to a different registry URL.
-
-The environment variable `OVSX_PAT` is used to set the personal access token obtained from the registry.
-
-After packaging the extensions and extension-packs as `.vsix` (see above), you may examine/test them under the `dist` folder. Remove any extension that you do not wish to be published (e.g. those not working well), and when ready proceed with publishing:
-
-    yarn publish:vsix
-
-## Re-publishing individual built-in vscode extensions and built-in extension packs to open-vsx.
-
-### Solid version
-
-There is a GH action to help: `publish-vsx-specific-latest`. For this to work, the version to be published needs to be removed from open-vsx. Then one must push to branch `ovsx-publish`. Make sure the wanted solid version of the `vscode` git submodule is checked-out in the pushed change. We do not care about that branch - once the publishing is done, it can be force reset the next time.
-
-### Intermediary (next) version
-
-There is a GH action to help: `publish-vsx-specific-next`. For this to work, the version to be published needs to be removed from open-vsx. Then one must push to branch `ovsx-publish-next`. Make sure the wanted intermediary version of the `vscode` git submodule is checked-out in the pushed change. We do not care about that branch - once the publishing is done, it can be force reset the next time.
+Building the built-in `*.vsix` files locally is described in [Building.md](./doc/Building.md). If you need to publish a new version of the built-ins for use with Theia, please follow the 
+process described in [Publishing.md](./doc/Publishing.md). 
 
 ## License
 

doc/Building.md

@@ -0,0 +1,63 @@
+# Building VS Code built-in Extensions
+
+## Setup
+
+1. Install the VS Code prerequisites as described in (link)
+2. Open a command line inside this repo
+3. Set up the version of VS Code you want to build:
+    git submodule init
+    git submodule update
+4. Check out the version of VS code you want to use
+    cd vscode
+    git checkout <git tag or branch>
+5. Install project dependencies
+    yarn
+
+## Building
+Building the exensions from VS Code is done simply with 
+
+    yarn build:extensions
+
+This will compile a production ("minified") version of the built-in extensions into the `vscode/.build` folder. In order to produce unminified versions for debugging,
+you will need to edit the build script at `vscode/build/lib/extensions.js`. Find the line that creates the webpack config. It should look like this:
+```
+const webpackConfig = {
+   ...config,
+   ...{ mode: 'production' }
+};
+```
+Remove part saying `mode: production` and redo the build
+
+## Packaging
+
+### Packaging the built-in vscode extensions
+
+Once we have built our extensions, we can packge them into `*.vsix`-files using this package script:
+
+    yarn package-vsix:latest
+
+The script will produce `*.vsix` files in a folder called `./dist`. The vsix files will be named like `<name>-<vscode-version>.vsix`. Note that the publisher (msvscode)
+is not included.
+
+If you want to create a prerelease version, you can do so by invoking 
+
+    yarn package-vsix:next
+
+This will generate `*.vsix` files of the form `<name>-<vscode-version>-next.<commit hash>.vsix`
+
+**Implementation Note:** the VS Code build process puts some shared depenencies in a `node_modules` folder which is located in the "extensions" folder at run time. In order to 
+produce self-contained extensions, we need to include those modules (at the time, it's the typescript language server) into the packaged extensions (currently for `typescript-language-features` and `html-language-features`). The code doing this is located in `src/package-vsix.js`. We also need to patch the `typescript-language-features`
+extension because it contains a hard-code reference to `../node_modules`.
+
+### Creating the built-ins extension-pack
+
+We also create an extension pack from the internal and external built-ins into the `dist` folder with a package script. The file name will be of the form:
+builtin-extension-pack-<vscode version>.vsix.
+
+   yarn create-extension-pack:latest
+
+Again, we can produce a preview release of the form `builtin-extension-pack-<vscode version>-next-.vsix
+
+    yarn create-extension-pack:next
+
+Note that you will have to package the `next` versions of the built-in extensions before they can be included in a `next` extensions pack.