Skip to content

Commit

Permalink
Merge pull request #6 from scrapfly/deno
Browse files Browse the repository at this point in the history
Deno rework
  • Loading branch information
Granitosaurus authored Jul 29, 2024
2 parents 6caab7a + e5a1a9f commit 0c93447
Show file tree
Hide file tree
Showing 36 changed files with 3,133 additions and 10,766 deletions.
15 changes: 15 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
root = true

[*]
charset = utf-8
end_of_line = lf
trim_trailing_whitespace = true
insert_final_newline = true

[*.md]
insert_final_newline = false
trim_trailing_whitespace = false

[*.{js,json,ts,mts,yml,yaml}]
indent_size = 2
indent_style = space
2 changes: 0 additions & 2 deletions .eslintignore

This file was deleted.

23 changes: 0 additions & 23 deletions .eslintrc.json

This file was deleted.

20 changes: 0 additions & 20 deletions .github/workflows/nodejs.yml

This file was deleted.

39 changes: 24 additions & 15 deletions .github/workflows/publish.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

name: Publish to NPM

on:
Expand All @@ -11,18 +12,26 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- uses: volta-cli/action@v1
- run: npm ci --no-audit
- run: npm run lint --if-present
- run: npm test
- run: npm run build --if-present
env:
CI: true
- name: Setup .npmrc file to publish to npm
run: |
echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > .npmrc
env:
NPM_TOKEN: ${{secrets.NPM_AUTOMATION_TOKEN}}
- name: Publish to NPM
run: npm publish
- name: Clone repository
uses: actions/checkout@v3

- name: Install Deno
uses: denoland/setup-deno@v1
with:
deno-version: v1.x

- name: Run Deno build script
run: deno task build-npm

- name: Navigate to npm directory
run: cd ./npm

- name: Setup .npmrc file to publish to npm
run: |
echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > .npmrc
env:
NPM_TOKEN: ${{ secrets.NPM_AUTOMATION_TOKEN }}

- name: Publish to NPM
run: npm publish --dry-run
working-directory: ./npm
28 changes: 28 additions & 0 deletions .github/workflows/run-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: Run tests

on:
workflow_dispatch:
pull_request:


jobs:
build:
runs-on: ubuntu-latest

steps:
- name: Clone repository
uses: actions/checkout@v3

- name: Install Deno
uses: denoland/setup-deno@v1
with:
deno-version: v1.x

- name: Run Deno Test
run: deno task test

- name: Build JSR
run: deno task build-jsr

- name: Build NPM
run: deno task build-npm
8 changes: 7 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,8 @@
node_modules
coverage
coverage
scripts
npm
dev
cov_profile
.vscode
.history
3 changes: 0 additions & 3 deletions .prettierignore

This file was deleted.

14 changes: 0 additions & 14 deletions .prettierrc

This file was deleted.

47 changes: 33 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# Scrapfly SDK

`npm install scrapfly-sdk`
`npm install scrapfly-sdk`
`deno add jsr:@scrapfly/scrapfly-sdk`
`bun jsr add @scrapfly/scrapfly-sdk`

Typescript/NodeJS SDK for [Scrapfly.io](https://scrapfly.io/) web scraping API which allows to:
Typescript/Javascript SDK for [Scrapfly.io](https://scrapfly.io/) web scraping API which allows to:

- Scrape the web without being blocked.
- Use headless browsers to access Javascript-powered page data.
Expand All @@ -11,14 +13,21 @@ Typescript/NodeJS SDK for [Scrapfly.io](https://scrapfly.io/) web scraping API w

For web scraping guides see [our blog](https://scrapfly.io/blog/) and [#scrapeguide](https://scrapfly.io/blog/tag/scrapeguide/) tag for how to scrape specific targets.

The SDK is distributed through:
- [npmjs.com/package/scrapfly-sdk](https://www.npmjs.com/package/scrapfly-sdk)
- [jsr.io/@scrapfly/scrapfly-sdk](https://jsr.io/@scrapfly/scrapfly-sdk)

## Quick Intro

1. Register a [Scrapfly account for free](https://scrapfly.io/register)
2. Get your API Key on [scrapfly.io/dashboard](https://scrapfly.io/dashboard)
3. Start scraping: 🚀

```javascript
// node or bun:
import { ScrapflyClient, ScrapeConfig } from 'scrapfly-sdk';
// deno:
import { ScrapflyClient, ScrapeConfig } from 'jsr:@scrapfly/scrapfly-sdk';

const key = 'YOUR SCRAPFLY KEY';
const client = new ScrapflyClient({ key });
Expand Down Expand Up @@ -70,15 +79,25 @@ new ScrapeConfig({

## Development

Install and setup environment:

```shell
$ npm install
```

Build and test:

```shell
$ npm run build
$ npm run test
```
This is a Deno Typescript project that builds to NPM through [DNT](https://github.com/denoland/dnt).

- `/src` directory contains all of the source code with `main.ts` being the entry point.
- `__tests__` directory contains tests for the source code.
- `deno.json` contains meta information
- `build.ts` is the build script that builds the project to nodejs ESM package.
- `/npm` directory will be produced when `built.ts` is executed for building node package.

```bash
# make modifications and run tests
$ deno task test
# format
$ deno fmt
# lint
$ deno lint
# publish JSR:
$ deno publish
# build NPM package:
$ deno build-npm
# publish NPM:
$ cd npm && npm publish
```
Loading

0 comments on commit 0c93447

Please sign in to comment.