forked from NaiboWang/EasySpider
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
73 changed files
with
4,857 additions
and
533 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,9 @@ | ||
.idea/ | ||
node_modules/ | ||
ServiceWrapper/ | ||
Data/ | ||
/GPUCache | ||
*.exe | ||
*.ini | ||
*.7z | ||
*.mp4 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -7,3 +7,5 @@ dist/ | |
build/ | ||
__pycache__/ | ||
*.spec | ||
Chrome/ | ||
Data/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
pyinstaller -F --icon=favicon.ico ServiceWrapper_ExcuteStage.py | ||
出现permission denied请关闭杀毒软件 |
Large diffs are not rendered by default.
Oops, something went wrong.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,28 +1,28 @@ | ||
-----BEGIN PRIVATE KEY----- | ||
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCuAZmXoS3RN1dQ | ||
IWpPjMZMX08JQRiR9XF8fPuZ4r/aauafNKncQmC2FEFylAcBMaRVSjAsZlXVqTlE | ||
s1LRtt9PiJEiDv73alerwznzNfuN0Dqrj2YpvKlck/sdFDeJNGi3dAl4kWZOKlE0 | ||
kpHXO6BQBKB9t9Up5qBku0epGYNZ6/foCl82U5bw8bBaZ5p0F2hhE4/PWP0p/MED | ||
HR7yvnr5qO0B6qQia/59q4opxH6sswa6XAc3mgU1Yq8m1cXRctmvxCBt7biU7Gpw | ||
jLB4Qdgikq02hWxn1YUbGAjHRjLrNxOIFBP6Sx2SNllM664bUmzwchnEzsLILyDQ | ||
jzKlafobAgMBAAECggEAGdp7APXIYkbKs56DYSvAECYzuWLxTPPJgimVExmG+o4v | ||
kzAL8iyamfMLK676wMlTaqi0h6RHfztpcDUEn8wqrx0naIa53fZFmGulTJGUvU6s | ||
U5k2y19l+oSyJsaTidCTo85F1KLQhISdFCaYLKvbTIhOIDUUIumf+IIq9OD+4c6C | ||
nhAKhkjspDt1ctyFy9QXE5Wbi6o8AdOuaw9QW+y7msi1kcTWovxFBQkN1Ck4D2m7 | ||
y72M5dDYq3nWEyQDJvioDM2yxvywiX3piHjULBj60Ybt52EEyRNlBMkbafU0puYU | ||
aajlPSv0X0mQjdtYLmd2dEVTH5rNLkVRwtx0i5MYoQKBgQDnJkJY6sj6HzhnFtWo | ||
NiorwAteNG6fIchZPAJjdtAQNu2stakGtnRoQ3WfVi2ePUKntoyqS4WanJE/z4pX | ||
lTMAePB8madwqe+AgSJzl0+YnOqBQCk6DeIrJ9u0O+nb7wlkZMbaULj6Sx8bg+N4 | ||
jM/MSD4uo1eYJdo6bIq6KNHXTQKBgQDAtqJlZ+EweMD9K652htwW0HE0IXD7zxMh | ||
8rODaubAp+glapW77GYltFrJ++Q8ObBdm+jLo8CbWUTus61qpsinv+CV//dIEiYp | ||
vbdf+oFnCEHvFTN1opzsS9ThYZhRnnonuxYmMCWsx55GG7oyAVtYLBUERm6XurLY | ||
FXkceAvzBwKBgD/UB8Qn6SFwV5HX813EvzZfjIQR9G4K1RkXAg3XNDMWB+GGNEt/ | ||
PHvCDQvXrzcf8XUAaq1nt1fvXgiB1dDDiVbbDoVJpLvuoQ0aG5pRsRASXIseXYQD | ||
a23BTuDhHn217yEC1wpX+gxbjBZ7/+c88vCVDl6wijKgWTeC23f2Z6ONAoGAP2CQ | ||
3cqg+2DrDxe7g41sejBI2n0Y/CcowqRftxuEEd3mcc+wuKHRIhtDNGQbtla+krqA | ||
f+A1qqFcEyiSIp4BJXHr+Ui52UDmvhl/YhgvUGQd0vPk/Y3iD7AMraZ8AnOc9s4H | ||
Rb3V1hG8EpBx1potoTy2GLbVDh8/S+Mb9mngfCcCgYEAlL/oEGR6j7K/29gF8KjQ | ||
J6HfNxdisP0b14her/mX0NSSv8WNFX77XO6pjXmpiIQPsm4I1KlzeFEVBy9RetGb | ||
5OKLQdfILNe8aDrYxMKcjbuA9zLqIzYep8siLdrYI1RDSxKt7ZUW6JejPthV0vr5 | ||
408+uF8kwRJscoRPn7+x7Uk= | ||
MIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQCr3o0YTuz2UMy7 | ||
qolwd4CWXet3SjJL0XawKKgp3ZRo78zER1Z/LQ5/rZqc6DISBPCycAoOCBk2EcBZ | ||
IlPKQFxq1PMj10ccyswjjb+4mf4ZRVveeflBqT9I4khrzwk9gymSZiD7TbkDc4fp | ||
oxoHCv2f7VMR5m/IwsnXEsm7QyuRSA6ETbhp2Faa2CIGI4l6r+gtuVYkUSGEN/0H | ||
0dk3kfh6rcjwvroLSzWcGJnhyOLDYxnV1kTHZDUBzGj0UOqL0fdpAuyXxSikadxa | ||
Ec4BZ4s3GHIlFR5N7itrwYN28IInLObxUQXJ3unEdnon/T/XhC5KsdjMp8D6Xb8/ | ||
0dObgaG/AgMBAAECggEAR0xyZQCs/fwECuBS1uTEY9y/h4OwvNI+9hJPvNpurF+6 | ||
pzxe7jBPYPkweGI9D0ucXLHtAegrKUqQ7Ik4kGFF1Y3U9evjVNbue9tciyTbDDnB | ||
RFh+ZlZjagVxfMPtNeb5MoIKsaYLGBrv6aUfcYeGMre96+GYQwVHvWDOblCNvN0k | ||
fBInSGlDcLkx9EJ9vFbCf8IErg4RWtiEdSvSG1s99IPc17j4cJn4qq9OyQrECpdz | ||
y7AnkvsMOiOzQawO+C2I5SgA26eaxNJrz/3ph4r1x1iCdPvY7iD/awkQXCgid6kq | ||
Z0R8ByxTbwqr4NYXtUyHVkwiL9QZSKjnvrOfdmm0iQKBgQDkQ5KRMftmDi9cJfOj | ||
0xY/TwCymsWp+MdMQ6YKAd57oMM+2DzgZ//LFgo4rcYWjTGlErE8tXhT4fM5PEUQ | ||
1CgwozmkPcVnRzLyiNWNpiMCzPVPsXDxzoEQFHL0qLA/Xj7npH5d+Eg6yunDc9JS | ||
D/OlqSEZQ6W4lWFvAkz7Mh5M6QKBgQDAwMFzAMYVXwcRmH0gvzmqA46MqjdfaXyq | ||
NzNADiVCaGgqYpNygeftiHGCBQiXr3QkVVp+XihbY/5KnzVYiS0ICKMCwqxizvRT | ||
gR3N4+UyVaK9FTWLgtSu8VN4JM3E+nigBO/QfD4WI4VZ3tZEsOveV70j97NpfaPN | ||
lhjr2J0wZwKBgG3pj7i0bY5dxDZF3ASFjw/a9cYKuqU2spipdlkZP00eQwWOz/lq | ||
eoQKz88s9dZEFKSc1JUb+J3Djf+AYu1Qiy2oWwgX6mbppMGeW95CIlel43WFRpJY | ||
0lKw1g2y8HMC6Z1W2rZa6ETPEjLYIWz8W2DoiJSGu1SfMXRnkjPelTKJAoGACvbs | ||
Ci6xFeYh8M5Lz+EQ1qr9IONN4w+NF+Gr+KjVVcG6qy6QVKMvHkP0sQC4TGieorJ4 | ||
Q1f307sMbBJCZpbnCN305+NLCxPasiVWHLAqCYL1juv178mxb4IqzVrKmbnlwrSF | ||
L8bhgUDkBQi4B5BI2o0DJVihzA5pkvhG0qOvzWECgYAHITSiOJBF1Lr0hw7JnS5r | ||
Hre8bQdbDeqSd5iRM/+0MJnBvkp7RDgvh0c01Wc90A0gjhhPqBS9z5UNptiJKhJQ | ||
kYDP7iiftM7FsG45IGKf6RrulLk6pCyFEFb/roBlC1xy/6uNCqAHbd+6K6UjQAFB | ||
+c9hCtSDDh3/irZrY34lDw== | ||
-----END PRIVATE KEY----- |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,31 +1,155 @@ | ||
# 请您Star Please Star | ||
|
||
如果你觉得此工具不错,请轻轻点击此页面右上角**Star**按钮增加项目曝光度,谢谢! | ||
|
||
If you think this tool is good, please gently click the **Star** button in the upper right corner at this page to increase the project exposure, thank you! | ||
|
||
# 无代码服务可视化Web数据采集爬虫器 Code-Free Visual Web Data Crawler/Spider (Service Wrapper) | ||
|
||
一个可以可视化无代码设计和执行的面向服务架构的爬虫软件(服务包装器)。 | ||
A service oriented architecture GUI visual code-free web crawler/spider (service wrapper). | ||
|
||
|
||
# 发布版本 | ||
## Windows版本可执行程序:<http://naibo.wang/exe/ServiceWrapper.7z> | ||
打开压缩包内的ServiceWrapper.exe即可在Windows10系统执行,无需配置环境。 | ||
## Windows版本可执行程序:<https://github.com/NaiboWang/ServiceWrapper/releases/download/v0.5.0/ServiceWrapper.7z> | ||
打开压缩包内的ServiceWrapper.exe即可在Windows10/11或以上系统执行,无需配置环境(其余Windows系统需手动安装.net Framework 4.5)。 | ||
数据存储后放在Data/文件夹内 | ||
## 视频教程:<http://naibo.wang/exe/tutorial.mp4> | ||
## 中文视频教程:<https://github.com/NaiboWang/ServiceWrapper/releases/download/v0.5.0/tutorial_CN.mp4> | ||
|
||
|
||
|
||
<!-- START doctoc generated TOC please keep comment here to allow auto update --> | ||
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> | ||
# 目录 | ||
注:文档待完善 | ||
- [请您Star Please Star](#请您star-please-star) | ||
- [无代码服务可视化Web数据采集爬虫器 Code-Free Visual Web Data Crawler/Spider (Service Wrapper)](#无代码服务可视化web数据采集爬虫器-code-free-visual-web-data-crawlerspider-service-wrapper) | ||
- [发布版本](#发布版本) | ||
- [Windows版本可执行程序:<https://github.com/NaiboWang/ServiceWrapper/releases/download/v0.5.0/ServiceWrapper.7z>](#windows版本可执行程序httpsgithubcomnaibowangservicewrapperreleasesdownloadv050servicewrapper7z) | ||
- [中文视频教程:<https://github.com/NaiboWang/ServiceWrapper/releases/download/v0.5.0/tutorial_CN.mp4>](#中文视频教程httpsgithubcomnaibowangservicewrapperreleasesdownloadv050tutorial_cnmp4) | ||
- [目录](#目录) | ||
- [界面截图](#界面截图) | ||
- [软件界面示例](#软件界面示例) | ||
- [块和子块及表单定义](#块和子块及表单定义) | ||
- [已选中和待选择示例](#已选中和待选择示例) | ||
- [京东商品块选择示例:](#京东商品块选择示例) | ||
- [京东商品标题自动匹配选择示例](#京东商品标题自动匹配选择示例) | ||
- [分块选择所有子元素示例](#分块选择所有子元素示例) | ||
- [同类型元素自动和手动匹配示例](#同类型元素自动和手动匹配示例) | ||
- [四种选择方式示例](#四种选择方式示例) | ||
- [输入文字示例](#输入文字示例) | ||
- [循环点击58同城房屋标题以进入详情页采集示例](#循环点击58同城房屋标题以进入详情页采集示例) | ||
- [采集元素文本示例](#采集元素文本示例) | ||
- [流程图界面介绍](#流程图界面介绍) | ||
- [循环选项示例](#循环选项示例) | ||
- [循环点击下一页示例](#循环点击下一页示例) | ||
- [条件分支示例](#条件分支示例) | ||
- [完整采集流程图示例](#完整采集流程图示例) | ||
- [完整采集流程图转换为常规流程图示例](#完整采集流程图转换为常规流程图示例) | ||
- [服务信息示例](#服务信息示例) | ||
- [服务调用示例](#服务调用示例) | ||
- [58 同城房源信息采集服务部分采集结果展示](#58-同城房源信息采集服务部分采集结果展示) | ||
- [服务包装手动版程序结构](#服务包装手动版程序结构) | ||
- [Chrome插件部分](#chrome插件部分) | ||
- [后台流程图部分](#后台流程图部分) | ||
- [服务展示部分](#服务展示部分) | ||
- [C#部分](#c部分) | ||
- [后台服务页面](#后台服务页面) | ||
- [服务执行](#服务执行) | ||
|
||
<!-- END doctoc generated TOC please keep comment here to allow auto update --> | ||
|
||
## 界面截图 | ||
|
||
#### 软件界面示例 | ||
|
||
![pic](media/Picture.png) | ||
#### 块和子块及表单定义 | ||
|
||
![pic](media/Picture2.png) | ||
#### 已选中和待选择示例 | ||
|
||
![pic](media/Picture7.png) | ||
#### 京东商品块选择示例: | ||
|
||
![pic](media/Picture1.png) | ||
|
||
|
||
#### 京东商品标题自动匹配选择示例 | ||
|
||
![pic](media/Picture5.png) | ||
#### 分块选择所有子元素示例 | ||
|
||
![pic](media/Picture6.png) | ||
|
||
#### 同类型元素自动和手动匹配示例 | ||
|
||
![pic](media/Picture8.png) | ||
#### 四种选择方式示例 | ||
|
||
![pic](media/Picture90.png) | ||
#### 输入文字示例 | ||
|
||
![pic](media/Picture10.png) | ||
#### 循环点击58同城房屋标题以进入详情页采集示例 | ||
|
||
![pic](media/Picture12.png) | ||
#### 采集元素文本示例 | ||
|
||
![pic](media/Picture14.png) | ||
#### 流程图界面介绍 | ||
|
||
![pic](media/Picture4.png) | ||
#### 循环选项示例 | ||
|
||
![pic](media/Picture9.png) | ||
|
||
#### 循环点击下一页示例 | ||
|
||
![pic](media/Picture11.png) | ||
|
||
#### 条件分支示例 | ||
|
||
![pic](media/Picture13.png) | ||
#### 完整采集流程图示例 | ||
|
||
![pic](media/Picture16.png) | ||
#### 完整采集流程图转换为常规流程图示例 | ||
|
||
![pic](media/Picture91.png) | ||
#### 服务信息示例 | ||
|
||
![pic](media/Picture15.png) | ||
|
||
#### 服务调用示例 | ||
|
||
![pic](media/Picture17.png) | ||
|
||
|
||
#### 58 同城房源信息采集服务部分采集结果展示 | ||
![pic](media/Picture18.png) | ||
|
||
# 服务包装手动版程序结构 | ||
## Chrome插件部分 | ||
## 服务包装手动版程序结构 | ||
### Chrome插件部分 | ||
* Extension/app内的文件 | ||
|
||
## 后台流程图部分 | ||
### 后台流程图部分 | ||
* ServiceGrid/frontEnd/FlowChart.html | ||
* ServiceGrid/frontEnd/FlowChart.js | ||
* ServiceGrid/frontEnd/FlowChart.css | ||
* ServiceGrid/frontEnd/logic.css | ||
|
||
## 服务展示部分 | ||
### 服务展示部分 | ||
* 服务列表:ServiceGrid/frontEnd/serviceList.html | ||
* 服务信息:ServiceGrid/frontEnd/serviceInfo.html | ||
* 新服务:ServiceGrid/frontEnd/newService.html | ||
* 调用服务:ServiceGrid/frontEnd/invokeService.html | ||
|
||
## C#部分 | ||
### C#部分 | ||
* C#/内的文件 | ||
|
||
## 后台服务页面 | ||
### 后台服务页面 | ||
* Django后台:ServiceGrid/backEnd/* | ||
|
||
## 服务执行 | ||
### 服务执行 | ||
* ExcuteStage/ServiceWrapper_ExcuteStage.py |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
__pycache__/ | ||
nohup.out | ||
*.pid |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
dbconfig.py |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.