Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[xdoctest][task 226-230] reformat example code with google style in python/paddle/distributed/fleet/* #57787

Merged
merged 9 commits into from
Oct 12, 2023
Merged

[xdoctest][task 226-230] reformat example code with google style in python/paddle/distributed/fleet/* #57787

merged 9 commits into from
Oct 12, 2023

Conversation

jinyouzhi
Copy link
Contributor

@jinyouzhi jinyouzhi commented Sep 26, 2023 •
edited
Loading

@paddle-bot
Copy link

paddle-bot bot commented Sep 26, 2023

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@paddle-bot paddle-bot bot added the contributor External developers label Sep 26, 2023
@paddle-bot
Copy link

paddle-bot bot commented Sep 26, 2023

❌ The PR is not created using PR's template. You can refer to this Demo.
Please use PR's template, it helps save our maintainers' time so that more developers get helped.

@luotao1 luotao1 added the HappyOpenSource Pro 进阶版快乐开源活动，更具挑战性的任务 label Sep 27, 2023
@jinyouzhi jinyouzhi marked this pull request as ready for review September 27, 2023 05:29
@jinyouzhi
Copy link
Contributor Author

@sunzhongkai588 @SigureMo @megemini

megemini
megemini reviewed Sep 27, 2023
View reviewed changes
Copy link
Contributor

@megemini megemini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

有一些相同问题的没有逐个写出来，比如格式问题，输出问题 ～ 请检查一下 ：）

python/paddle/distributed/parallel.py Outdated Show resolved Hide resolved
python/paddle/distributed/fleet/recompute/recompute.py Outdated Show resolved Hide resolved
python/paddle/distributed/fleet/utils/__init__.py Outdated Show resolved Hide resolved
python/paddle/distributed/fleet/utils/fs.py Outdated Show resolved Hide resolved
python/paddle/distributed/fleet/utils/fs.py Outdated Show resolved Hide resolved
python/paddle/distributed/parallel.py Outdated Show resolved Hide resolved
python/paddle/distributed/parallel.py Outdated Show resolved Hide resolved
python/paddle/distributed/parallel.py Outdated Show resolved Hide resolved
python/paddle/distributed/parallel.py Outdated Show resolved Hide resolved
python/paddle/distributed/parallel.py Outdated Show resolved Hide resolved
@jinyouzhi
Copy link
Contributor Author

有一些相同问题的没有逐个写出来，比如格式问题，输出问题 ～ 请检查一下 ：）

不太确定是不是HDFS这种凡是设定外部地址的都SKIP？

@megemini
Copy link
Contributor

megemini commented Oct 1, 2023

不太确定是不是HDFS这种凡是设定外部地址的都SKIP？

分布式这部分确认比较麻烦，我自己的测试方法是，在 aistudio 上用 gpu 环境安装最新的 develop 版本，然后测试代码能否运行。

develop 版本好像是编译了分布式的部分，所以，如果：

  • 能运行，则把这部分示例代码标记为 doctest: +REQUIRES(env:DITSTRIBUTED)
  • 不能跑，则把这部分示例代码标记为 doctest: +SKIP("depend on external files")

另外，也可以参考单元测试的情况 ～

我个人对分布式这块不是很有把握，上面这种测试方法参考一下？！～～～

@jinyouzhi
Copy link
Contributor Author

不太确定是不是HDFS这种凡是设定外部地址的都SKIP？

分布式这部分确认比较麻烦，我自己的测试方法是，在 aistudio 上用 gpu 环境安装最新的 develop 版本，然后测试代码能否运行。

develop 版本好像是编译了分布式的部分，所以，如果：

  • 能运行，则把这部分示例代码标记为 doctest: +REQUIRES(env:DITSTRIBUTED)

  • 不能跑，则把这部分示例代码标记为 doctest: +SKIP("depend on external files")

另外，也可以参考单元测试的情况 ～

我个人对分布式这块不是很有把握，上面这种测试方法参考一下？！～～～

thanks 好主意，aistudio现在有多卡了

@paddle-ci-bot
Copy link

paddle-ci-bot bot commented Oct 6, 2023

Sorry to inform you that 74feae5's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

@luotao1 luotao1 mentioned this pull request Oct 10, 2023
Closed
@jinyouzhi jinyouzhi requested a review from megemini October 12, 2023 02:04
@jinyouzhi
Copy link
Contributor Author

@megemini updated. AFSClient 这部分大概是develop版本默认WITH_PSLIB=OFF，测不到，全部SKIP了。

megemini
megemini reviewed Oct 12, 2023
View reviewed changes
Copy link
Contributor

@megemini megemini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFSClient 部分比较简单，应该问题不大。

还有两个地方改一下应该就可以了 ～ 辛苦 :)

python/paddle/distributed/fleet/utils/fs.py Outdated Show resolved Hide resolved
python/paddle/distributed/fleet/utils/hybrid_parallel_inference.py Outdated Show resolved Hide resolved
megemini
megemini approved these changes Oct 12, 2023
View reviewed changes
Copy link
Contributor

@megemini megemini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM～ 辛苦 🫡

@luotao1 luotao1 merged commit 17da03f into PaddlePaddle:develop Oct 12, 2023
Frida-a pushed a commit to Frida-a/Paddle that referenced this pull request Oct 14, 2023
@jinyouzhi @Frida-a
[xdoctest][task 226-230] reformat example code with google style in `…
bb93ea4 
…python/paddle/distributed/fleet/*` (PaddlePaddle#57787)

* [Doctest]fix No.226-230, test=docs_preview

* reply comments

* missing some

* missing some

* modify fs.py

* modify hybrid_parallel_inference.py

* modify parallel.py

* fix examples
jiahy0825 pushed a commit to jiahy0825/Paddle that referenced this pull request Oct 16, 2023
@jinyouzhi @jiahy0825
[xdoctest][task 226-230] reformat example code with google style in `…
1fa4ba1 
…python/paddle/distributed/fleet/*` (PaddlePaddle#57787)

* [Doctest]fix No.226-230, test=docs_preview

* reply comments

* missing some

* missing some

* modify fs.py

* modify hybrid_parallel_inference.py

* modify parallel.py

* fix examples
danleifeng pushed a commit to danleifeng/Paddle that referenced this pull request Nov 14, 2023
@jinyouzhi
[xdoctest][task 226-230] reformat example code with google style in `…
6e16012 
…python/paddle/distributed/fleet/*` (PaddlePaddle#57787)

* [Doctest]fix No.226-230, test=docs_preview

* reply comments

* missing some

* missing some

* modify fs.py

* modify hybrid_parallel_inference.py

* modify parallel.py

* fix examples
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@megemini megemini megemini approved these changes

@luotao1 luotao1 luotao1 approved these changes

Assignees

@luotao1 luotao1

@SigureMo SigureMo

@sunzhongkai588 sunzhongkai588

Labels
contributor External developers HappyOpenSource Pro 进阶版快乐开源活动，更具挑战性的任务
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@jinyouzhi @megemini @luotao1 @SigureMo @sunzhongkai588