Zygote AD backend #783

mohamed82008 · 2019-05-12T07:56:29Z

This is a WIP PR that introduces Zygote to our AD family. MvNormal is still not working and some inplace operations are tripping Zygote, so I need to figure out how to make them play well together.

xukai92 · 2019-05-14T10:56:26Z

By inplace operations do you mean the way we use vi::VarInfo requires the following?

vi[spl] = theta
model(vi, spl)

mohamed82008 · 2019-05-14T11:43:49Z

Seems so https://travis-ci.org/TuringLang/Turing.jl/jobs/531342906#L624, it is unclear to me what the solution here is.

cpfiffer · 2019-05-14T13:45:26Z

Doesn't seem like much, so far: FluxML/Zygote.jl#75. The PR to fix this hasn't made it through and may have some caveats for our case.

yebai · 2019-05-29T16:17:19Z

@mohamed82008 @willtebbutt Just a quick thought, can we use Tracker to define an in-place mutation function together with customised gradient rules, and then use it inside Zygote?

yebai · 2019-12-17T17:21:59Z

@mohamed82008 Perhaps revisit this PR when refactoring VarInfo?

mohamed82008 · 2019-12-17T23:08:18Z

Sure, I will give it another look. Although the underlying issue in Zygote (FluxML/Zygote.jl#265) was not resolved yet. But maybe I can work-around it.

mohamed82008 · 2020-01-14T12:00:56Z

FluxML/Zygote.jl#459

mohamed82008 · 2020-01-14T12:06:19Z

TuringLang/Bijectors.jl#73

mohamed82008 · 2020-01-14T12:06:31Z

TuringLang/DynamicPPL.jl#31

src/core/ad.jl

Project.toml

docs/src/using-turing/compiler.md

src/core/compat/zygote.jl

src/core/ad.jl

mohamed82008 · 2020-01-16T14:10:13Z

I added tests for Zygote, many of which are currently failing. I will try shooting them down one by one in the next few days.

mohamed82008 · 2020-01-18T08:52:57Z

Ran into FluxML/Zygote.jl#468 for Wishart distribution.

mohamed82008 · 2020-01-18T12:23:33Z

Tests should pass after the following PRs are merged and released:

in this order.

mohamed82008 · 2020-03-11T02:00:07Z

The only test failures here are because of DistributionsAD. I will work on them.

src/variational/VariationalInference.jl

Project.toml

src/Turing.jl

src/core/ad.jl

Co-Authored-By: David Widmann <[email protected]>

willtebbutt

Minor compat things.

Project.toml

…into mt/zygote_ad

Project.toml

Co-Authored-By: David Widmann <[email protected]>

src/core/ad.jl

Co-Authored-By: David Widmann <[email protected]>

src/core/ad.jl

Co-Authored-By: David Widmann <[email protected]>

…into mt/zygote_ad

src/core/ad.jl

src/variational/VariationalInference.jl

cpfiffer

Lookgs great! 👍

yebai · 2020-03-15T20:50:10Z

thanks @mohamed82008. Looks good to me, ready to merge from my side!

mohamed82008 mentioned this pull request Jun 27, 2019

Package scope and comparison to Turing chriselrod/ProbabilityModels.jl#6

Open

mohamed82008 force-pushed the mt/zygote_ad branch from f88e568 to 09ac6a7 Compare July 20, 2019 05:52

mohamed82008 force-pushed the mt/zygote_ad branch from 09ac6a7 to 9adad9b Compare January 14, 2020 12:05

devmotion reviewed Jan 14, 2020

View reviewed changes

src/core/ad.jl Outdated Show resolved Hide resolved

devmotion reviewed Jan 14, 2020

View reviewed changes

Project.toml Outdated Show resolved Hide resolved

devmotion mentioned this pull request Jan 14, 2020

Zygote support - workaround setproperty! and getproperty TuringLang/DynamicPPL.jl#31

Merged

devmotion reviewed Jan 15, 2020

View reviewed changes

docs/src/using-turing/compiler.md Outdated Show resolved Hide resolved

devmotion reviewed Jan 15, 2020

View reviewed changes

src/core/compat/zygote.jl Outdated Show resolved Hide resolved

devmotion reviewed Jan 15, 2020

View reviewed changes

src/core/compat/zygote.jl Show resolved Hide resolved

devmotion reviewed Jan 15, 2020

View reviewed changes

src/core/ad.jl Outdated Show resolved Hide resolved

mohamed82008 mentioned this pull request Jan 18, 2020

Converting UniformScaling to Matrix causes mutation FluxML/Zygote.jl#468

Closed

mohamed82008 force-pushed the mt/zygote_ad branch from 4f1fed6 to 51afb27 Compare January 31, 2020 22:28

mohamed82008 force-pushed the mt/zygote_ad branch from eb56f89 to 75d3e66 Compare March 10, 2020 11:12

mohamed82008 force-pushed the mt/zygote_ad branch from 189b860 to f235cf0 Compare March 15, 2020 05:19

mohamed82008 added 3 commits March 15, 2020 16:20

squash commit

e49f6d9

minor fixes

510b5e5

bump DAD compat version

c2c21cd

apply David's suggestions

2cc0357

devmotion reviewed Mar 15, 2020

View reviewed changes

src/variational/VariationalInference.jl Show resolved Hide resolved

devmotion reviewed Mar 15, 2020

View reviewed changes

Project.toml Outdated Show resolved Hide resolved

devmotion reviewed Mar 15, 2020

View reviewed changes

src/Turing.jl Outdated Show resolved Hide resolved

devmotion reviewed Mar 15, 2020

View reviewed changes

src/core/ad.jl Outdated Show resolved Hide resolved

Update src/Turing.jl

2ea9519

Co-Authored-By: David Widmann <[email protected]>

willtebbutt reviewed Mar 15, 2020

View reviewed changes

Project.toml Show resolved Hide resolved

Project.toml Show resolved Hide resolved

mohamed82008 added 3 commits March 15, 2020 23:58

David's comment

d748eda

David's and Will's comments

aca7549

Merge branch 'mt/zygote_ad' of https://github.com/TuringLang/Turing.jl …

c69d053

…into mt/zygote_ad

devmotion reviewed Mar 15, 2020

View reviewed changes

Project.toml Outdated Show resolved Hide resolved

devmotion reviewed Mar 15, 2020

View reviewed changes

Project.toml Outdated Show resolved Hide resolved

mohamed82008 and others added 2 commits March 16, 2020 00:22

Update Project.toml

4bacdaa

Co-Authored-By: David Widmann <[email protected]>

Update Project.toml

5789f80

Co-Authored-By: David Widmann <[email protected]>

devmotion reviewed Mar 15, 2020

View reviewed changes

src/core/ad.jl Outdated Show resolved Hide resolved

devmotion reviewed Mar 15, 2020

View reviewed changes

src/core/ad.jl Outdated Show resolved Hide resolved

mohamed82008 and others added 2 commits March 16, 2020 00:25

Update src/core/ad.jl

f261e1e

Co-Authored-By: David Widmann <[email protected]>

Update src/core/ad.jl

e749b7a

Co-Authored-By: David Widmann <[email protected]>

devmotion reviewed Mar 15, 2020

View reviewed changes

src/core/ad.jl Show resolved Hide resolved

devmotion reviewed Mar 15, 2020

View reviewed changes

src/core/ad.jl Outdated Show resolved Hide resolved

mohamed82008 and others added 3 commits March 16, 2020 00:27

Update src/core/ad.jl

32e7591

Co-Authored-By: David Widmann <[email protected]>

indentation

dcd2577

Merge branch 'mt/zygote_ad' of https://github.com/TuringLang/Turing.jl …

e0ccf32

…into mt/zygote_ad

devmotion approved these changes Mar 15, 2020

View reviewed changes

torfjelde approved these changes Mar 15, 2020

View reviewed changes

src/core/ad.jl Show resolved Hide resolved

src/variational/VariationalInference.jl Show resolved Hide resolved

cpfiffer approved these changes Mar 15, 2020

View reviewed changes

yebai approved these changes Mar 15, 2020

View reviewed changes

yebai merged commit ba7556d into master Mar 15, 2020

delete-merged-branch bot deleted the mt/zygote_ad branch March 15, 2020 21:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Zygote AD backend #783

Zygote AD backend #783

mohamed82008 commented May 12, 2019

xukai92 commented May 14, 2019

mohamed82008 commented May 14, 2019

cpfiffer commented May 14, 2019 •

edited

Loading

yebai commented May 29, 2019 •

edited

Loading

yebai commented Dec 17, 2019

mohamed82008 commented Dec 17, 2019

mohamed82008 commented Jan 14, 2020

mohamed82008 commented Jan 14, 2020

mohamed82008 commented Jan 14, 2020

mohamed82008 commented Jan 16, 2020

mohamed82008 commented Jan 18, 2020

mohamed82008 commented Jan 18, 2020 •

edited

Loading

mohamed82008 commented Mar 11, 2020

willtebbutt left a comment

cpfiffer left a comment

yebai commented Mar 15, 2020

Zygote AD backend #783

Zygote AD backend #783

Conversation

mohamed82008 commented May 12, 2019

xukai92 commented May 14, 2019

mohamed82008 commented May 14, 2019

cpfiffer commented May 14, 2019 • edited Loading

yebai commented May 29, 2019 • edited Loading

yebai commented Dec 17, 2019

mohamed82008 commented Dec 17, 2019

mohamed82008 commented Jan 14, 2020

mohamed82008 commented Jan 14, 2020

mohamed82008 commented Jan 14, 2020

mohamed82008 commented Jan 16, 2020

mohamed82008 commented Jan 18, 2020

mohamed82008 commented Jan 18, 2020 • edited Loading

mohamed82008 commented Mar 11, 2020

willtebbutt left a comment

Choose a reason for hiding this comment

cpfiffer left a comment

Choose a reason for hiding this comment

yebai commented Mar 15, 2020

cpfiffer commented May 14, 2019 •

edited

Loading

yebai commented May 29, 2019 •

edited

Loading

mohamed82008 commented Jan 18, 2020 •

edited

Loading