Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

generate_statistics_from_csv without blocking code execution? #192

Open
santiagotoscanini opened this issue Aug 19, 2021 · 2 comments
Open

Comments

@santiagotoscanini
Copy link

Hi!

Is there a way to don't block the execution when calling the generate_statistics_from_csv?

Maybe return an Operation.

@santiagotoscanini santiagotoscanini changed the title generate_statistics_from_csv async? generate_statistics_from_csv without blocking code execution? Aug 19, 2021
@sanatmpa1 sanatmpa1 self-assigned this Aug 24, 2021
@sanatmpa1 sanatmpa1 assigned caveness and unassigned sanatmpa1 Aug 24, 2021
@zoyahav
Copy link
Member

zoyahav commented Aug 25, 2021

What do you mean by not block the execution? Are you looking for it to "run in the background" or for a way to get the execution deferred?
Note that generate_statistics_from_csv is a convinience method, an alternative is to write the beam pipeline manually if you'd like to tweak the pipeline options. Read the data with tfx_bsl.public.tfxio.CsvTFXIO, then call tfdv.GenerateStatistics followed by tfdv.WriteStatisticsToTFRecord.

@santiagotoscanini
Copy link
Author

Hi!

Like many other Google APIs support, I look for a way to get the execution deferred and fetch the results with an operation name. Is there anything similar to this already implemented?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants