Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Max/Min Expressions in Utils.scala #23

Closed
oshritf opened this issue Mar 8, 2018 · 2 comments · Fixed by #24
Closed

Max/Min Expressions in Utils.scala #23

oshritf opened this issue Mar 8, 2018 · 2 comments · Fixed by #24

Comments

@oshritf
Copy link
Contributor

oshritf commented Mar 8, 2018

Groupby with Max/Min throws null exception when column is other than IntegerType

More info:
scala/edu/berkeley/cs/rise/opaque/Utils.scala:

flatbuffersCreateField supports only creation of Null IntegerType (sets int value to 0)
Max and Min methods create Null Literal which currently not fully supported for all DataTypes and results in an exception.

Setting init value to zero (default for IntegerType currently) might not return valid results when data contains also negative values.

scala.MatchError: (null,LongType) (of class scala.Tuple2)
at edu.berkeley.cs.rise.opaque.Utils$.flatbuffersCreateField(Utils.scala:256)
origins - creation of Literal.create(null, child.dataType) in createInitialValuesVector

@ankurdave
Copy link
Collaborator

ankurdave commented Mar 8, 2018

Thanks, we need to support serializing nulls of other types than IntegerType.

Setting init value to zero (default for IntegerType currently) might not return valid results when data contains also negative values.

I don't think this should occur for Max and Min, because the definitions of those aggregate expressions include null checks (e.g. for Max). The zero is only there as a placeholder.

Edit: Oops, never mind. The null check there is not sufficient since it only checks the nullity of the current max, not the new value. We need to fix that as well.

oshritf added a commit to oshritf/opaque that referenced this issue Mar 8, 2018
oshritf added a commit to oshritf/opaque that referenced this issue Mar 8, 2018
Enables Min, Max and Count on column of those types

fixes mc2-project#23
ankurdave pushed a commit that referenced this issue Mar 8, 2018
…24)

Enables Min, Max and Count on column of those types

fixes #23
@ankurdave
Copy link
Collaborator

Moved the second part of this issue to #25.

octaviansima added a commit to octaviansima/opaque that referenced this issue May 3, 2021
This PR introduces an alternative way to lint Scala and C/C++ files before submitting a PR: by running ./format.sh. This is for contributors who may not want to use the pre-commit hook.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants