Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(materialization): dont always run get column on update if no query is there #29271

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

EDsCODE
Copy link
Member

@EDsCODE EDsCODE commented Feb 26, 2025

Problem

  • every view update is running get column

Changes

  • unneeded if query didn't change

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

Does this work well for both Cloud and self-hosted?

How did you test this code?

@EDsCODE EDsCODE requested a review from phixMe February 26, 2025 20:23
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

This PR optimizes the update process for saved queries in the data warehouse by avoiding unnecessary column recalculation when the query itself hasn't changed.

  • Modified DataWarehouseSavedQuerySerializer.update() in posthog/warehouse/api/saved_query.py to only call get_columns() and update status when the query field is present in validated data
  • Added test test_update_without_query_change_doesnt_call_get_columns in posthog/warehouse/api/test/test_saved_query.py to verify optimization when only updating the name
  • Added test test_update_with_query_change_calls_get_columns to ensure column recalculation still happens when the query changes
  • This optimization improves performance for view updates by eliminating redundant processing

2 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile

except RecursionError:
raise serializers.ValidationError("Model contains a cycle")
except Exception as err:
raise serializers.ValidationError(str(err))

Check warning

Code scanning / CodeQL

Information exposure through an exception Medium

Stack trace information
flows to this location and may be exposed to an external user.
Stack trace information
flows to this location and may be exposed to an external user.

Copilot Autofix AI 1 day ago

To fix the problem, we should avoid exposing the detailed error message to the end user. Instead, we can log the detailed error message on the server and return a generic error message to the user. This approach ensures that developers can still access the detailed error information for debugging purposes, while the end user only sees a generic message.

  • Modify the code on line 154 to log the detailed error message using the logger and raise a serializers.ValidationError with a generic error message.
  • Ensure that the logger is used to capture the detailed error information.
Suggested changeset 1
posthog/warehouse/api/saved_query.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/posthog/warehouse/api/saved_query.py b/posthog/warehouse/api/saved_query.py
--- a/posthog/warehouse/api/saved_query.py
+++ b/posthog/warehouse/api/saved_query.py
@@ -153,3 +153,4 @@
                 except Exception as err:
-                    raise serializers.ValidationError(str(err))
+                    logger.exception("An error occurred while updating the view: %s", err)
+                    raise serializers.ValidationError("An internal error has occurred.")
 
EOF
@@ -153,3 +153,4 @@
except Exception as err:
raise serializers.ValidationError(str(err))
logger.exception("An error occurred while updating the view: %s", err)
raise serializers.ValidationError("An internal error has occurred.")

Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
Copy link
Contributor

@phixMe phixMe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Best to avoid doing this when we can.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants