Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: Use UDF everywhere #29216

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

perf: Use UDF everywhere #29216

wants to merge 7 commits into from

Conversation

timgl
Copy link
Collaborator

@timgl timgl commented Feb 25, 2025

Problem

clicking on a funnel step to get persons is slow, mostly because we take a long time to parse hogql

Changes

Use UDF instead

image

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

Does this work well for both Cloud and self-hosted?

How did you test this code?

@timgl timgl requested a review from aspicer February 25, 2025 21:51
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

This PR improves performance by consistently using User Defined Functions (UDFs) across all funnel queries, reducing HogQL parsing time when retrieving persons from funnel steps.

  • Removed the conditional use_udf() function from posthog/hogql_queries/insights/funnels/utils.py and its related imports
  • Modified posthog/hogql_queries/insights/funnels/funnel_correlation_query_runner.py to hardcode True for UDF usage in get_funnel_actor_class()
  • Removed the _use_udf cached property from posthog/hogql_queries/insights/funnels/funnels_query_runner.py and set use_udf=True in funnel class instantiations
  • Set isUdf=True in the response object to ensure consistent UDF usage throughout the funnel query process

3 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile

Copy link
Contributor

@aspicer aspicer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@timgl
Copy link
Collaborator Author

timgl commented Feb 25, 2025

@aspicer was there a reason you didn't do this before?

@aspicer
Copy link
Contributor

aspicer commented Feb 25, 2025

Does this do anything other than removing the feature flags? The feature flags are fully enabled, so I'm not sure this functionally changes anything.

How is this supposed to address the hogql parsing performance? Maybe I'm missing something there?

@timgl
Copy link
Collaborator Author

timgl commented Feb 25, 2025

@aspicer It was specifically for clicking on a step in a funnel. With this change we now use UDF, which means we don't have to spend ~5 seconds parsing Hogql

@aspicer
Copy link
Contributor

aspicer commented Feb 25, 2025

@aspicer It was specifically for clicking on a step in a funnel. With this change we now use UDF, which means we don't have to spend ~5 seconds parsing Hogql

Do you mean clicking on the funnel and bringing up the actors modal like below? If so, that already uses UDFs.
image

Or something else that I'm missing?

UDFs don't support unordered funnels yet, and I'm converting time to convert funnels now.

@timgl
Copy link
Collaborator Author

timgl commented Feb 25, 2025

@aspicer Ah, this is the context, and they use unordered funnels which I suppose is why it wasn't using UDF. Is unordered support on your roadmap for this?

EDIT: looks like Thomas came to the same conclusion a few hours ago: #28878 (comment). Let me know if you still want to merge this just for clean up or not.

@aspicer
Copy link
Contributor

aspicer commented Feb 26, 2025

@timgl Yes, it's on the roadmap for this quarter. The goal is to create the ability for more funnel flexibility, so that people could, for example, do

Step 1
Step 2,3,4 (any order, perhaps optional)
Step 5

and track that throughout the funnel.

There's a chance that fully unordered funnels would come first, but I'll have to see if it makes sense when I get into the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants