Redis Semantic Cache is not working

[rakesh-srivastava-negd]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

semantic_cache = RedisSemanticCache(
redis_url=REDIS_URL,
embeddings=embeddings,
distance_threshold=0.3,
ttl=REDIS_TTL,
prefix=hashlib.sha256(question.encode('utf-8')).hexdigest(),
name=hashlib.sha256(question.encode('utf-8')).hexdigest(),
)

Error Message and Stack Trace (if applicable)

No response

Description

I'm working on a project where I am using LangChain's RedisSemanticCache to speed up response times for repetitive queries by caching results. I am using OpenAI embeddings to create vector representations of questions and set up the cache using Redis. The problem I'm encountering is that semantically similar questions, like "What is the capital of France?" and "Can you tell me the capital city of France?", are being treated as different cache entries because they generate different embeddings.

Here’s how I’m currently setting up the cache:

semantic_cache = RedisSemanticCache(
    redis_url=REDIS_URL,
    embeddings=embeddings,
    distance_threshold=0.3,
    ttl=REDIS_TTL,
    prefix=hashlib.sha256(question.encode('utf-8')).hexdigest(),
    name=hashlib.sha256(question.encode('utf-8')).hexdigest(),
)

The issue arises because when I use hashlib.sha256() on the question string, each query generates a different cache key, even though the meaning is the same. This leads to cache misses for semantically similar queries and poor performance.

What I want to achieve:
I want to ensure that semantically similar queries are stored and retrieved using the same cache key, which would allow the cache to be more effective. My goal is to avoid cache overwrites and speed up response times for similar questions without needing to re-run embeddings or queries.

However, when I try to use a static cache key (e.g., always using the same cache key), I face another problem:
If I keep the cache key static, it always returns the same response from the first question, even if I ask a completely unrelated question. This prevents the system from generating answers for new queries.

Questions:

How can I ensure that semantically similar queries (with different wording) generate the same cache key?
Should I preprocess the queries before hashing them to generate a standardized cache key?
Is there a way to append different answers to the same key in Redis without overwriting previous ones?
What is the best way to handle different variations of the same query without missing cache hits?
How can I avoid the issue of always getting the first answer when using a static key for unrelated queries?

System Info

langchain==0.2.16
langchain-anthropic==0.1.23
langchain-aws==0.1.18
langchain-community==0.2.17
langchain-core==0.2.43
langchain-openai==0.1.25
langchain-redis==0.0.1
langchain-text-splitters==0.2.0
langgraph==0.2.3
langgraph-checkpoint==1.0.12
langsmith==0.1.147
pycparser==2.22
pycryptodome==3.20.0
pydantic==2.10.4
pydantic-settings==2.7.0
pydantic_core==2.27.2
Pygments==2.18.0
pymongo==4.10.1
pyOpenSSL==24.3.0
pypandoc==1.14
pyparsing==3.2.0
typing-inspect==0.9.0
typing_extensions==4.12.2
tzdata==2024.2
unstructured==0.7.12
unstructured-client==0.18.0
urllib3==2.3.0

[keenborder786]

I think there’s some confusion in your understanding. Behind the scenes, RedisSemanticCache utilizes RedisVectorstore. When a query is added to RedisSemanticCache, it is embedded and stored in RedisVectorstore. During lookup, the user query is embedded again, and the most similar queries (those with a similarity score above the given threshold) are retrieved from RedisVectorstore.