GPTCache: [Bug]: Wrong behavior when used from the Docker Image

Current Behavior

I am following the doc to be used with docker from here.

I noticed that no matter what Prompt I set, it always returns the 1st item inserted, for example

curl -X GET  "http://localhost:8000?prompt=hello"
null
curl -X PUT -d "receive a hello message" "http://localhost:8000?prompt=hello+world"
curl -X GET  "http://localhost:8000?prompt=hello"
"receive a hello message"
curl -X GET  "http://localhost:8000?prompt=bye"
"receive a hello message"

I expect that the last bye query will not return any value, just a null.

I created a sample gptcache.yml with the following

model_src:
    openai
...
config:
    similarity_threshold: 0.2

By the way if I use threshold (as detailed in the docs) I got an error. Then another error using the -f gptcache.yml parameter

Traceback (most recent call last):
  File "/usr/local/bin/gptcache_server", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/site-packages/gptcache_server/server.py", line 55, in main
    init_similar_cache_from_config(config_dir=args.cache_config_file)
  File "/usr/local/lib/python3.8/site-packages/gptcache/adapter/api.py", line 167, in init_similar_cache_from_config
    with open(config_dir, "r", encoding="utf-8") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'gptcache.yml'

Modified the command to something like this as I am using Windows and with the volume got it running

docker run -p 8000:8000 -v %cd%:"/workspace":rw -it zilliz/gptcache:latest gptcache_server -s 0.0.0.0 -p 8000 -f gptcache.yml

I expected to get a better result lowering the similarity_threshold value but exactly the same.

Am I missing something in how the component should be used?

thanks!

Expected Behavior

No response

Steps To Reproduce

No response

Environment

Windows 10 + Docker Image.

Anything else?

No response

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 15

Most upvoted comments

ok no problem let’s close it. Will be monitoring it to test it again in the future when more options are included. thanks,

ok not sure what I am doing wrong. Noticed the python interface has several tweaks I can change, but using as a service behing a docker image there is not much I can do to change behavior. It is expected to evolve this interface to be used from other languages? It would be nice to have

  • logging
  • support for different sessions or tenant
  • API to handle sessions
  • better tweaking of parameters, rather than the yaml file.