GPTCache: [Bug]: Wrong behavior when used from the Docker Image
Current Behavior
I am following the doc to be used with docker from here.
I noticed that no matter what Prompt I set, it always returns the 1st item inserted, for example
curl -X GET "http://localhost:8000?prompt=hello"
null
curl -X PUT -d "receive a hello message" "http://localhost:8000?prompt=hello+world"
curl -X GET "http://localhost:8000?prompt=hello"
"receive a hello message"
curl -X GET "http://localhost:8000?prompt=bye"
"receive a hello message"
I expect that the last bye
query will not return any value, just a null.
I created a sample gptcache.yml
with the following
model_src:
openai
...
config:
similarity_threshold: 0.2
By the way if I use threshold
(as detailed in the docs) I got an error. Then another error using the -f gptcache.yml
parameter
Traceback (most recent call last):
File "/usr/local/bin/gptcache_server", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.8/site-packages/gptcache_server/server.py", line 55, in main
init_similar_cache_from_config(config_dir=args.cache_config_file)
File "/usr/local/lib/python3.8/site-packages/gptcache/adapter/api.py", line 167, in init_similar_cache_from_config
with open(config_dir, "r", encoding="utf-8") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'gptcache.yml'
Modified the command to something like this as I am using Windows and with the volume got it running
docker run -p 8000:8000 -v %cd%:"/workspace":rw -it zilliz/gptcache:latest gptcache_server -s 0.0.0.0 -p 8000 -f gptcache.yml
I expected to get a better result lowering the similarity_threshold
value but exactly the same.
Am I missing something in how the component should be used?
thanks!
Expected Behavior
No response
Steps To Reproduce
No response
Environment
Windows 10 + Docker Image.
Anything else?
No response
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 15
ok no problem let’s close it. Will be monitoring it to test it again in the future when more options are included. thanks,
ok not sure what I am doing wrong. Noticed the python interface has several tweaks I can change, but using as a service behing a docker image there is not much I can do to change behavior. It is expected to evolve this interface to be used from other languages? It would be nice to have