opensearch-benchmark: [BUG] Unable to run opensearch-benchmark in test mode

Describe the bug

  • opensearch-benchmark fails without running the benchmark on a local OpenSearch server
  • Logs below

To Reproduce

  1. Install latest version from pypi (pip3 install opensearch-benchmark)
  2. Execute the test command
opensearch-benchmark execute_test --target-host=localhost:9200 --workload=nyc_taxis --pipeline=benchmark-only --test-mode --kill-running-processes

Expected behavior

  • Benchmarks to run in test mode

Logs

023-03-27 23:25:05,311 ActorAddr-(T|:49871)/PID:73989 osbenchmark.actor ERROR Error in test execution orchestrator
Traceback (most recent call last):

  File "/Users/kkotwani/.pyenv/versions/3.8.16/lib/python3.8/site-packages/osbenchmark/actor.py", line 92, in guard
    return f(self, msg, sender)

  File "/Users/kkotwani/.pyenv/versions/3.8.16/lib/python3.8/site-packages/osbenchmark/test_execution_orchestrator.py", line 108, in receiveMsg_Setup
    self.coordinator.setup(sources=msg.sources)

  File "/Users/kkotwani/.pyenv/versions/3.8.16/lib/python3.8/site-packages/osbenchmark/test_execution_orchestrator.py", line 195, in setup
    self.current_workload = workload.load_workload(self.cfg)

  File "/Users/kkotwani/.pyenv/versions/3.8.16/lib/python3.8/site-packages/osbenchmark/workload/loader.py", line 192, in load_workload
    repo = workload_repo(cfg)

  File "/Users/kkotwani/.pyenv/versions/3.8.16/lib/python3.8/site-packages/osbenchmark/workload/loader.py", line 290, in workload_repo
    return GitWorkloadRepository(cfg, fetch, update)

  File "/Users/kkotwani/.pyenv/versions/3.8.16/lib/python3.8/site-packages/osbenchmark/workload/loader.py", line 331, in __init__
    self.repo.update(distribution_version)

  File "/Users/kkotwani/.pyenv/versions/3.8.16/lib/python3.8/site-packages/osbenchmark/utils/repo.py", line 68, in update
    branch = versions.best_match(git.branches(self.repo_dir, remote=self.remote), distribution_version)

  File "/Users/kkotwani/.pyenv/versions/3.8.16/lib/python3.8/site-packages/osbenchmark/utils/git.py", line 44, in probe
    return f(src, *args, **kwargs)

  File "/Users/kkotwani/.pyenv/versions/3.8.16/lib/python3.8/site-packages/osbenchmark/utils/git.py", line 123, in branches
    return _cleanup_remote_branch_names(process.run_subprocess_with_output(

  File "/Users/kkotwani/.pyenv/versions/3.8.16/lib/python3.8/site-packages/osbenchmark/utils/git.py", line 137, in _cleanup_remote_branch_names
    return [(b[b.index("/") + 1:]).strip() for b in branch_names if not b.endswith("/HEAD")]

  File "/Users/kkotwani/.pyenv/versions/3.8.16/lib/python3.8/site-packages/osbenchmark/utils/git.py", line 137, in <listcomp>
    return [(b[b.index("/") + 1:]).strip() for b in branch_names if not b.endswith("/HEAD")]

ValueError: substring not found

More Context (please complete the following information):

  • Workload(Share link for custom workloads): nyc_taxis
  • Service(E.g OpenSearch): OpenSearch
  • Version (E.g. 1.0): 3.0.0/Latest main

Additional context

  • Running on an arm64 M1 Macbook

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 22 (10 by maintainers)

Commits related to this issue

Most upvoted comments

It looks like fixing this will need either:

In the meantime, here’s a rather ugly workaround:

  • First, run:
git -C ~/.benchmark/benchmarks/workloads/default update-ref -d refs/remotes/origin/main
  • Then, run the following command to open up the git config in your editor:
git -C ~/.benchmark/benchmarks/workloads/default config -e

and the change the following line:

fetch = +refs/heads/*:refs/remotes/origin/*

to something like:

fetch = +refs/heads/*:refs/real-remotes/origin/*

(or any string other than refs/remotes/). This should now allow OSB to execute normally