autotrain-advanced: pip install autotrain-advanced fails due to sentencepiece setup failure

unable to resolve

Building wheels for collected packages: sentencepiece Building wheel for sentencepiece (setup.py) … error error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [55 lines of output] /home/stav/miniconda3/envs/test/lib/python3.11/site-packages/setuptools/dist.py:755: SetuptoolsDeprecationWarning: Invalid dash-separated options !!

          ********************************************************************************
          Usage of dash-separated 'description-file' will not be supported in future
          versions. Please use the underscore name 'description_file' instead.

          By 2023-Sep-26, you need to update your project and remove deprecated calls
          or your builds will no longer be supported.

          See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
          ********************************************************************************

  !!
    opt = self.warn_dash_deprecation(opt, section)
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-cpython-311
  creating build/lib.linux-x86_64-cpython-311/sentencepiece
  copying src/sentencepiece/__init__.py -> build/lib.linux-x86_64-cpython-311/sentencepiece
  copying src/sentencepiece/sentencepiece_model_pb2.py -> build/lib.linux-x86_64-cpython-311/sentencepiece
  copying src/sentencepiece/sentencepiece_pb2.py -> build/lib.linux-x86_64-cpython-311/sentencepiece
  running build_ext
  Package sentencepiece was not found in the pkg-config search path.
  Perhaps you should add the directory containing `sentencepiece.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'sentencepiece' found
  Cloning into 'sentencepiece'...
  Note: switching to '0e6dfbf86e2fa6d86a3d9a8a08a628da71c073e0'.

  You are in 'detached HEAD' state. You can look around, make experimental
  changes and commit them, and you can discard any commits you make in this
  state without impacting any branches by switching back to a branch.

  If you want to create a new branch to retain commits you create, you may
  do so (now or later) by using -c with the switch command. Example:

    git switch -c <new-branch-name>

  Or undo this operation with:

    git switch -

  Turn off this advice by setting config variable advice.detachedHead to false

  ./build_bundled.sh: 15: cmake: Permission denied
  make: *** No targets specified and no makefile found.  Stop.
  make: *** No rule to make target 'install'.  Stop.
  Package sentencepiece was not found in the pkg-config search path.
  Perhaps you should add the directory containing `sentencepiece.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'sentencepiece' found
  Failed to find sentencepiece pkg-config
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for sentencepiece Running setup.py clean for sentencepiece Failed to build sentencepiece ERROR: Could not build wheels for sentencepiece, which is required to install pyproject.toml-based projects

About this issue

Original URL
State: closed
Created a year ago
Reactions: 1
Comments: 25 (5 by maintainers)

Most upvoted comments

m2 isnt supported yet

abhishekkrthakur on Sep 29, 2023

it works on 3.10 python too now. tested on m2 mac.

abhishekkrthakur on Aug 5, 2023

hello I am using Mac M1 and run pip install autotrain-advanced. it shows error below

Collecting pyyaml==5.4.1 (from autotrain-advanced) Using cached PyYAML-5.4.1.tar.gz (175 kB) Installing build dependencies … done Getting requirements to build wheel … error error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> [62 lines of output] /private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in setup.cfg !!

          ********************************************************************************
          The license_file parameter is deprecated, use license_files instead.
  
          By 2023-Oct-30, you need to update your project and remove deprecated calls
          or your builds will no longer be supported.
  
          See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
          ********************************************************************************
  
  !!
    parsed = self.parsers.get(option_name, lambda x: x)(value)
  running egg_info
  writing lib3/PyYAML.egg-info/PKG-INFO
  writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
  writing top-level names to lib3/PyYAML.egg-info/top_level.txt
  Traceback (most recent call last):
    File "/Users/development/opt/anaconda3/envs/llama2/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/Users/development/opt/anaconda3/envs/llama2/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/Users/development/opt/anaconda3/envs/llama2/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
      self.run_setup()
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in run_setup
      exec(code, locals())
    File "<string>", line 271, in <module>
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 107, in setup
      return distutils.core.setup(**attrs)
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
      super().run_command(command)
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 314, in run
      self.find_sources()
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 322, in find_sources
      mm.run()
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 551, in run
      self.add_defaults()
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 589, in add_defaults
      sdist.add_defaults(self)
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/command/sdist.py", line 104, in add_defaults
      super().add_defaults()
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
      self._add_defaults_ext()
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
      self.filelist.extend(build_ext.get_source_files())
    File "<string>", line 201, in get_source_files
    File "/private/var/folders/0m/fj0bcx4n16dd2snclpw414080000gn/T/pip-build-env-xn8c1tlr/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
      raise AttributeError(attr)
  AttributeError: cython_sources
  [end of output]

any clues?

alexng88 on Jul 21, 2023

m2 for llm and tabular training is supported now.

abhishekkrthakur on Oct 6, 2023

I think that only mac is not working, I’ve tried changing and creating multiple envs just won’t work still. If anyone finds anything keep me posted pls

I am finally able to install it on my machine, I’ve installed it on a conda environment with Python 3.11.

nerdlab53 on Aug 5, 2023

I had the same issue on w11. But switched to a 3.9 conda env and i was okay

Do you mind to state your environment and software version? I suspect mac is not working

alexng88 on Jul 27, 2023

i was able to install by creating conda env on Ubuntu 22.04 wsl

conda create -n <name> python=3.9.0

then run:

conda install cudatoolkit -y conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia -y pip install -U bitsandbytes -y pip install -U git+https://github.com/huggingface/transformers.git -y pip install -U git+https://github.com/huggingface/peft.git -y pip install -U git+https://github.com/huggingface/accelerate.git -y pip install datasets -y pip install scipy -y

sudo apt install cmake -y

sudo apt-get install postgresql python3-dev -y sudo apt-get install --reinstall libpq-dev -y pip install psycopg2 -y pip install psycopg2-binary -y pip install sentencepiece -y conda install -c conda-forge hdbscan -y

sudo apt-get install --reinstall libpq-dev -y

pip install autotrain-advanced

stavsap on Jul 13, 2023