dask: FileNotFoundError: [Errno 2] No such file or directory when writing to_csv

I’m getting the following error when trying to write using to_csv. It works fine using pandas.

import dask.dataframe as dd

Filename = '\FreshPlanet_SongPop_0317.txt'
Filepath = r'O:\Song Pop\01 Originals\2017'
OutputFilepath = r'O:\Song Pop\01 Originals\2017\FreshPlanet_SongPop_0317_FORMATTED.txt'
df = dd.read_csv(Filepath + Filename, sep='\t', encoding='ansi', keep_default_na=False)

dd.to_csv(df=df, filename=OutputFilepath, index=False, header=False, float_format='%.7f', sep='\t')

Error:

Traceback (most recent call last):
  File "C:/Users/PycharmProjects/Freshplanet/TestDask.py", line 12, in <module>
    dd.to_csv(df=df, filename=OutputFilepath, index=False, header=False, float_format='%.7f', sep='\t')
  File "C:\ProgramData\Miniconda3\lib\site-packages\dask\dataframe\io\csv.py", line 505, in to_csv
    delayed(values).compute(get=get)
  File "C:\ProgramData\Miniconda3\lib\site-packages\dask\base.py", line 99, in compute
    (result,) = compute(self, traverse=False, **kwargs)
  File "C:\ProgramData\Miniconda3\lib\site-packages\dask\base.py", line 206, in compute
    results = get(dsk, keys, **kwargs)
  File "C:\ProgramData\Miniconda3\lib\site-packages\dask\threaded.py", line 75, in get
    pack_exception=pack_exception, **kwargs)
  File "C:\ProgramData\Miniconda3\lib\site-packages\dask\local.py", line 521, in get_async
    raise_exception(exc, tb)
  File "C:\ProgramData\Miniconda3\lib\site-packages\dask\compatibility.py", line 60, in reraise
    raise exc
  File "C:\ProgramData\Miniconda3\lib\site-packages\dask\local.py", line 290, in execute_task
    result = _execute_task(task, data)
  File "C:\ProgramData\Miniconda3\lib\site-packages\dask\local.py", line 271, in _execute_task
    return func(*args2)
  File "C:\ProgramData\Miniconda3\lib\site-packages\dask\bytes\core.py", line 29, in write_block_to_file
    with lazy_file as f:
  File "C:\ProgramData\Miniconda3\lib\site-packages\dask\bytes\core.py", line 311, in __enter__
    f = f2 = self.myopen(self.path, mode=mode)
  File "C:\ProgramData\Miniconda3\lib\site-packages\dask\bytes\local.py", line 61, in open
    return open(path, mode=mode)
FileNotFoundError: [Errno 2] No such file or directory: 'O:\\Song Pop\\01 Originals\\2017\\FreshPlanet_SongPop_0317_FORMATTED.txt\\6.part'

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 25 (12 by maintainers)

Most upvoted comments

@kingblah you need to provide a globstring (including a *) or a list of filenames: http://dask.pydata.org/en/latest/dataframe-api.html#dask.dataframe.to_csv

TomAugspurger on Sep 5, 2017

If the user does not pass the full path to the file (on Unix type systems this means a path that starts with a slash), the python file path is interpreted relatively to the current working directory. The current working directory usually is the directory in which you started the program. In order to make this work, the directory containing the python executable must be in the PATH, a so-called environment variable that contains directories that are automatically used for searching executables when you enter a command.

In any case, if your Python script file and your data input file are not in the same directory, you always have to specify either a relative path between them or you have to use an absolute path for one of them.

linehammer on Jul 20, 2021