ssh2-sftp-client: Why can't this create a new file on the remote server? "no such file or directory"

The issue is present in the example code for me

let Client = require('ssh2-sftp-client');
let sftp = new Client();

sftp.connect({
    host: '...',
    port: '...',
    username: '...',
    password: '...'
}).then(() => {
    return sftp.fastPut('./test.txt', '/tf/maps/test.txt') //this doesn't work unless a '/tf/maps/test.txt' already exists which it will overwrite
    //return sftp.put(require(`fs`).createReadStream('./test.txt'), '/tf/maps/test.txt') //Same issue as above
    //return sftp.list('/tf/maps/') //This works
}).then(data => {
    console.log(data, 'the data info');
}).catch(err => {
    console.log(err, 'catch error');
});

I get the error Error: fastPut->exists->_stat: open /var/lib/pufferd/servers/18a5d777-a56d-4d07-a6f4-7d0e3f619081/tf/maps/test.txt: no such file or directory /tf/maps/test.txt for fastPut and put, unless I already have a test.txt on the remote server. Does this package only overwrite existing files? I have no idea how to actually create a remote file In the readme it says remotePath: string. Path to the remote file to be created on the server. which suggests that it should create the file yet refuses to write unless it’s already there.

I have the server open in Filezilla and am able to create files there so I don’t think it’s a permission issue. I have also tried explicitly giving the flag w , and tried different modes as if that would do anything

Node: 14.7.0 OS: tried on Windows 10, Debian 4.19.118, and Ubuntu 18.04.4 ssh2-sftp-client: 5.2.1 Server is using https://github.com/PufferPanel/pufferd

Full errors:

Error: put->exists->_stat: open /var/lib/pufferd/servers/18a5d777-a56d-4d07-a6f4-7d0e3f619081/tf/maps/test.txt: no such file or directory /tf/maps/test.txt
    at Object.formatError (C:\Users\x\source\repos\sftptest\sftptest\node_modules\ssh2-sftp-client\src\utils.js:62:18)
    at SftpClient.put (C:\Users\x\source\repos\sftptest\sftptest\node_modules\ssh2-sftp-client\src\index.js:740:19)
    at processTicksAndRejections (internal/process/task_queues.js:93:5) {
  code: 'ERR_GENERIC_CLIENT',
  custom: true
} catch error
Error: fastPut->exists->_stat: open /var/lib/pufferd/servers/18a5d777-a56d-4d07-a6f4-7d0e3f619081/tf/maps/test.txt: no such file or directory /tf/maps/test.txt
    at Object.formatError (C:\Users\x\source\repos\sftptest\sftptest\node_modules\ssh2-sftp-client\src\utils.js:62:18)
    at SftpClient.fastPut (C:\Users\x\source\repos\sftptest\sftptest\node_modules\ssh2-sftp-client\src\index.js:652:19)
    at processTicksAndRejections (internal/process/task_queues.js:93:5) {
  code: 'ERR_GENERIC_CLIENT',
  custom: true
} catch error

Debug of fastPut https://pastebin.com/8AfrzSwE

When I already made a blank test.txt

test.txt was successfully uploaded to /tf/maps/test.txt! the data info

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 18 (15 by maintainers)

Commits related to this issue

Most upvoted comments

While the FTP protocol doe snot specify any wildcard characters for filenames, many FTP (and SFTP) servers do support wildcards. Might be worthwhile using openSSH’s sftp client to connect to your remote sftp server and experiment to see what behaviour it has. Important to use a very basic CLI sftp client for this as more ‘advanced’ clients, like lftp or GUI clients may manipulate arguments locally before sending to the remote server.

The ssh2-sftp-client does not do any interpretation or escaping of file paths. It just passes whatever path you supply through to the ssh2 library, which in turn will pass through that path to the remote SFTP server. The remote SFTP server will interpret that path. As sftp is often implemented on top of the ssh protocol, support for whatever the remote parent shell supports is common. This is especially the case when your remote sftp server is not a dedicated sftp server, but rather just an ssh server which allows the sftp protocol ‘on top’. Under that config, logging into your remote system using ssh can often clarify what is going on (you can ‘play’ around in the shell, see what the default ‘login’ directory is etc). With a dedicated sftp server, you are connecting to something much more in-line with traditional FTP i.e. in a ‘chroot’ environment separate from the ‘base’ system and not inside a user’s home directory. These dedicated sftp server setups usually have a lot more restrictions. For example, only being able to list/get files you have uploaded, no ssh access, etc.

Fix released in v5.2.2

Just looking more closely at the error message and the server code, can you verify the server supports the stat() method e.g.

sftp.stat(/path/to/remote/file’);

I suspect the command is not supportedor it is returning data in a non-comnpliant manner.

run the command with argument for a file which exists, a file which does not exist and a directory which exists. Send me the results.

Are you able to ssh into the remote server?

If so, you can also just use that as the remote target.

Although you don’t have control over the sftp on the remote server, if you cannot get it to work with that server, it might be worth pointing out to whomever sets up the remote server that setting up a dedicated chroot sftp server using openssh sftp module is trivial and as openssh is a standard package on pretty much all linux servers, would result in less maintenance over using the go server (not sure why people think you need to install specialised sftp server software on Linux when it already has it by default. Really only need specialised sftp servers when you have extremely high throughput)

I’ll see if I can install the server on my ubuntu or arch boxes and try to test with it.

OK, I don’t know anything about that server, but I suspect your issue is something specific to that server. I will try to install it and test against it, but if there is an issue due to low level sftp interaction, there is little I can do as ssh2-sftp-client relies on the ssh2/ssh2-streams libraries, which are maintained upstream.

Looking at the limited documentation and code for that server, it does not look like it is a fully compliant sftp server. I would definitely not attempt to use fastget/fastput as these commands rely on the server’s ability to handle concurrent download/upload processes and I doubt it is supported. You should only use get/put.

Are you also running the server on Windows?

My recommendation would be to use a more maintained, tested and robust server, such as openssh.

The other thing you could try is use just the low level ssh2/ssh2-streams libraries without ssh2-sftp-client. This will determine if the issue is with ssh2-sftp-clie, testednt or the upstream dependencies. There are example scripts for put and fastput in the tools directory of the ssh2-sftp-client repository.

If it turns out the issue also exists with ssh2/ssh2-streams, then it will need to be addressed upstream. However, as the package works reliably with many other servers, I suspect the upstream maintainer will argue the problem is with the puffered server (another reason to try an alternative server like openssh). As the server does not appear to be a fully compliant sftp server, I doubt the maintainer will be willing to investigate.

There is a known bug with node versions greater than 14.0 affecting both fastput/fastget and put(). This is a bug in the underlying ssh2/ssh2-streams libraries used by ssh2-sftp-client. A issue has been logged with the upstream maintainer. So, first thing to do is downgrade your node to at least 14.0 (or use LTS version).

Once you have done that, if your still seeing the error, let me know and I’ll look at it more closely. If issue does persist, lease also provide details of platform and remote sftp server so that I can try and reproduce a similar environment.