Performance issue to upload a file.

[jimmygilles]

When I want to upload a file, I can either use the method SftpClient.UploadFile or I can use

            using (Stream destinationStream = sftpClient.OpenWrite(destinationPath))
            {
                byte[] buffer = new byte[10 * 1024 * 1024];  // 10 MB buffer
                long totalBytesCopied = 0;
                int currentBlockSize = 0;

                Progress(totalBytesCopied);

                while ((currentBlockSize = fileStream.Read(buffer, 0, buffer.Length)) > 0)
                {
                    destinationStream.Write(buffer, 0, currentBlockSize);

                    totalBytesCopied += currentBlockSize;
                    Progress(totalBytesCopied);
                }
            }

With SftpClient.UploadFile, there is no problem.
But, if I use the previous code, the performances are very, very, very... very... bad.
And I am sure "fileStream.Read" is not the problem.
The method "Progress" is just a Console.WriteLine.

It seems the SftpFileStream is problematic when we want to upload a file.

Can you please fix this problem ?
Thank you very much.

[drieseng]

What version of SSH.NET are you using?
What is the buffer size of the SftpClient?
Why are you using a 10 MB buffer?

Note that it's perfectly normal that UploadFile is a little faster as it reads and writes in a tight loop.
Also, UploadFile reads from the input stream in chunks of (roughly) SftpClient.BufferSize, and immediately writes each chunk.

What results are you getting?

[jimmygilles]

Hello,

I am using the latest version on nuget : v2016.1.0-beta4

Concerning the buffer size in the SftpClient, I don't change de size ; so, it's the default size. I will check on monday.

I am using a 10 MB buffer because I need to copy files of 15 GB or more.
I read a very big block of data to reduce the number of "read" operations from the disk.

Today I had to copy a file.
With UploadFile, to copy 10 MB it took approximately 10 seconds.
With the other solution, it took more than 2 minutes.

[drieseng]

Both UploadFile and SftpFileStream should be a lot faster than that (depending on your network).
I'll also try to find time to do some comparisons.

[RaheemFrancois]

I am having the same issue

[RaheemFrancois]

A few weeks ago UploadFile worked quick (< 5 seconds) on transferring 4gb text file. Now it takes about 3-4 minutes to transfer. Any help?

[drieseng]

What version(s) of SSH.NET did you test, and which one(s) were fast?

[RaheemFrancois]

I have been using 2016.0.0 up until today after experiencing issues. I then updated to the latest 2016.1.0 and still experiencing same issue.

[RaheemFrancois]

While using 106.0.0, the transfer was smooth. 9/30/2017 was the last time I seen a smooth transfer. A few days ago, I tested this and notice it had gotten really slow.

[Sahuri]

Hi, how to detect file uploading success/fail?

[apretth]

Hi, I am also seeing the similar performance issue when using SFTPStream with SSH.NET-2016.1.0 version. Uploading a 1GB file using the SFTPStream with 50 MB chunks takes around 30 mins. This time is specifically for writing to already got SFTPStream.
Did anyone find a solution for it?

[anthonyjmccann]

Hi, I am seeing very poor performance with 200 files around 10 MB total taking 8+ minutes to download (upload is similar performance). I'm attaching the code used for download. Using latest version - 2016.1.0.
DownloadFiles.txt

[Disham]

Any update? I am facing same issue as described by @jimmygilles .

[lucastheisen]

I did a little looking into this as well... The UploadFile implementation is very different. I think it makes optimizations because it knows it is going to upload an entire file all at once rather than possibly seeking around between writes. In UploadFile all the write requests are sent with a callback for the responses processing the responses asynchronously. When using OpenWrite it will sends an a non-null wait handle, causing each write to synchronously wait.

In my experimentation this caused at least an order of magnitude difference in performance.

namespace SftpTest
{
    using System;
    using System.Buffers;
    using System.Collections.Generic;
    using System.Diagnostics;
    using System.IO;
    using Renci.SshNet;

    static class StreamExtensions
    {
        // We pick a value that is the largest multiple of 4096 that is still smaller than the large object heap threshold (85K).
        // The CopyTo/CopyToAsync buffer is short-lived and is likely to be collected at Gen0, and it offers a significant
        // improvement in Copy performance.
        private const int DefaultCopyBufferSize = 81920;

        public static void CopyTo(
            this Stream source,
            Stream destination,
            Action<ulong> progress = null)
        {
            ulong totalBytes = 0;
            var buffer = ArrayPool<byte>.Shared.Rent(DefaultCopyBufferSize);
            try
            {
                int read;
                while ((read = source.Read(buffer, 0, buffer.Length)) != 0)
                {
                    if (progress != null)
                    {
                        totalBytes += (ulong)read;
                        progress(totalBytes);
                    }

                    destination.Write(buffer, 0, read);
                }
            }
            finally
            {
                ArrayPool<byte>.Shared.Return(buffer);
            }
        }
    }

    class Program
    {
        private const string hostname = "test-server.example.com";
        private const string username = "me";

        private const int ProgressInterval = 1024 * 1024 * 10;

        private static SftpClient ConnectedClient()
        {
            var home = System.Environment.GetFolderPath(
                System.Environment.SpecialFolder.UserProfile);

            var authenticationMethods = new List<AuthenticationMethod>();
            authenticationMethods.Add(
                new PrivateKeyAuthenticationMethod(
                    username,
                    new PrivateKeyFile(Path.Join(home, ".ssh", "id_rsa"))));

            var client = new SftpClient(
                new ConnectionInfo(hostname, 22, username, authenticationMethods.ToArray()));
            client.Connect();

            return client;
        }

        static void ErrorExit(string format, params object[] args)
        {
            System.Console.Error.WriteLine(format, args);
            System.Environment.Exit(1);
        }

        static void UseCopyTo(string fromPath, string toPath)
        {
            try
            {
                using var client = ConnectedClient();
                using var input = File.OpenRead(fromPath);
                using var output = client.OpenWrite(toPath);

                var stopwatch = new Stopwatch();
                stopwatch.Start();
                ulong nextProgress = 0;
                input.CopyTo(
                    output,
                    (bytes) =>
                    {
                        if (bytes > nextProgress)
                        {
                            nextProgress += ProgressInterval;
                            Console.Error.WriteLine($"Wrote {bytes} bytes in {stopwatch.Elapsed}");
                        }
                    });
            }
            catch (Exception e)
            {
                ErrorExit("Failed: {0}\n{1}", e.Message, e.StackTrace.ToString());
            }
        }

        static void UseUploadFile(string fromPath, string toPath)
        {
            try
            {
                using var client = ConnectedClient();
                using var input = File.OpenRead(fromPath);

                var stopwatch = new Stopwatch();
                stopwatch.Start();
                ulong nextProgress = 0;
                client.UploadFile(
                    input,
                    toPath,
                    (bytes) =>
                    {
                        if (bytes > nextProgress)
                        {
                            nextProgress += ProgressInterval;
                            Console.Error.WriteLine($"Wrote {bytes} bytes in {stopwatch.Elapsed}");
                        }
                    });
            }
            catch (Exception e)
            {
                ErrorExit("Failed: {0}\n{1}", e.Message, e.StackTrace.ToString());
            }
        }

        static void Main(string[] args)
        {
            var fromPath = args[0];
            var toName = args[1];
            if (string.IsNullOrEmpty(fromPath) || string.IsNullOrEmpty(toName))
            {
                ErrorExit("requires `dotnet run --project ./SftpTest.csproj -- <fromPath> <to>`");
            }

            var toPath = $"/tmp/{toName}";

            System.Console.Out.WriteLine($"CopyTo writing {fromPath} to {toPath} on {hostname}");
            UseCopyTo(fromPath, toPath);
            System.Console.Out.WriteLine($"UploadFile writing {fromPath} to {toPath} on {hostname}");
            UseUploadFile(fromPath, toPath);
        }
    }
}

[lucastheisen]

Initial attempts at using OpenRead seem to indicate the same performance problems, but switching to DownloadFile does not alleviate them...