Downloading Files from FTP and Doing Something With Them In D365FO

Although D365FO is now thoroughly modern and web based, with all the good things that entails, we still have to interface with some less modern systems. Sometimes those systems are literally older than we are. There are some middleware solutions that can help bridge the gap but we have quite a few tools in D365FO that can help us. Let's talk about FTP for a bit. The tried and true battle horse of Odin himself, this is never going to go away, ever. So, let's see what we can do from inside D365FO with no C# in play; just some standard tools available from x++. Consider the very simple example:

using System.Net;

class AAX_FTPDownloadSimple
{

    public static void main(Args args)
    {
        System.Net.WebClient webClient = new System.Net.WebClient();
        webClient.Credentials = new NetworkCredential("userName", "password");
        webClient.DownloadFile("ftp://nadummy.sharefileftp.com/test/test.txt", @"C:\temp\test.txt");
        info("complete");
    }

}

3 line of code to download a file and place it in an arbitrary location. It's not particularly helpful as we have to know the source of the file and we have lots of hard coding. However, we now have a file that we could use TextIo on, if we wanted. I'm positive this is a dev box only example as you wouldn't have access in that way to the underlying server disk on anything that isn't a one box. Let's look at something a little more feature rich.

using System.IO;
using WebClient = System.Net.WebClient;
using NetworkCredential = System.Net.NetworkCredential;

class AAX_FileDownloadIntermediate
{
    public static void main(Args args)
    {
        TextIo textIo;

        str inputfilepath = System.IO.Path::GetTempPath() + System.IO.Path::GetRandomFileName();
        str ftphost = "nadummy.sharefileftp.com";
        str ftpfilepath = "/test/test.txt";

        str ftpfullpath = "ftp://" + ftphost + ftpfilepath;

        using (WebClient request = new WebClient())
        {
            request.Credentials = new NetworkCredential("userName", "password");

            System.Byte[] fileData = request.DownloadData(ftpfullpath);

            using (System.IO.FileStream file = System.IO.File::Create(inputfilepath))
            {
                file.Write(fileData, 0, fileData.Length);
                file.Close();
            }
            info("download Complete");
        }

        textIo = new TextIo(inputfilepath, "r");
        
        info(con2Str(textIo.read()));

        TextIo.finalize();

        System.IO.File::Delete(inputfilepath);

        info("Complete");
    }

}

Similar to that last one but we're handling data in a file, rather than a file plus a few other tweaks. First, we're using webClient.DownloadData rather than webClient.DownloadFile then writing that to a FileStream. Next, we're taking that FileStream and writing it to temporary (.NET) disk storage. Additionally, we're opening the file for the FileStream using TextIo, which we've had for years, so we've working with something familiar. Once done, we are deleting the temp file. I'm not sure if this would work in a non-onebox environment but .NET may be aware of its deployment type and sort itself out. I can't really say so definitely test in a Tier 2 environment. Finally, we're using using in a way most x++ people may be unfamiliar with. Next, let's keep on adding stuff:

using FTPWebRequest = System.Net.FtpWebRequest;
using WebRequest = System.Net.WebRequest;
using NetworkCredential = System.Net.NetworkCredential;
using WebRequestMethods = System.Net.WebRequestMethods;
using Stream = System.IO.Stream;

class AAX_FileDownloadAdvanced
{
    public static void main(Args args)
    {
        TextStreamIo textStreamIo;

        System.Byte[] buffer = new System.Byte[10240]();
 
        System.IO.Stream fileStream = new System.IO.MemoryStream();

        str targetFile = System.IO.Path::GetRandomFileName();
        FtpWebRequest request = WebRequest::Create("ftp://nadummy.sharefileftp.com/test/test.txt");
        request.Credentials = new NetworkCredential("userName", "password");    

        using (Stream ftpStream = request.GetResponse().GetResponseStream())
        {
            int read = 0;
            
            while(true)
            {
                read = ftpStream.Read(buffer, 0, buffer.Length);
                if(read > 0)
                {
                    fileStream.Write(buffer, 0, read);
                    info(strFmt("Downloaded %1 bytes", fileStream.Position));
                }
                else
                {
                    break;
                }
            }
        }

        textStreamIo = textStreamIo::constructForRead(fileStream);
        
        info(con2Str(textStreamIo.read()));
      
        textStreamIo.finalize();
        fileStream.Dispose();

        info("Complete");
    }

}

Here we have quite a few differences. First, we're using a WebRequest rather than a WebClient. WebRequests support encryption so stuff like SFTP or FTPS becomes an option. Next we're grabbing chunks from our source file so we can report on it's progress. If we had a large file, this would be helpful to log what is happening. Next, we're taking the data we placed in a Stream and are interacting with it without placing it in any type of local storage, temporary or otherwise. You could use File::SendFileToTempStore() and File::UseFileFromURL() to store/retrieve in/from blob storage if you like. Finally, we are passing the stream to a stream implementation of TextIo as TextStreamIo.

Feel free to download the code from Github.