Smart Connector with access to SMB Fileshare

[FrankDK]

Anyone aware if the Smart Connector solution supports a way to access a SMB fileshare on the clients network? Use case is that client uses a file-share for flat-file exchange

[Ryan_Berry]

Apologizes for the delay in a response!  

SIC Would work for this scenario using Smart Integration Functions.  You can invoke these to pick-up remote data from file shares up to 500-600mb or so and retrieve them from a OneStream BR.  (Simple example below).  We cannot tunnel this traffic over SIC using the direct connect capabilities as SMB shares use port 445 when a UNC path is specified and there is not a means to direct that traffic over a SIC managed port.  Smart Integration Functions would be the path to use in this scenario -- would this work for your needs?

namespace OneStream.BusinessRule.SmartIntegrationFunction.TestFileRead
{
public class MainClass
{
public byte[] RunOperation(string year)
{
string fname = @"\\someshare\path\hw_" + year + ".csv";
byte[] buffer = System.IO.File.ReadAllBytes(fname);
return buffer;
}

[MZ]

Hi Berry,

What is the best solution to out bound files if we have existing UNC copy file setup currently in v73, but now we need to replace it with similar process in v82 through SIC? It is a regular network driver which we use "Shell(Net Use ...)" command to copy files currently. Thanks,

Michael Z.

[Ryan_Berry]

Hello @MZ ,

Good question; one thing worth noting is that while this path may functionally work, using SMB/CIFS over a WAN/VPN could have poor performance especially if the file sizes are small.  That aside, you have a solution in place that is working and looking for an alternative with SIC.  There's a few options I'd suggest:

1) If SFTP is at all an option, you can establish a SFTP endpoint on your network mounted to the share of interest and use the SIC Direct-connect capability to surface that to OneStream.  This will scale to large file sizes into the 5+GB range as one option.

2) You could leverage a remote Business rule that is invoked as a job or a synchronous function 

Both of these options have examples outlined here: Use Smart Integration Connector (onestream.com)

Would either of these 2 approaches work for your scenario?

[MZ]

Thanks Berry,

We have SFTP through SIC works now, however, we are not able to replace the UNC approach with SFTP currently because the server team is not available in August and we are going live on v8 on the 19th. I saw the sample codes, but those were all for inbounding that the SIF can return bytes. I will double check the SIF to invoke as Job option. I am also exploring the possibility of SCP now.

[Ryan_Berry]

No problem @MZ.  The example code does return bytes; I could provide an example to save those bytes into the OS fileshare when it's retrieved if that would help.  What are you doing with this data today when you retrieve it from the share?  I assume this is a stage/load connector type rule you use today?

[MZ]

We are looking for a solution to copy files from OS shared folders to the remote servers via SIC gateway. The data will be consumed by other applications. I know there are multiple options to handle this type of data outputs, however, we are limited to duplicate similar protocol which is to directly copy the file to the shared folder in the remote servers.

[Ryan_Berry]

Ahh -- so you are egressing data; not ingesting.  Do you know how large these files could potentially be?  

[MZ]

Correct! Average size around 10 MB per file, could be multiple files.

[Ryan_Berry · Accepted solution]

@MZ ,

I have a working example for you -- have to do some tricks to convert the file to base64 encoded string to pass it over but it works.

public object Main(SessionInfo si, BRGlobals globals, object api, ExtenderArgs args)

{

try

{

string fileName = BRApi.Utilities.GetFileShareFolder(si, FileShareFolderTypes.ApplicationOutgoing, null) + "\\test5mb.txt";

//string fileName = BRApi.Utilities.GetFileShareFolder(si, FileShareFolderTypes.ApplicationOutgoing, null) + "\\hw_25000.csv";

BRApi.ErrorLog.LogMessage(si, "Reading File: " + fileName);

byte[] fileBytes = System.IO.File.ReadAllBytes(fileName);

string base64String = Convert.ToBase64String(fileBytes);

if (fileBytes != null && !String.IsNullOrEmpty(base64String))

{

   object[] functionParams = new object[1] {base64String};

   RemoteRequestResultDto objResult = BRApi.Utilities.ExecRemoteGatewayBusinessRule(si, "TestFileRead", functionParams, "ryantestconnection2", "WriteFile");

   if (objResult.RemoteResultStatus == RemoteMessageResultType.Success)

   {

    BRApi.ErrorLog.LogMessage(si, "Successfully Submitted File - Result: " + objResult?.ObjectResultValue?.ToString());

   }

   else

   {

      if (objResult.RemoteException != null)

  {

  BRApi.ErrorLog.LogMessage(si, "Remote Exception: " + objResult.RemoteException.Message);

  }

  else

  {

  BRApi.ErrorLog.LogMessage(si, "Unknown Failure");

  }

   }

}

else

{

BRApi.ErrorLog.LogMessage(si, "Missing File Bytes");

}

return null;

}

catch (Exception ex)

{

throw ErrorHandler.LogWrite(si, new XFException(si, ex));

}

}

And the corresponding remote BR TestFileRead with the WriteFile method:

namespace OneStream.BusinessRule.SmartIntegrationFunction.TestFileRead

{

public class MainClass

{

public bool WriteFile(string inboundData)

{

byte[] buffer = Convert.FromBase64String(inboundData);

string fname = @"c:\temp\hwtemp.txt";

System.IO.File.WriteAllBytes(fname,buffer);

return true;

}

}

[MZ]

Thanks Berry! I got it to work here with a sample code from OneStream. The logics is very similar that I compress the file into bytes and then pass one file a time to SIF as a part of the argument object. I was also able to write to remote server directly on my SIF once the transferred files inflated. To avoid size concern, i created a loop on the extender to process one file a time. It works perfectly, thanks again!