How to transfer upload download huge files to and from wcf service using streaming
In this article we will see how to transfer a huge file to and from WCF services efficiently. WCF supports streaming of data. We will use streaming to transfer huge file efficiently to and from a WCF service. This article is all about transferring file using streaming transfer mode in WCF and it does not enforce you to use streaming always, because of its limitations.
To transfer huge files or data, streaming is the effective and efficient method. By default WCF uses buffered transfer mode. Here in this article, to get an efficient system which does not eat up resources, we will use Streaming transfer mode of WCF. But streaming transfer mode of WCF has some limitations. We will be covering those limitations in this article. In this article we will see how to implement the operation contract which uses the built in data type which is stream and then we will see how to use data contract and message contract to transfer file and if not possible then we will analyze why it is not possible to transfer file using any of this methods.
Problem it addresses
When we use the default WCF transfer mode, which is Buffered transfer mode, WCF will load or buffer the entire file content into memory and then will transfer. Here if the file or data is huge, then it is not possible to load the entire data to the memory. In such cases it is impossible to transfer such huge data. To handle such situation, we can use the streaming transfer mode of WCF. In fact, you should consider streaming when you have a reasonably big file say 2MB file to transfer. This is because if you have many clients which will access the same service at the same time, then the memory requirement will be the product of file size and the number of client accessing the service simultaneously. We can’t select streaming only based on this fact. Streaming has some limitations.
Limitations in Streaming Transfer mode of WCF
Using the streamed transfer mode causes the run time to enforce additional restrictions. Operations that occur across a streamed transport can have a contract with at most one input or output parameter. That parameter corresponds to the entire body of the message and must be a Message, a derived type of Stream, or an IXmlSerializable implementation. Having a return value for an operation is equivalent to having an output parameter. Some WCF features, such as reliable messaging, transactions, and SOAP message-level security, rely on buffering messages for transmissions. Using these features may reduce or eliminate the performance benefits gained by using streaming. To secure a streamed transport, use transport-level security only or use transport-level security plus authentication-only message security.
SOAP headers are always buffered, even when the transfer mode is set to streamed. The headers for a message must not exceed the size of the MaxBufferSize transport quota. So when you take the decision to go for streaming, you should consider these facts.
Table of contents
- Creating operation contract which returns Stream.
- Creating operation which returns DataContract to stream file.
- Creating operation which returns MessageContract to stream file.
There are certain configurations that we have to set before using streaming. The first thing that we need to do is to change the default transfer mode of WCF. We have to configure the default transfer mode of WCF to Streaming. To do this set the transferMode=”Streamed”.You can set this property only if you are using any of these binding.
After this we have to decide on the maximum file size which we allow in our application to be transferred. It is good to limit the size of the file which is transferred. Once you have decided on the size, you have to set maxReceivedMessageSize property of the binding to the size which you have decided. This property sets the Max size of the received message which is processed by the binding. This property value is in bites. You can set it up to long.MaxValue which is 9,223,372,036,854,775,807. Here in our example we are setting it to 4294967294 which is around 4 GB.
Now we have to set the maxBufferSize of the binding. This property sets the Max number of bytes used to buffer the incoming message in the memory. Here this is the max size of the message which is processed at a time. SOAP headers are always buffered, even when the transfer mode is set to streamed. The headers for a message must not exceed the size of the MaxBufferSize transport quota. Here in our example we are sending 64k chunks of data at a time so we are setting it to 65536 which is 64K. Here this is the default setting of WCF so we don’t have to change it explicitly. If you decide to change this value, then you have to change this value.
As in our example we will be using upload and download features, we have to set these properties both at service and client side.
Using the code
We will be using MessageContract in our example. Lets create our message contract before moving further. Our message contract class, which we will be using in our code will look like as shown below
1. Creating operation contract which returns Stream Using Stream in contract to upload file.
Your service side code will be like as shown below.
And at the client side your code will be as shown below. Remember this is a pseudo code. Please optimize it while using it in your code.
2. Creating operation which returns DataContract to stream file using DataContract in contract to stream file.
If you try to use data contract in your contract where you use Stream as one of the data members to transfer file, then it will throw exception. This is because of the limitation with streaming in WCF. And the limitation which we face here is that the parameter that we use in contract should be of type IXmlSerializable. Here as we are using data contract, it is not IXmlSerializable. i.e. WCF expects XmlSerializer to serialize and de-serialize messages. Data contract uses DataContractSerializer by default to serialize and de-serialize messages. So we can’t use data contract to pass stream.
3. Creating operation which returns MessageContract to stream file using MessageContract to stream file.
When you are using MessageContract to stream file, your service side contract code is as shown below
And on the client side…
4. Downoad file using MesageContract.
For download contract, your service side code looks like as shown below.
And at the client side code should look like as shown below.
5. Download file using simple Stream.
Your service side code when you are using simple stream to transfer file is as shown below.
Point of interest
Possible exception when you try to stream data.
- The maximum message size quota for incoming messages (65536) has been exceeded. To increase the quota, use the MaxReceivedMessageSize property on the appropriate binding element.
- This can happen if the max size of the message or file exceed the size which we set for the property MaxReceivedMessageSize in the binding. Set it to more than the size of the file which you are streaming.