There was an error while trying to serialize parameter fileStream. The InnerException message was Type System.IO.FileStream with data contract name FileStream: System.IO is not expected.

Introduction

This fix is helpful if you are using WCF streaming and you got this exception while transferring huge file using data contract. If you are using streaming and the stream is transferred to or from WCF service, then you will get an exception like

“There was an error while trying to serialize parameter http://tempuri.org/:fileStream. The InnerException message was ‘Type ‘System.IO.FileStream’ with data contract name ‘FileStream: http://schemas.datacontract.org/2004/07/System.IO’ is not expected. Consider using a DataContractResolver or add any types not known statically to the list of known types - for example, by using the KnownTypeAttribute attribute or by adding them to the list of known types passed to DataContractSerializer.’. Please see InnerException for more details.”

How to fix this exception

Streaming in WCF has some limitations. And the limitation which we face here is that the parameter that we use in contract should be of type IXmlSerializable. Here as we are using data contract, it is not IXmlSerializable. i.e. WCF expects XmlSerializer to serialize and de-serialize messages. Data contract uses DataContractSerializer by default to serialize and de-serialize messages. Unfortunately there is no fix for this bug if you use data contract. You can fix this by using message contract.

Parquet file experiments, findings and recommendations

Parquet is a binary file format designed with big data in mind where we must access data frequently and efficiently. The way it stores file on the disk is also different from other file formats. It is a column-based data file. And in reality it uses both row based and column based approach to bring the best of both worlds. The data is encoded on disk which ensures that the size remains small compared to actual data and is then compressed where the file is scanned as whole and cut out redundant parts. The query/read speed is dramatically fast when compared to other file formats. Nested data is handled efficiently which is quite cumbersome in other file format to achieve. Doesn’t require to parse the entire file to find data due to its way of storing data. This makes it efficient in reading data. Works quite efficiently with data processing frameworks. Automatically stores schema information. SQL querying is possible with this file format using Continue reading

Libish Varghese Jacob

Libish Varghese JacobI am currently working as a lead engineer in one of the leading wind turbine manufacturing firm. I have wide range of interests and getting my hands dirty in technology is one among them. I use this platform primarily as my knowledge base. I also use this platform to share my experience and experiments so that it might help someone who is walking the way I already did. The suggestions expressed here are the best to my knowledge at the time of writing and this may not necessarily be the best possible solution. I would pretty much appreciate if you could comment on it to bring into my notice on what could have been done better.