How to Extract Email Attachments By Pipeline

Tweet

Introduction

In this article I will demonstrate step by step instructions of how to extract email attachments from POP3 adapter by using custom receive pipeline.

Problem Statement

You can have some customers that need to integrate with your integration middle-ware using POP3. So, They will send emails with multiple attachments and these attachments represent the messages that you need to process.

Solution

There are 2 BizTalk solutions that we can implement either in Message Level or Orchestration level. As performance perspective, a message level is much better because orchestration level has much persistence points which can reduce the performance of our integration solution.
If you would like to see the orchestration solution you can check this link.

To process email attachments through message level, then we need to process them using pipeline.
So, we need to implement custom receive pipeline and add it to disassemble part of receive pipeline
For now let's start our walk-though demonstration.

Create Custom Receive Pipeline

1. Create a new solution as shown in figure 1
   
   Figure 1- Creating a new Solution
   
   
2. Create a new class library projects as shown in figure 2
   
   Figure 2- Creating a new Class library project
   
   
3. Remove Class1.cs file
4. Create a new class as shown in figure 3
   
   Figure 3- Creating a new class
   
   
5. Add Microsoft.BizTalk.Pipeline.dll from this path %BTSINSTALLPATH%
6. Add Microsoft.BizTalk.Pipeline.Components.dll as a reference from this path %BTSINSTALLPATH%\Pipeline Components
7. Implement the IBaseComponent, IComponentUI interfaces as shown in figure 4
   
   namespace TechNetWiki.AttachmentsExtractor.PipelineComponent

   {

       using System;

       using System.IO;

       using System.Text;

       using System.Reflection;

       using System.ComponentModel;

       using Microsoft.BizTalk.Message.Interop;

       using Microsoft.BizTalk.Component.Interop;

       using System.Runtime.InteropServices;

     

       [ComponentCategory(CategoryTypes.CATID_PipelineComponent)]

       [ComponentCategory(CategoryTypes.CATID_DisassemblingParser)]

       [Guid("01F04E58-7AD8-42EF-A3CB-D939B960E8F9")]

       public class AttachmentsExtractor : IBaseComponent, IComponentUI

       {

           #region IBaseComponent Members

           /// <summary>

           /// Gets Description of the component

           /// </summary> 

           public string Description

           {

               get

               {

                   return "Email Attachments Extractor";

               }

           }

     

           /// <summary>

           /// Gets Name of the component

           /// </summary> 

           public string Name

           {

               get

               {

                   return "EmailAttachmentsExtractor";

               }

           }

     

           /// <summary>

           /// Gets Version of the component

           /// </summary>

           public string Version

           {

               get

               {

                   return "1.0";

               }

           }

     

           #endregion

     

           #region IComponentUI members

           /// <summary>

           /// Component icon to use in BizTalk Editor

           /// </summary>    

           public IntPtr Icon

           {

               get

               {

                   return System.IntPtr.Zero;

               }

           }

     

           /// <summary>

           /// The Validate method is called by the BizTalk Editor during the build

           /// of a BizTalk project.

           /// </summary>

           /// <param name="obj">An Object containing the configuration properties.</param>

           /// <returns>The IEnumerator enables the caller to enumerate through a collection of strings containing error messages. These error messages appear as compiler error messages. To report successful property validation, the method should return an empty enumerator.</returns>

           public System.Collections.IEnumerator Validate(object obj)

           {

               return null;

           }

           #endregion

       }

   }
   Figure 4-Implementing  IBaseComponent, IComponentUI
   
   
8. Implement IDisassemblerComponent interface which is mandatory to implement GetNext and Disassemble methods 
   • GetNext method returns messages resulting from the disassemble method execution 
   • Disassemble method used to split or process the incoming message document
     
9. When we receive an email we will receive it as a message associated with multi-parts messages so In Disassemble method we need to loop in each Part of the message and create a new message.We will will start looping from index 1 because index zero is the body content which we are not interested as shown in figure 5
   #region IDisassemblerComponent members

           /// <summary>

           /// Returns messages resulting from the disassemble method execution

           /// </summary>

           /// <param name="pc">the pipeline context</param>

           /// <returns></returns>

           public IBaseMessage

               GetNext(IPipelineContext pc)

           {

               // get the next message from the Queue and return it

               IBaseMessage msg = null;

               if ((_msgs.Count > 0))

               {

                   msg = ((IBaseMessage)(_msgs.Dequeue()));

               }

               return msg;

           }

           /// <summary>

           /// called by the messaging engine when a new message arrives

           /// </summary>

           /// <param name="pc">the pipeline context</param>

           /// <param name="inmsg">the actual message</param>

           public void Disassemble(Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)

           {

               var partName = string.Empty;

               // we start from index 1 because index zero contain the body of the message

               // which we are not interested

               for (int i = 1; i < inmsg.PartCount; i++)

               {

                   IBaseMessagePart currentPart = inmsg.GetPartByIndex(i, out partName);

                   Stream currentPartStream = currentPart.GetOriginalDataStream();

                   var ms = new MemoryStream();

                   IBaseMessage outMsg;

                   outMsg = pc.GetMessageFactory().CreateMessage();

                   outMsg.AddPart("Body", pc.GetMessageFactory().CreateMessagePart(), true);

                   outMsg.Context = inmsg.Context;

                   currentPartStream.CopyTo(ms);

                   string attachmentContent = Encoding.UTF8.GetString(ms.ToArray());

                   MemoryStream attachmentStream = new System.IO.MemoryStream(

                   System.Text.Encoding.UTF8.GetBytes(attachmentContent));

                   outMsg.GetPart("Body").Data = attachmentStream;

                   outMsg.Context.Write("ReceivedFileName", "http://schemas.microsoft.com/BizTalk/2003/file-properties", partName);

                   _msgs.Enqueue(outMsg);

               }

           }

           #endregion
   Figure 5- Implementing IDisassemblerComponent interface
   
   the Full code of custom pipeline as show in figure 6
   
   
   namespace TechNetWiki.AttachmentsExtractor.PipelineComponent

   {

       using System;

       using System.IO;

       using System.Text;

       using System.Reflection;

       using System.ComponentModel;

       using Microsoft.BizTalk.Message.Interop;

       using Microsoft.BizTalk.Component.Interop;

       using System.Runtime.InteropServices;

       [ComponentCategory(CategoryTypes.CATID_PipelineComponent)]

       [ComponentCategory(CategoryTypes.CATID_DisassemblingParser)]

       [Guid("01F04E58-7AD8-42EF-A3CB-D939B960E8F9")]

       public class AttachmentsExtractor : IBaseComponent, IComponentUI, IDisassemblerComponent

       {

           #region IBaseComponent Members

           /// <summary>

           /// Gets Description of the component

           /// </summary>  

           public string Description

           {

               get

               {

                   return "Email Attachments Extractor";

               }

           }

           /// <summary>

           /// Gets Name of the component

           /// </summary>  

           public string Name

           {

               get

               {

                   return "EmailAttachmentsExtractor";

               }

           }

           /// <summary>

           /// Gets Version of the component

           /// </summary>

           public string Version

           {

               get

               {

                   return "1.0";

               }

           }

           #endregion

           #region IComponentUI members

           /// <summary>

           /// Component icon to use in BizTalk Editor

           /// </summary>     

           public IntPtr Icon

           {

               get

               {

                   return System.IntPtr.Zero;

               }

           }

           /// <summary>

           /// The Validate method is called by the BizTalk Editor during the build

           /// of a BizTalk project.

           /// </summary>

           /// <param name="obj">An Object containing the configuration properties.</param>

           /// <returns>The IEnumerator enables the caller to enumerate through a collection of strings containing error messages. These error messages appear as compiler error messages. To report successful property validation, the method should return an empty enumerator.</returns>

           public System.Collections.IEnumerator Validate(object obj)

           {

               return null;

           }

           #endregion

           /// <summary>

           /// this variable will contain any message generated by the Disassemble method

           /// </summary>

           private System.Collections.Queue _msgs = new System.Collections.Queue();

           #region IDisassemblerComponent members

           /// <summary>

           /// Returns messages resulting from the disassemble method execution

           /// </summary>

           /// <param name="pc">the pipeline context</param>

           /// <returns></returns>

           public IBaseMessage

               GetNext(IPipelineContext pc)

           {

               // get the next message from the Queue and return it

               IBaseMessage msg = null;

               if ((_msgs.Count > 0))

               {

                   msg = ((IBaseMessage)(_msgs.Dequeue()));

               }

               return msg;

           }

           /// <summary>

           /// called by the messaging engine when a new message arrives

           /// </summary>

           /// <param name="pc">the pipeline context</param>

           /// <param name="inmsg">the actual message</param>

           public void Disassemble(Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)

           {

               var partName = string.Empty;

               // we start from index 1 because index zero contains the body of the message

               // which we are not interested

               for (int i = 1; i < inmsg.PartCount; i++)

               {

                   IBaseMessagePart currentPart = inmsg.GetPartByIndex(i, out partName);

                   Stream currentPartStream = currentPart.GetOriginalDataStream();

                   var ms = new MemoryStream();

                   IBaseMessage outMsg;

                   outMsg = pc.GetMessageFactory().CreateMessage();

                   outMsg.AddPart("Body", pc.GetMessageFactory().CreateMessagePart(), true);

                   outMsg.Context = inmsg.Context;

                   currentPartStream.CopyTo(ms);

                   string attachmentContent = Encoding.UTF8.GetString(ms.ToArray());

                   MemoryStream attachmentStream = new System.IO.MemoryStream(

                   System.Text.Encoding.UTF8.GetBytes(attachmentContent));

                   outMsg.GetPart("Body").Data = attachmentStream;

                   outMsg.Context.Write("ReceivedFileName", "http://schemas.microsoft.com/BizTalk/2003/file-properties", partName);

                   _msgs.Enqueue(outMsg);

               }

           }

           #endregion

       }

   }
   Figure 6- Custom pipeline full code
   
   
   
10. Build the project then copy the generated TechNetWiki.AttachmentsExtractor.PipelineComponent.dll to %BTSINSTALLPATH%\Pipeline Components

Create Receive Pipeline

1. Create a new BizTalk Project by right click solution and select new project then select Empty BizTalk Server Project and name it TechNetWiki.AttachmentsExtractor.Pipelines as shown in figure 7
   
   Figure 7- Create Pipeline BizTalk Project
   
   
2. Right click TechNetWiki.AttachmentsExtractor.Pipelines project -> Select Add-> New Item -> Select Receive Pipeline-> Name it AttachmentsExtractorPipeline as shown in figure 8 
   
   Figure 8- Create a new receive pipeline
   
   
3. Right click to ToolBox window ->select Choose Items... -> select BizTalk Pipeline Components -> check EmailAttachmentsExtractor as shown in figure 9
   
   Figure 9- Add pipeline component to ToolBox
   
   
4. Drag drop EmailAttachmentsExtractor to Disassemble part as shown in figure 10 
   
   Figure 10- Setting pipeline component in Disassemble part
   
   
5. Right click project TechNetWiki.AttachmentsExtractor.Pipelines -> select Properties -> select Sign tab-> check Sign the assembly -> choose a strong name key file -> select New and rename it to key.snk -> select Deployment tab -> set Application Name to TechNetWiki.AttachmentsExtractor
6. Right click TechNetWiki.AttachmentsExtractor.Pipelines -> select Deploy

Configure POP3 Adapter

1. Open BizTalk Server Administration Console 
2. Right click TechNetWiki.AttachmentsExtractor - Receive Port -> select New -> One-way Receive Port... -> change name to Rcv_EmailTest -> select Receive Location tab -> change name to RcvLoc_EmailTest
3. Select POP3 adapter-> click Configure button -> set POP3 adapter configure as shown in figure 11 In this demo I am using my hotmail account to configure POP3 adapter
   
   Figure 11 - Configuring POP3 adapter
   
   
4. Create a new send port and name it Snd_EmailTest -> select File Adapter and configure it as shown in figure 12
   figure 12- Creating send port
   
   
5. Select Filters tab then set filter to our receive port name as shown in figure 13
   
   Figure 13- Setting filter

Test the Solution

1. Send email with sample attachment message to the configured email that we configured on POP3 adapter which is my demo YouUserName@hotmail.com
2. Check send port location

Sample Code

You can find the sample source code in the following link

Conclusion

In BizTalk Server there are two ways to extract email attachments either using custom receive pipeline or orchestration

In this article, I walked-through in step by step how to create a custom receive pipeline to extract email attachments.

See Also

Read suggested related topics:

• Splitting POP3 Email Attachments using orchestration
• MSDN Pop3 Adapter documentation

Another important place to find a huge amount of BizTalk related articles is the TechNet Wiki itself. The best entry point is BizTalk Server Resources on the TechNet Wiki.

---